The technical content of this article relies largely or entirely on documentation from Amazon.com.(November 2018) |
Amazon Simple Storage Service(S3) is a service offered byAmazon Web Services(AWS) that providesobject storagethrough aweb serviceinterface.[1][2]Amazon S3 uses the same scalable storage infrastructure thatAmazon.comuses to run its e-commerce network.[3]Amazon S3 can store any type of object, which allows uses like storage for Internet applications, backups, disaster recovery, data archives,data lakesfor analytics, andhybrid cloud storage.AWS launched Amazon S3 in the United States on March 14, 2006,[1][4]then in Europe in November 2007.[5]
![]() | |
Type of site | Cloud storage |
---|---|
Available in | English |
Owner | Amazon.com |
URL | aws |
IPv6support | Yes |
Commercial | Yes |
Registration | Required (included in free tier layer) |
Launched | March 14, 2006 |
Current status | Active |
Technical details
editDesign
editAmazon S3 manages data with anobject storagearchitecture[6]which aims to providescalability,high availability,andlow latencywith highdurability.[3]The basic storage units of Amazon S3 are objects which are organized into buckets. Each object is identified by a unique, user-assigned key.[7]Buckets can be managed using the console provided by Amazon S3, programmatically with the AWSSDK,or theRESTapplication programming interface. Objects can be up to fiveterabytesin size.[8][9]Requests are authorized using anaccess control listassociated with each object bucket and supportversioning[10]which is disabled by default.[11]Since buckets are typically the size of an entire file system mount in other systems, this access control scheme is very coarse-grained. In other words, unique access controls cannot be associated with individual files.[citation needed]Amazon S3 can be used to replace staticweb-hostinginfrastructure with HTTP client-accessible objects,[12]index document support, and error document support.[13] The Amazon AWS authentication mechanism allows the creation of authenticated URLs, valid for a specified amount of time. Every item in a bucket can also be served as aBitTorrentfeed. The Amazon S3 store can act as a seed host for atorrentand any BitTorrent client can retrieve the file. This can drastically reduce the bandwidth cost for the download of popular objects. A bucket can be configured to save HTTP log information to a sibling bucket; this can be used indata miningoperations.[14]There are variousUser Mode File System (FUSE)–based file systems for Unix-like operating systems (for example,Linux) that can be used to mount an S3 bucket as a file system. The semantics of the Amazon S3 file system are not that of aPOSIXfile system, so the file system may not behave entirely as expected.[15]
Amazon S3 storage classes
editAmazon S3 offers nine different storage classes with different levels of durability, availability, and performance requirements.[16]
- Amazon S3 Standard is the default. It is general purpose storage for frequently accessed data.
- Amazon S3 Express One Zone is a single-digit millisecond latency storage for frequently accessed data and latency-sensitive applications. It stores data only in one availability zone.[17]
- Amazon S3 Standard-Infrequent Access (Standard-IA) is designed for less frequently accessed data, such as backups and disaster recovery data.
- Amazon S3 One Zone-Infrequent Access (One Zone-IA) performs like the Standard-IA, but stores data only in one availability zone.
- Amazon S3 Intelligent-Tiering moves objects automatically to a more cost-efficient storage class.
- Amazon S3 on Outposts brings storage to installations not hosted by Amazon.
- Amazon S3 Glacier Instant Retrieval is a low-cost storage for rarely accessed data, but which still requires rapid retrieval.
- Amazon S3 Glacier Flexible Retrieval is also a low-cost option for long-lived data; it offers 3 retrieval speeds, ranging from minutes to hours.
- Amazon S3 Glacier Deep Archive is the lowest cost storage for long-lived archive data that is accessed less than once per year and is retrieved asynchronously.
The Amazon S3 Glacier storage classes above are distinct fromAmazon Glacier,which is a separate product with its own APIs.
File size limits
editAn object in S3 can be between 0 bytes and 5TB. If an object is larger than 5TB, it must be divided into chunks prior to uploading. When uploading, Amazon S3 allows a maximum of 5GB in a single upload operation; hence, objects larger than 5GB must be uploaded via the S3 multipart upload API.[18]
Uses
editNotable users
edit- Photo hosting serviceSmugMughas used Amazon S3 since April 2006. They experienced a number of initial outages and slowdowns, but after one year they described it as being "considerably more reliable than our own internal storage" and claimed to have saved almost $1 million in storage costs.[19]
- Netflixuses Amazon S3 as theirsystem of record.Netflix implemented a tool, S3mper,[20]to address the Amazon S3 limitations ofeventual consistency.[21]S3mper stores the filesystem metadata: filenames, directory structure, and permissions inAmazon DynamoDB.[22]
- Redditis hosted on Amazon S3.[23]
- Bitcasa,[24]andTahoe-LAFS-on-S3,[25]among others, use Amazon S3 for online backup and synchronization services. In 2016, Dropbox stopped using Amazon S3 services and developed its own cloud server.[26][27]
- Swiftype's CEO has mentioned that the company uses Amazon S3.[28]
S3 API and competing services
editThe broad adoption of Amazon S3 and related tooling has given rise to competing services based on the S3API.These services use the standard programming interface but are differentiated by their underlying technologies and business models.[29]A standard interface enables better competition from rival providers and allowseconomies of scalein implementation, among other benefits.[30]
History
editAmazon Web Services introduced Amazon S3 in 2006.[31][32]
Date | Number of Items Stored |
---|---|
October 2007 | 10 billion[33] |
January 2008 | 14 billion[33] |
October 2008 | 29 billion[34] |
March 2009 | 52 billion[35] |
August 2009 | 64 billion[36] |
March 2010 | 102 billion[37] |
April 2013 | 2 trillion[38] |
March 2021 | 100 trillion[39] |
March 2023 | 280 trillion[40] |
In November 2017 AWS added default encryption capabilities at bucket level.[41]
See also
editReferences
editCitations
edit- ^ab"Amazon Web Services Launches" Amazon S3 ""(Press release). 2006-03-14.Archivedfrom the original on 2018-11-15.Retrieved2018-11-14.
- ^Huang, Dijiang; Wu, Huijun (2017-09-08).Mobile Cloud Computing: Foundations and Service Models.Morgan Kaufmann. p. 67.ISBN9780128096444.Archivedfrom the original on 2018-11-15.Retrieved2018-11-15.
- ^ab"Cloud Object Storage - Store & Retrieve Data Anywhere - Amazon Simple Storage Service".Amazon Web Services, Inc.Archivedfrom the original on 2018-05-17.Retrieved2018-05-17.
- ^"5 Key Events in the history of Cloud Computing - DZone Cloud".dzone.com.Archivedfrom the original on 2018-09-29.Retrieved2018-09-28.
- ^"Amazon Web Services Offers European Storage for Amazon S3"(Press release). 2007-11-06.Archivedfrom the original on 2018-11-15.Retrieved2018-11-14.
- ^"What is Cloud Object Storage? – AWS".Amazon Web Services, Inc.2019-10-16.Archivedfrom the original on 2018-09-20.Retrieved2018-07-09.
- ^"Tech Blog » Starting Websphere in Cloud and saving the data in S3".techblog.aasisvinayak.com.Archived fromthe originalon 2010-03-12.
- ^"open-guides/og-aws".GitHub.Archivedfrom the original on 2018-01-03.Retrieved2018-05-17.
- ^"Error Responses - Amazon Simple Storage Service".docs.aws.amazon.com.Archivedfrom the original on 2017-12-24.Retrieved2018-05-21.
- ^"Using versioning in S3 buckets - Amazon Simple Storage Service".
- ^"Introduction to Amazon S3 - Amazon Simple Storage Service".docs.aws.amazon.com.Archivedfrom the original on 2018-05-12.Retrieved2018-05-17.
- ^"How to use Amazon S3 for Web Hosting".bucketexplorer.com.Archived fromthe originalon 2008-04-08.Retrieved2008-05-06.
- ^Amazon Simple Storage ServiceArchived2011-02-20 at theWayback MachineDocs.amazonwebservices.com. Retrieved on 2013-08-09.
- ^http://docs.aws.amazon.com/AmazonS3/latest/dev/ServerLogs.htmlArchived2014-12-23 at theWayback MachineServer Access Logging
- ^"Comparison of S3QL and other S3 file systems".Archivedfrom the original on 2012-08-05.Retrieved2012-06-29.
- ^"Cloud Storage Classes – Amazon Simple Storage Service (S3) – AWS".Amazon Web Services, Inc.Archivedfrom the original on 2018-06-13.Retrieved2018-05-17.
- ^"Announcing the new Amazon S3 Express One Zone high performance storage class | AWS News Blog".aws.amazon.com.2023-11-28.Retrieved2023-12-01.
- ^"How to Upload Large Files to S3".June 21, 2022. Archived fromthe originalon October 1, 2022.RetrievedJune 22,2022.
- ^"Amazon S3: Show Me the Money".SmugMug Blog.SmugMug. November 10, 2006.Archivedfrom the original on 2017-03-03.Retrieved2017-03-03.
- ^"S3mper: Consistency in the Cloud".Archivedfrom the original on 2016-04-24.Retrieved2016-05-01.
- ^"Introduction to Amazon S3".Amazon.Archivedfrom the original on 2017-12-25.Retrieved28 December2017.
- ^Hern, Alex (2017-02-02)."Amazon Web Services: the secret to the online retailer's future success".the Guardian.Archivedfrom the original on 2018-05-02.Retrieved2018-04-23.
- ^"AWS Case Study: reddit".aws.amazon.com.2015.Archivedfrom the original on 2015-03-17.RetrievedMarch 18,2015.
- ^"Bitcasa Legal".May 16, 2013. Archived fromthe originalon 2013-06-28.Retrieved2013-05-16.
- ^"What is Tahoe-LAFS-on-S3?".August 21, 2012.Archivedfrom the original on 2013-05-06.Retrieved2012-08-21.
- ^"The Epic Story of Dropbox's Exodus From the Amazon Cloud Empire".WIRED.Archivedfrom the original on 2018-01-25.Retrieved2018-04-23.
- ^"Dropbox saved almost $75 million over two years by building its own tech infrastructure".GeekWire.2018-02-23.Archivedfrom the original on 2018-04-23.Retrieved2018-04-23.
- ^"Swiftype Explains Their Cloud Stack".July 1, 2013.Archivedfrom the original on 2014-12-08.Retrieved2014-12-08.
- ^Watters, Audrey (12 July 2010)."Cloud Community Debates, Is Amazon S3's API the Standard? (And Should It Be?)".SAY Media, Inc.Archivedfrom the original on 2013-02-17.Retrieved19 December2012.
- ^Crossroads of Information Technology Standards.Committee on Standards Workshop Planning, Board on Telecommunications and Computer Applications, Commission on Engineering and Technical Systems, National Research Council. Washington, DC: The National Academies Press, 1990. 1990. pp. 36–37.doi:10.17226/10440.ISBN978-0-309-58171-4.Archivedfrom the original on 2014-03-25.Retrieved2014-03-25.
{{cite book}}
:CS1 maint: others (link) - ^Overview of Amazon Web Services, 2018,https://docs.aws.amazon.com/whitepapers/latest/aws-overview/introduction.htmlArchived2017-11-18 at theWayback Machine
- ^Garfinkel, Simson L. 2007. An Evaluation of Amazon's Grid Computing Services: EC2, S3, and SQS. Harvard Computer Science Group Technical Report TR-08-07.https://dash.harvard.edu/bitstream/handle/1/24829568/tr-08-07.pdf?sequence=1Archived2018-07-29 at theWayback Machine
- ^abVogels, Werner (2008-03-19)."Happy Birthday, Amazon S3!".All Things Distributed.Archivedfrom the original on 2008-05-09.Retrieved2008-05-23.
- ^"Amazon S3 - Busier Than Ever".2008-10-08. Archived fromthe originalon 2008-10-11.Retrieved2008-10-09.
- ^"Celebrating S3's Third Birthday With Special Anniversary Pricing - Amazon Web Services".typepad.com.31 March 2009.Archivedfrom the original on 2011-07-07.Retrieved2009-04-01.
- ^"Amazon's Head Start in the Cloud Pays Off".eweek.com.[permanent dead link]
- ^"Amazon S3 Now Hosts 100 Billion Objects".datacenterknowledge.com.9 March 2010.Archivedfrom the original on 2010-03-12.Retrieved2010-03-09.
- ^"Amazon S3 – Two Trillion Objects, 1.1 Million Requests / Second - Amazon Web Services".typepad.com.18 April 2013.
- ^"Celebrate 15 Years of Amazon S3 with 'Pi Week' Livestream Events".amazon.com.14 March 2021.
- ^"Celebrate Amazon S3's 17th birthday at AWS Pi Day 2023".amazon.com.14 March 2023.
- ^"New Amazon S3 Encryption & Security Features".6 November 2017.
Sources
edit- "Server Access Logging".
- "Amazon S3 Developer Guide".2006-03-01.
- "Amazon S3 Introduces Storage Pricing Tiers".2008-10-08.
- "RightScale Ruby library to access Amazon CloudFront, EC2, S3, SQS, and SDB".2007-10-27. Archived fromthe originalon 2008-11-03.Retrieved2009-01-07.