S3 Basics

S3 – The Basics

S3 provides developers and IT teams with secure, durable, highly-scalable objects storage.

Amazon S3 is easy to use, with a simple web services interface to store and retrieve any amount of data from anywhere on the web.

Ques: What is S3?

Answer: 

• S3 is a safe place to store your files.
• It is Object-based storage.
• The data is spread across multiple devices and facilities.
• S3 is Object based- i.e. allows you to upload files.
• Files can be from 0 Bytes to 5 TB.
• There is unlimited storage.
• Files are stored in Buckets (Folders).
• Built for 99.99% availability for the S3 platform.
• Amazon Guarantee 99.9% availability.
• Amazon guarantees 99.999999999 % durability for S3 information. (Remember 11 * 9s).
• S3 is a universal namespace. That is, names must be unique globally. 

For example: https://<username>.s3.amazonaws.com/

When you upload a file to S3, you will receive a HTTP 200 code if the upload was successful.

S3 is object based. Think of Objects just as files. Objects consist of the following:

Key (This is simply the name of the object)

Value (This is simply the data and is made up of a sequence of bytes).

Version ID (Important for versioning)

Metadata (Data about data you are storing)

Subresources;

Access Control Lists

Torrent 

Ques: How does data consistency work for S3?

Answer: 

• Read after Write consistency for PUTS of new Objects.

Means: If you write a new file and read it immediately afterwards, you will be able to view that data.

• Eventual Consistency for overwrite PUTS and DELETES (can take some time to propagate)

Means: If you update AN EXISTING file or delete a file and read it immediately, you may get the older version, or you may not. Basically, changes to objects can take a little bit of time to propagate.

S3- Features

• Tiered Storage Available

• Lifecycle Management

• Versioning

• Encryption

• MFA Delete

• Secure your data using Access Control Lists and Bucket Policies.

S3 Storage Classes

1. S3 Standard: 99.99% availability, 99.999999999 % durability, stored redundantly across multiple devices in multiple facilities and is designed to sustain the loss of 2 facilities concurrently.

2. S3 – IA: (Infrequently Accessed): For data that is accessed less frequently, but requires rapid access when needed. Lower fee than S3, but you are charged a retrieval fee.

3. S3 – One Zone – IA: For where you want a lower-cost option for infrequently accessed data, but do not require the multiple Availability Zone data resilience

4. S3 – Intelligent Tiering – Designed to optimize costs by automatically moving data to the most cost-effective access tier, without performance impact or operational overhead.

5. S3 Glacier: S3 Glacier is a secure, durable, and low-cost storage class for data archiving. Retrieval times configurable from minutes to hours.

6. S3 Glacier Deep Achieve: S3 Glacier Deep Archive is Amazon S3’s lowest-cost storage where a retrieval time of 12 hours is acceptable.

7. Reduced redundancy: Frequently accessed, non-critical data.

S3 – Charges depends on

• Storage

• Requests

• Storage Management Pricing

• Data Transfer Pricing

• Transfer Acceleration

• Cross Region Replication Pricing

S3 Transfer Acceleration

Amazon S3 Transfer Acceleration enables fast, easy, and Secure transfers of files over long distances between your end users and an s3 bucket.

Transfer Acceleration takes advantage of Amazon CloudFront’s globally distributed edge locations. As the data arrives at an edge location, data is routed to Amazon S3 over an optimized network path.

Note: Bucket names share a common name space. We can’t have same bucket name twice globally. You can replicate the contents of one bucket to another bucket automatically by using cross region replication. You can change storage classes and encryption of your objects on the fly.

Restricting S3 Bucket Access

• Bucket Policies – Applies across the whole bucket
• Object Policies – Applies to individual files
• IAM Policies to Users & Groups – Applies to Users & Groups

By Default, all newly created buckets are PRIVATE. You can setup access control to your buckets using

Bucket Policies

Access Control Lists

S3 buckets can be configured to create access logs which log all requests made to the S3 bucket. This can be sent to another bucket and even another bucket in another account.

Encryption In Transit is achieved by 

1. SSL/TLS

Encryption At Rest (Server Side) is achieved by

• S3 Managed Keys – SSE-S3

• AWS KEY MANAGEMENT SERVICE, MANAGED KEYS-SSE-KMS

• SERVER SIDE ENCRYOTION WITH CUSTOMER PROVIDED KEYS – SSE-C

    2. CLIENT SIDE ENCRYPTION

We can encrypt individual objects and we can also encrypt bucket level which is much more efficient.

Versioning with S3:

• Stores all versions of an object (including all writes and even if you delete an object)

• Great backup tool

• Once enabled, Versioning cannot be disabled, only suspended.

• Integrates with Lifecycle rules.

• Versioning’s MFA Delete capability, which uses multi-factor authentication, can be used to provide an additional layer of security.

S3 LifeCycle Rule

• Automates moving your objects between the different storage tiers.

• Can be used in conjunction with versioning.

• Can be applied to current versions and previous versions.