S3 & Glacier

S3¶

General Info¶

Object storage

Features

Feature	Description
Regionality	Buckets are global (no region or account ID in their ARN) Automatically replicated within a region
Data Consistency	`PUT` to new object = `read-after-write` consistency `PUT` to existing object = `eventual` consistency `DELETE` object = `eventual` consistency
MFA Delete	Applies to deleting objects, not buckets Only the `root` account can enable MFA Delete
Versioning	The bucket owner, `root` account, and all authorized IAM users of a bucket are allowed to enable versioning Once enabled, it is not possible to disable or turn off versioning (it can only be suspended)
S3 Transfer Acceleration	Speeds up transfers to S3
Limits	S3 Per-prefix performance PUT/POST/COPY/DELETE: `3500/sec` GET/HEAD: `5500/sec` S3 Latency performance: `100-200ms` for first-byte-out

Naming

Use Case	Naming	Example
Direct bucket access	bucket name after FQDN (path-style)	https://s3.us-east-2.amazonaws.com/testBucket/images
Website hosting	bucket name part of FQDN (virtual-hosted-style)	https://testBucket.s3.us-east-2.amazonaws.com/images

Storage Classes¶

Class	Description	Availability	Durability
S3 Standard	Frequently accessed data	`99.99`	11 9s
S3 Intelligent-Tiering	Transitions objects between S3 Standard and S3-IA, depending on how often an object is accessed Monitoring and automation charges	`99.99`	11 9s
S3 Standard-IA	Infrequently accessed data (same performance) Stored for at least 30days Charges all objects as if they are at least 128 KB in size	`99.9`	11 9s
S3 One Zone-IA	Reduced Redundancy Storage Lower durability for derived data that can be replaced	`99.5`	11 9s
Glacier	Long term archives Retrieve time of several hours (3-5) Issue `RESTORE` command → 5h later the object is copied into S3 RRS (original remains in Glacier)	`99.99`	11 9s
Glacier Deep Archive	12-48 hours latency	`99.99`	11 9s

Access Control¶

The account (not user) that creates objects/buckets owns them, even if the bucket is hosted by a different account

Contexts

Context	Description
User needs to have permission	Using identity policies (or user is the root of an account)
For bucket operations: bucket needs to have permission	User in a different account → just bucket policy/ACL User in the same account → both bucket policy/ACL and identity policy
For object operations: User has to have permission (or be root)	Bucket policy/ACL has to not deny Object ACL (or bucket policy) has to allow

Grant Access

Option	Description
IAM Policies	Can only grant users within your AWS account
S3 ACLs	Legacy: bucket and object resource-based policy Default ACL grants the owner account full control List of grants, each grant gives a grantee (an AWS account or predefined group, NOT specific users) a permission Grantee groups: `Authenticated Users` = any AWS user `All Users` = include `anonymous` `Log Delivery` = S3 audit logs Permissions: `READ`, `WRITE` (only applies to buckets - allows overwriting and deleting objects), `FULL CONTROL` (all of the above)
S3 Bucket policies	Recommended: Bucket resource-based policy Add/Deny permissions across some or all objects in one bucket Resource needs to end in "`/`" Policies can be attached to: users, groups, S3 buckets Differences with IAM: Associated with bucket, no IAM principal Explicit reference to IAM principal, which can belong to different account*
Query String Auth	Can use query params to provide request info, including auth info This part of URL (named PRE-SIGNED URL) share objects through URLs
Conditions	Specified by policy keys Options Request time (DATE Condition) If over SSL (BOOLEAN Condition) Client IP (IP Condition) Client app (STRING Condition)

Policy Precedence (IAM Policy / S3 Bucket Policy / S3 ACL)

RulesExamples

Decisions ALWAYS default to DENY
- If no method specifies an ALLOW —> DENY by default
An EXPLICIT DENY ALWAYS override an ALLOW
Only if no method specify a DENY && 1+ methods specify an ALLOW —> ALLOW
S3 bucket policies has precedence over IAM policy

Example
- IAM policy grants access to an object
- S3 bucket policy denies access to that object
- No S3 ACL
- —> access denied (S3 bucket policy win)
Example
- 1 bucket policy deny all ops to all users (EXPLICIT DENY)
- 1 bucket policy allowing one specific user to upload
- -> EXPLICIT DENY ALWAYS OVERRIDE AN ALLOW

Encryption¶

Server Side


`SSE-S3`	AWS S3-managed Keys	AWS handles key management/protection Each object is encrypted with unique key Object key encrypted with master key
`SSE-KMS`	AWS KMS-managed Keys	AWS handles key management/protection but you manage the keys Separate permissions for using the master key Provides auditing
`SSE-C`	Customer-provided Keys	AWS only performs encryption/decryption of the object You send the plaintext encryption key in the request

Client Side


`CSE-KMS`	AWS KMS-managed Customer Master Key
`CSE-C`	Client-side Master Key

Monitoring¶

CloudTrail by default records bucket-level actions
Can enable CloudTrail logging of object-level actions by setting that property on a bucket in S3 (can choose read/write)
Server access logging: separate audit log, configured per-bucket, that stores events in a bucket

Specific Setups¶

Force TLS

Add bucket policy

"Condition": {
    "Bool": {
        { "aws:SecureTransport": false } }}

Pre-signed URLs

Give access (temporarily) to a specific object since by default buckets are private

$ aws s3 presign s3://<bucket>/file.txt —expires-in 300 # 300 seconds (default is 1h)

Block Public Access

Applied to specific buckets, or all buckets in an account


`BlockPublicAcls`	Can't create new public bucket or object ACLs
`IgnorePublicAcls`	Existing (and new) public ACLs are ignored
`BlockPublicPolicy`	Can't create public bucket policies
`RestrictPublicBuckets`	Blocks all anonymous and cross-account access to a bucket

Cross region replication

Features
- Uses SSL by default (no need for policy)
- You can replicate objects from a source bucket to only one destination
- After S3 replicates an object, the object cannot be replicated again
- Enabling cross-region replication does not copy existing objects by default → need to manually copy existing objects
Conditions
- Object owner must grant the bucket owner the READ and READ_ACP permissions via the object ACL
- The owner of the destination bucket must grant the owner of the source bucket permissions to replicate objects with a bucket policy
- Versioning must be enabled
What replicated
- Unencrypted objects
- Objects encrypted using SSE-S3 and SSE-KMS (no SSE-C)
- Object metadata
- Any object ACL updates
- Any object tags
- Only object in the source bucket for which the bucket owner has permission to read objects and read ACLs
- NOT REPLICATED deletes to a particular VERSION of an object (security mechanism)

Glacier¶

Use cases

Files stored as Archives, Archives stored in Vaults
Data Retrieval requires job initiation then getting the output from the job

Characteristics

Data Storage

Automatically encrypts data (AES-256)
Regular data integrity checks
Only your account can access data
Can use IAM to specify which users within the account have rights to operate on a given Vault

Archive

Stores data <40TB
Each assigned a unique ID
Automatically encrypted
Immutable (after creation can't be modified)

Vault

Container for Archives (can store infinite number)
Max 1000 Vaults per region

Vault Lock Policies

A vault access policy that can be locked to prevent changes to it
- configure and enforce compliance controls for individual Glacier Vaults (~ IAM policy)
Use cases
- configure WORM (write once read many) archives
- create data retention policies
2 steps to configure
- initiate the lock by attaching a vault lock policy to the vault (sets the lock to an "in-progress" state)
- you then have 24h to validate the lock policy
- once validated, lock policies are immutable