AWS Security Operations Object Storage
About Me
 |
 |
 |
DevOps@RajeshKumar.XYZ
AWS Security Operations: Securing Access to S3 Object Storage
Why have a course focused on AWS security?
Security is still 25 / 7 in the cloud
On the Internet, attack is easier than defense
This course will help you with all things in securing AWS S3 Storage
A working knowledge of AWS services is a bonus but not essential
AWS Security Operations
- Part 1: Designing for Acceptable Risk and Compliance
- Part 2: Securing Core AWS Infrastructure Services
- Part 3: Securing Access to Object Storage
Topics
- The Simple Storage Service
- S3 and Regulated Workloads
- S3 Resource Control–Public vs. Private
- S3 Data Protection
- S3 Versioning
- Lifecycle Management
- Glacier Storage
- Importing Data into S3
- Best Practices When Planning S3
Simple Storage Service
S + S + S = S3
Storage Review
S3 Storage Concepts
Redundantly stored
- On multiple devices
- Across multiple facilities
- Within the selected region
Highly scalable, cloud storage
- Store and retrieve any amount of data
S3 buckets store data in selected region
- Technically defined as "Object Storage"
Shared Responsibility Model
- Customers manage their data
- AWS manages the infrastructure layer
S3 Design
S3 Design
Data never leaves region
- Unless you move it cross-region
- Data is stored as objects
- Objects can be any file type
- Objects are assigned a object key
- The object key linked to each object is unique
- Metadata can also be stored with each object
Permissions can be set several ways:
- On each object
- On each bucket
- IAM policy or bucket policy
S3 Security
S3 Security Features
- Access Logging
- Event Notifications
- CloudTrail Audit
- CORS Configuration
- Versioning
- CloudWatch S3 Metrics
Managed Services Integration with S3
S3 Durability
S3 Durability
Choice of Storage Classes for S3
Amazon S3 can be used with PCI DSS, HIPAA, ITAR, SOC3, and ISO 27001 standards.
S3 Storage Setup
S3 Resource Control
Accessing S3 Data
S3 storage starts as "private" by default
- Buckets, objects, and configuration
- The resource owner can always access
Two methods to share data:
- Objects can be made Public
- Policies for selective access control
Resource policies
- Bucket policies –groups (of users)
- Add or deny permissions
- ACLs –read or write access to objects to groups of users, not specific users
User policies
- IAM policies for fine grained control for users within your AWS account
S3 Object Tags
S3 Bucket Policy
S3 Data Protection
S3 Encryption Choices
Client side encryption is performed by the client directly to the selected file
Server Side Encryption
- Data is encrypted during the writing to disk process
- Data is decrypted during the reading process
- Encryption is performed at the object level
- SSE encrypts the object data but not the associated metadata
- Unencrypted metadata could be a compliance issue
Protecting Data in Transit
Data moving to and from AWS S3
- Use SSL endpoints over HTTPS
- Secure upload and download of data
- Client side encryption before upload
Using VPC endpoints
- Connect to a specific VPC endpoint
- Use bucket policies for multiple endpoints
Protecting Data at Rest
Server side encryption
- AWS does the encryption and decryption
- Key management; encrypted by AWS S3 master key
- Stored separately from your data
- AES 256-bit encryption
- Optional user provided keys
- Integrated with AWS KMS
Client side encryption
- Choice depends on OSand storage
Server Side Encryption Steps
AWS S3 Server-side Encryption
Customer Provided Encryption Key (SSE-C)
Managing S3 Encryption
S3 Versioning
S3 Versioning
Versioning allows you to keep multiple versions of the same object, in the same bucket
Preserve, retrieve, or restore every version of every object
Versioning is enabled at the bucket level
- Disabled by default
- Can be disabled, but not deleted
- Versioning operations managed by AWS
Create lifecycle policies to control dataflow
- Archival and expiration dates
- Integrated with AWS glacier
Overriding an object result in a new object being created, with a new version number attached
Versioning applies to all objects in the bucket
Glacier Storage
Glacier Storage
- Low-cost Storage
- Backup and Archiving
- Files are archived in Vaults
- Vaults are stored in Regions
- "Data at Rest" AES-256
- "Data in Transit" SSL end-points
Amazon Glacier Vault Lock
Compliance controls for matching your compliance objectives
- Ex: SEC Rule 17a-4(f)
Create vault lock policies
- Ex: WORM
- Deny deletes
Contents cannot be modified once locked with a vault policy
Vault access policy = Access
Vault lock policy = Content
Amazon Glacier supports PCI DSS, HIPAA, ITAR, SOC3, and ISO 27001 standards
Versioning and Lifecycle Policies
Monitoring S3 and Glacier
Monitor Your Storage Usage
CloudWatch S3 Metrics
Operational and performance monitoring
- Metrics for your choice of data
- Bucket, vaults, tags, and prefixes
- Ex: Changes to lifecycle policies
Monitor S3 with CloudWatch
- Total bytes for Standard storage
- Total bytes for Reduced Redundancy storage
Monitor Glacier with CloudWatch
- Who made the request?
- IP address where request was made from
Alarm metrics
- Set thresholds for alarms
- One minute CloudWatch metrics; and longer
S3 Data Events in CloudTrail
S3 Monitoring Options
Importing Data into S3
S3 Data Transfer Options
- AWS Direct Connect
- AWS Snowball @ Snowball Edge
- Amazon Kinesis Firehouse
- S3 Transfer Acceleration
- AWS Snowmobile
- AWS Storage Gateway
"File Gateway" (Hybrid Storage Solution)
Best Practices Planning S3 Storage
Enable Versioning
Protect from accidental deletion
- New version created from each upload
- Easy retrieval of "deleted objects"
Reminder:
- The default –no versioning
- Once versioning is enabled
- Versioning can only be suspended
Enable MFA Bucket Protection
Enabling MFA requires:
- Provide your AWS account's access key
- A valid MFA code
- Changing the versioning state of your bucket
- Permanently deleting an object version (MFA Delete)
Protect Data and Data Cost with Lifecycle Policies
Data Cost with Tiering
- Transition –Standard to Reduced Redundancy, or Amazon Glacier; after defined timeframe
- Expiration: Delete objects after defined period of time
Data Protection with Security Controls
- Policies can also be set at the prefix, or object tag level
- Ex: Key Name Prefix: logs/
Versioning and Lifecycle Policies
Consider Cross–region Replication
Control: Replicate all deleted objects to destination bucket in separate region
- Both source and destination buckets need to enable versioning
- Uploads into source bucket will then be replicated
Compliance: Records are stored hundreds of miles apart
Secure: Remote replicas managed by separate AWS accounts
Faster: Lower latency
Parallel PUT Uploads
Increase your aggregate throughput by parallelizing PUTs on fast networks
- Upload multiple parts of large objects
- Removes bottleneck from the upload process
- Helps to reduce network connectivity issues
Parallelize GETS
TransferManager provides multi-threaded download performance
- Useful for larger objects
- Upgrade SDK for Java to 1.11.0 or later
- If the object is determined to be multipart, download is performed in parallel
IAM Roles
Use IAM roles for S3 control
- IAM policies for users, groups, and roles
- Access control is kept at the IAM level
- Central permission management
"What can this user do in AWS?"
Bucket Policies
S3 Bucket policies are directly attached to the S3 bucket
- Simple method to grant cross-account policies
- Access control in the S3 environment
"Who can access this bucket?"
Note: S3 ACLs control individual objects within a bucket at the user level
Wrap Up
- The Simple Storage Service
- S3 and Regulated Workloads
- S3 Resource Control–Public vs. Private
- S3 Data Protection
- S3 Versioning
- Lifecycle Management
- Glacier Storage
- Importing Data into S3
- Best Practices When Planning S3
Up Next:
Securing Content Access with CloudFront
Securing Content Access with CloudFront
Why have a course focused on AWS security?
Security is still 25 / 7 in the cloud
43% of C-level executives say negligent insiders are the greatest threat to sensitive data
This course will help you with all things in securing AWS content access
A working knowledge of AWS services is a bonus but not essential
Topics
- Content Access with CloudFront
- Static and Dynamic content
- CloudFront Origins
- Securing CloudFront distributions
- Monitoring CloudFront
- Best Practices when securing CloudFront content
CloudFront
CloudFront
A content delivery web service delivered from AWS Edge locations
Distribution of static web content (html, css, php, fonts, and image files)
The user that requests content is routed to an AWS Edge location:
- The selected Edge location will have the lowest latency ensuring the best delivery performance
- If the content is locally cached at the Edge, it is immediately delivered
- If the content is not cached, it is retrieved from the S3 bucket or HTTP web server
CloudFront supports the processing, storage, and transmission of credit card data by a merchant (PCI DSS up to Level 1)
So in other words “Your coffee order is safe”, when ordering from Wired Brain Coffee!
CloudFront Setup
CloudFront Delivery
Catching provides the ability to serve static content to more than one user at a time
CloudFront Setup
CloudFront Origins
Origin Server Types
S3 Content Origin
S3 buckets
- Upload content to bucket(s)
- Bucket name must conform to DNS naming requirements
- Origin path
- Origin ID
- Security (Restricts bucket access, OIN)
If bucket is not configured as website endpoint:
- bucket–name.s3.amazonaws.com
If bucket is configured as website:
- S3 static website end-point
Origin Servers
EC2 web server
- Matches the URL requested by end-user with distribution rules
- Utilizes "Regional Cache" service
Private HTTP server
- Must be publicly accessible
Recommended guidelines:
- Host and serve the same content from "team of redundant servers"
- Restrict access requests to ports your custom origin is listening on
- Use the (WAF) Windows Application Firewall to control access to your content
Securing CloudFront Distribution
Restricting Distribution of Your Content
Control Access to S3 with OAI
CloudFront Private Content
Allow access only to subscribers and customers
Create a policy using:
- Signed URLs (query string)
- Marketing emails
- Signed Cookies: (set cookie header)
- Whole site authentication, media streaming
Provides end-to-end content access control when combined with OAI
Setup Origin Access Identities
CoudFront Connection Details
Client to CloudFront Connection
CloudFront to Origin Server Connection
Monitoring CloudFront
CloudFront Usage Charts
Track trends in data transfer and requests for each active CloudFront distribution
- Both HTTP and HTTPS
- Provides up to 60 days usage
- Hourly or daily
- Shows totals, peaks, and averages on selected timeframe
Monitoring with CloudWatch
Provide almost real-time monitoring of traffic flow
- Six different metrics with alarms can be configured
- Automatically published at one minute intervals
- Set CloudWatch alarms on any abnormal traffic patterns
No additional costs for monitoring
CloudWatch Metrics
- Requests
- Bytes Downloaded
- Bytes Uploaded
- 4xx Error Rates
- 5xx Error Rates
- Total Error Rate
Designing for Failure with Route 53
CloudFront Monitoring Options
Best Practices When Securing Content
Use End-to-end HTTPS
CloudFront supports HTTPS traffic:
- From browser to Edge locations
- From Edge locations to origin
Redirect all HTTP traffic to HTTPS traffic
- Defined in CloudFront distribution
Use CloudFront SSL or custom SLL certificate on edge location
Use Amazon S3 for Static Assets
Free data transfer from S3 to CloudFront
- Avoid having to resize your origin servers
Decrease the load on your web servers
- S3 is highly available and scalable
Control Access to Private Content
Consider using Origin Access Identity (OAI)
"Special" user that only has access to your specific S3 content
- Stops direct S3 access
With OAI, content can be only accessed via CloudFront
Use Route 53 ALIAS Records
Use Route 53 to direct queries to CloudFront distributions
DNS queries to ALIAS records are free!
ALIAS record can be created for your zone apex
Reduce DNS lookups when using ALIAS records
CloudFront will directly translate your ALIAS record into the CloudFront IP address
- No additional charges
Configure Custom Error Pages
- Create custom error pages to help improve customer experience
- Deliver error pages from S3 and not your origin server
- Set a low error cashing minimum TTL
- After the TTL has expired CloudFront will check and see if the requested resource is now available
Use CloudFront Access Logs
No additional cost for utilizing CloudFront logs
- Must be manually turned on
Take advantage of analytics to deep dive into your content usage
CloudFront Reports
- Cache statistics
- Popular objects
- Top referrers
- Viewer reports
- Locations
- Browsers
- Operating systems
Wrap Up
- Content Access with CloudFront
- Static and Dynamic content
- CloudFront Origins
- Securing CloudFront distributions
- Monitoring CloudFront
- Best Practices when securing CloudFront content
Up Next:Securing Managed Services
Question
