AWS Security Operations Object Storage

About Me

DevOps@RajeshKumar.XYZ

AWS Security Operations: Securing Access to S3 Object Storage

Why have a course focused on AWS security?

Security is still 25 / 7 in the cloud

On the Internet, attack is easier than defense

This course will help you with all things in securing AWS S3 Storage

A working knowledge of AWS services is a bonus but not essential

AWS Security Operations

  • Part 1: Designing for Acceptable Risk and Compliance
  • Part 2: Securing Core AWS Infrastructure Services
  • Part 3: Securing Access to Object Storage

Topics

  • The Simple Storage Service
  • S3 and Regulated Workloads
  • S3 Resource Control–Public vs. Private
  • S3 Data Protection
  • S3 Versioning
  • Lifecycle Management
  • Glacier Storage
  • Importing Data into S3
  • Best Practices When Planning S3

Simple Storage Service

S + S + S = S3

Storage Review

Storage Review

S3 Storage Concepts

Redundantly stored

  • On multiple devices
  • Across multiple facilities
  • Within the selected region

Highly scalable, cloud storage

  • Store and retrieve any amount of data

S3 buckets store data in selected region

  • Technically defined as "Object Storage"

Shared Responsibility Model

  • Customers manage their data
  • AWS manages the infrastructure layer

S3 Design

S3 Design

Data never leaves region

  • Unless you move it cross-region
  • Data is stored as objects
  • Objects can be any file type
  • Objects are assigned a object key
  • The object key linked to each object is unique
  • Metadata can also be stored with each object

Permissions can be set several ways:

  • On each object
  • On each bucket
  • IAM policy or bucket policy

S3 Security

S3 Security Features

  • Access Logging
  • Event Notifications
  • CloudTrail Audit
  • CORS Configuration
  • Versioning
  • CloudWatch S3 Metrics

Managed Services Integration with S3

Managed Services Integration with S3

S3 Durability

S3 Durability

S3 Durability

Choice of Storage Classes for S3

Choice of Storage Classes for S3

Amazon S3 can be used with PCI DSS, HIPAA, ITAR, SOC3, and ISO 27001 standards.

S3 Storage Setup

S3 Resource Control

Accessing S3 Data

S3 storage starts as "private" by default

  • Buckets, objects, and configuration
  • The resource owner can always access

Two methods to share data:

  • Objects can be made Public
  • Policies for selective access control

Resource policies

  • Bucket policies –groups (of users)
  • Add or deny permissions
  • ACLs –read or write access to objects to groups of users, not specific users

User policies

  • IAM policies for fine grained control for users within your AWS account

S3 Object Tags

S3 Object Tags

S3 Bucket Policy

S3 Data Protection

S3 Encryption Choices

S3 Encryption Choices

Client side encryption is performed by the client directly to the selected file

Server Side Encryption

  • Data is encrypted during the writing to disk process
  • Data is decrypted during the reading process
  • Encryption is performed at the object level
  • SSE encrypts the object data but not the associated metadata
  • Unencrypted metadata could be a compliance issue

Protecting Data in Transit

Data moving to and from AWS S3

  • Use SSL endpoints over HTTPS
  • Secure upload and download of data
  • Client side encryption before upload

Using VPC endpoints

  • Connect to a specific VPC endpoint
  • Use bucket policies for multiple endpoints

Protecting Data at Rest

Server side encryption

  • AWS does the encryption and decryption
  • Key management; encrypted by AWS S3 master key
  • Stored separately from your data
  • AES 256-bit encryption
  • Optional user provided keys
  • Integrated with AWS KMS

Client side encryption

  • Choice depends on OSand storage

Server Side Encryption Steps

AWS S3 Server-side Encryption

AWS S3 Server-side Encryption

Customer Provided Encryption Key (SSE-C)

Customer Provided Encryption Key (SSE-C)

Managing S3 Encryption

S3 Versioning

S3 Versioning

Versioning allows you to keep multiple versions of the same object, in the same bucket

Preserve, retrieve, or restore every version of every object

Versioning is enabled at the bucket level

  • Disabled by default
  • Can be disabled, but not deleted
  • Versioning operations managed by AWS

Create lifecycle policies to control dataflow

  • Archival and expiration dates
  • Integrated with AWS glacier

Overriding an object result in a new object being created, with a new version number attached

Versioning applies to all objects in the bucket

Glacier Storage

Glacier Storage

  • Low-cost Storage
  • Backup and Archiving
  • Files are archived in Vaults
  • Vaults are stored in Regions
  • "Data at Rest" AES-256
  • "Data in Transit" SSL end-points

Amazon Glacier Vault Lock

Compliance controls for matching your compliance objectives

  • Ex: SEC Rule 17a-4(f)

Create vault lock policies

  • Ex: WORM
  • Deny deletes

Contents cannot be modified once locked with a vault policy

Vault access policy = Access

Vault lock policy = Content

Amazon Glacier supports PCI DSS, HIPAA, ITAR, SOC3, and ISO 27001 standards

Versioning and Lifecycle Policies

Monitoring S3 and Glacier

Monitor Your Storage Usage

Monitor Your Storage Usage

CloudWatch S3 Metrics

Operational and performance monitoring

  • Metrics for your choice of data
    • Bucket, vaults, tags, and prefixes
    • Ex: Changes to lifecycle policies

Monitor S3 with CloudWatch

  • Total bytes for Standard storage
  • Total bytes for Reduced Redundancy storage

Monitor Glacier with CloudWatch

  • Who made the request?
  • IP address where request was made from

Alarm metrics

  • Set thresholds for alarms
  • One minute CloudWatch metrics; and longer

S3 Data Events in CloudTrail

S3 Data Events in CloudTrail

S3 Monitoring Options

Importing Data into S3

S3 Data Transfer Options

  • AWS Direct Connect
  • AWS Snowball @ Snowball Edge
  • Amazon Kinesis Firehouse
  • S3 Transfer Acceleration
  • AWS Snowmobile
  • AWS Storage Gateway

"File Gateway" (Hybrid Storage Solution)

Hybrid Storage Solution

Best Practices Planning S3 Storage

Enable Versioning

Protect from accidental deletion

  • New version created from each upload
  • Easy retrieval of "deleted objects"

Reminder:

  • The default –no versioning
  • Once versioning is enabled
  • Versioning can only be suspended

Enable MFA Bucket Protection

Enabling MFA requires:

  1. Provide your AWS account's access key
  2. A valid MFA code
  • Changing the versioning state of your bucket
  • Permanently deleting an object version (MFA Delete)

Protect Data and Data Cost with Lifecycle Policies

Data Cost with Tiering

  • Transition –Standard to Reduced Redundancy, or Amazon Glacier; after defined timeframe
  • Expiration: Delete objects after defined period of time

Data Protection with Security Controls

  • Policies can also be set at the prefix, or object tag level
  • Ex: Key Name Prefix: logs/

Versioning and Lifecycle Policies

Versioning and Lifecycle Policies

Consider Cross–region Replication

Control: Replicate all deleted objects to destination bucket in separate region

  • Both source and destination buckets need to enable versioning
  • Uploads into source bucket will then be replicated

Compliance: Records are stored hundreds of miles apart

Secure: Remote replicas managed by separate AWS accounts

Faster: Lower latency

Parallel PUT Uploads

Increase your aggregate throughput by parallelizing PUTs on fast networks

  • Upload multiple parts of large objects
  • Removes bottleneck from the upload process
  • Helps to reduce network connectivity issues

Parallelize GETS

TransferManager provides multi-threaded download performance

  • Useful for larger objects
  • Upgrade SDK for Java to 1.11.0 or later
  • If the object is determined to be multipart, download is performed in parallel

IAM Roles

Use IAM roles for S3 control

  • IAM policies for users, groups, and roles
  • Access control is kept at the IAM level
  • Central permission management

"What can this user do in AWS?"

Bucket Policies

S3 Bucket policies are directly attached to the S3 bucket

  • Simple method to grant cross-account policies
  • Access control in the S3 environment

"Who can access this bucket?"


Note: S3 ACLs control individual objects within a bucket at the user level

Wrap Up

  • The Simple Storage Service
  • S3 and Regulated Workloads
  • S3 Resource Control–Public vs. Private
  • S3 Data Protection
  • S3 Versioning
  • Lifecycle Management
  • Glacier Storage
  • Importing Data into S3
  • Best Practices When Planning S3

Up Next:
Securing Content Access with CloudFront

Securing Content Access with CloudFront

Why have a course focused on AWS security?

Security is still 25 / 7 in the cloud

43% of C-level executives say negligent insiders are the greatest threat to sensitive data

This course will help you with all things in securing AWS content access

A working knowledge of AWS services is a bonus but not essential

Topics

  • Content Access with CloudFront
  • Static and Dynamic content
  • CloudFront Origins
  • Securing CloudFront distributions
  • Monitoring CloudFront
  • Best Practices when securing CloudFront content

CloudFront

CloudFront

A content delivery web service delivered from AWS Edge locations

Distribution of static web content (html, css, php, fonts, and image files)

The user that requests content is routed to an AWS Edge location:

  • The selected Edge location will have the lowest latency ensuring the best delivery performance
  • If the content is locally cached at the Edge, it is immediately delivered
  • If the content is not cached, it is retrieved from the S3 bucket or HTTP web server

CloudFront supports the processing, storage, and transmission of credit card data by a merchant (PCI DSS up to Level 1)

So in other words “Your coffee order is safe”, when ordering from Wired Brain Coffee!

CloudFront Setup

CloudFront Setup

CloudFront Delivery

CloudFront Delivery

Catching provides the ability to serve static content to more than one user at a time

CloudFront Setup

CloudFront Origins

Origin Server Types

Origin Server Types

S3 Content Origin

S3 buckets

  • Upload content to bucket(s)
  • Bucket name must conform to DNS naming requirements
  • Origin path
  • Origin ID
  • Security (Restricts bucket access, OIN)

If bucket is not configured as website endpoint:

  • bucket–name.s3.amazonaws.com

If bucket is configured as website:

  • S3 static website end-point

Origin Servers

EC2 web server

  • Matches the URL requested by end-user with distribution rules
  • Utilizes "Regional Cache" service

Private HTTP server

  • Must be publicly accessible

Recommended guidelines:

  • Host and serve the same content from "team of redundant servers"
  • Restrict access requests to ports your custom origin is listening on
  • Use the (WAF) Windows Application Firewall to control access to your content

Securing CloudFront Distribution

Restricting Distribution of Your Content

Restricting Distribution of Your Content

Control Access to S3 with OAI

Control Access to S3 with OAI

CloudFront Private Content

Allow access only to subscribers and customers

Create a policy using:

  1. Signed URLs (query string)
    • Marketing emails
  2. Signed Cookies: (set cookie header)
    • Whole site authentication, media streaming

Provides end-to-end content access control when combined with OAI

Setup Origin Access Identities

CoudFront Connection Details

Client to CloudFront Connection

Client to CloudFront Connection

CloudFront to Origin Server Connection

Client to CloudFront Connection

Monitoring CloudFront

CloudFront Usage Charts

Track trends in data transfer and requests for each active CloudFront distribution

  • Both HTTP and HTTPS
  • Provides up to 60 days usage
  • Hourly or daily
  • Shows totals, peaks, and averages on selected timeframe

Monitoring with CloudWatch

Provide almost real-time monitoring of traffic flow

  • Six different metrics with alarms can be configured
  • Automatically published at one minute intervals
  • Set CloudWatch alarms on any abnormal traffic patterns

No additional costs for monitoring

CloudWatch Metrics

  • Requests
  • Bytes Downloaded
  • Bytes Uploaded
  • 4xx Error Rates
  • 5xx Error Rates
  • Total Error Rate

Designing for Failure with Route 53

Designing for Failure with Route 53
Designing for Failure with Route 53

CloudFront Monitoring Options

Best Practices When Securing Content

Use End-to-end HTTPS

CloudFront supports HTTPS traffic:

  • From browser to Edge locations
  • From Edge locations to origin

Redirect all HTTP traffic to HTTPS traffic

  • Defined in CloudFront distribution

Use CloudFront SSL or custom SLL certificate on edge location

Use Amazon S3 for Static Assets

Free data transfer from S3 to CloudFront

  • Avoid having to resize your origin servers

Decrease the load on your web servers

  • S3 is highly available and scalable

Control Access to Private Content

Consider using Origin Access Identity (OAI)

"Special" user that only has access to your specific S3 content

  • Stops direct S3 access

With OAI, content can be only accessed via CloudFront

Use Route 53 ALIAS Records

Use Route 53 to direct queries to CloudFront distributions

DNS queries to ALIAS records are free!

ALIAS record can be created for your zone apex

Reduce DNS lookups when using ALIAS records

CloudFront will directly translate your ALIAS record into the CloudFront IP address

  • No additional charges

Configure Custom Error Pages

  • Create custom error pages to help improve customer experience
  • Deliver error pages from S3 and not your origin server
  • Set a low error cashing minimum TTL
  • After the TTL has expired CloudFront will check and see if the requested resource is now available

Use CloudFront Access Logs

No additional cost for utilizing CloudFront logs

  • Must be manually turned on

Take advantage of analytics to deep dive into your content usage

CloudFront Reports

  • Cache statistics
  • Popular objects
  • Top referrers
  • Viewer reports
    • Locations
    • Browsers
    • Operating systems

Wrap Up

  • Content Access with CloudFront
  • Static and Dynamic content
  • CloudFront Origins
  • Securing CloudFront distributions
  • Monitoring CloudFront
  • Best Practices when securing CloudFront content

Up Next:Securing Managed Services

Question

Question

Thank You