Auditing AWS Environments for Security and Best Practices

AUDITING IAM

The Basics


By scmGalaxy.com

About Me

DevOps@RajeshKumar.XYZ

Course Introduction

Celtra - AdCreator

Module Summary

  • Auditing IAM
  • Authentication and Authorization
  • Users
  • Groups
  • Roles
  • Policies
  • Combined and summary CLI examples

Authentication and Authorization

Authentication

Verifies who you are

Authorization

Verifies what you are allowed to do


Source: http://www.cyberciti.biz/faq/authentication-vs-authorization/

Authentication and Authorization in AWS

Celtra - AdCreator

Objects in IAM

Celtra - AdCreator

Users in IAM

  • Represent actual person or application
  • Can be member of multiple groups
  • Can have inline or managed policies
  • Different methods of authentication
    • User/pass for console
    • Up to 2 apikeys for CLI/SDK
    • MFA
  • User Access Advisor

Auditing Users Using CLI


Generate list of usernames

aws iam list-users --query Users[*].UserName

List group membership for user

aws iam list-groups-for-user --user-name <username>

User Access Advisor

Groups in IAM


Useful for aggregating permissions for multiple users

Can have managed or inline policies

Group Access Advisor

Auditing Groups Using CLI


Generate list of Groups

aws iam list-groups --query Groups[*].GroupName--output text

Get group members

aws iam get-group --group-name <groupname>

Group Access Advisor

Roles in IAM

  • Cross-service access
  • Cross account access
  • Trust Relationship
  • Role Access Advisor

Auditing Roles Using CLI


Generate list of Roles

aws iam list-roles --query Roles[*].RoleName--output text

List inline and managed policies attached to role

aws iam list-role-policies --role-name <rolename>

aws iam list-attached-role-policies --role-name <rolename>

Role Access Advisor

Policies in IAM


  • Managed policies
    • Policy version
  • Inline policies
  • Policy Access Advisor

Auditing Policies Using CLI


Generate list of customer managed policies

aws iam list-policies --scope Local --query Policies[*].Arn--output text

List entities using a managed policy

aws iam list-entities-for-policy --policy-arn<policyarn>

Policy Access Advisor

Combined Example and Summary CLI Commands

Combined example


 for i in `aws iam list-users --query Users[*].UserName`
 do
   echo "username $i”
   aws iam list-user-policies --user-name $i
 done
					
  • Loop through a dynamically generated list of usernames
  • Print the username
  • Print inline policies attached to each user (which is against best practices –use group policies instead)

Summary commands


 aws iam get-account-authorization-details
 aws iam get-account-summary
 aws iam get-account-password-policy
					
  • List all IAM objects and relationships to each other
  • Return numeric count of each object
  • Describe the IAM password policy for console credentials
  • Analysis of an IAM Credential Report
  • What can we learn?

Summary

  • IAM Objects
    • How they relate to each other
    • Audit individually and together
  • Methods for Audit
    • CLI
    • AWS Console
    • Credential Report

Auditing VPC

Module Overview

  • Auditing VPC
  • Network configuration
  • VPC Options
  • Ingress/Egress Points
  • Network security

What Is a VPC?

  • Data center in the cloud
  • Services placed inside
    • EC2
    • RDS
    • ELB
    • And more!

VPC Network Configuration

    Question

  • Where do I put my VPC?
  • How big is my VPC network?
  • How do I segment my network?
  • How can I keep my resources private?

    Choices

  • Pick an AWS region
  • CIDR network up to /16 (65536 addrs)
  • Create subnets
  • Route tables, NACLs, Security Groups

Create a VPC inventory


for i in `aws ec2 describe-regions --query
Regions[*].RegionName`
do
 echo “region $i”
 aws ec2 describe-vpcs --region $i
done
					
  • Loop through a dynamically generated list of regions
  • Print the region
  • Print vpc id, cid rrange, and other details

Check VPCs for CIDR overlap


for i in `aws ec2 describe-regions --query
Regions[*].RegionName`
do
 echo “region $i”
 aws ec2 describe-vpcs--region $i--query Vpcs[*].CidrBlock
done
					
  • Loop through a dynamically generated list of regions
  • Print the region
  • Print cidr range only, to visually check for overlap

VPC Options

  • DNS Support
  • DNS Hostnames
  • DHCP Options
    • Domain
    • DNS servers
    • NTP servers
    • NetBIOS servers

Check DHCP option sets


for i in `aws ec2 describe-regions --query
Regions[*].RegionName`
do
 echo “region $i”
 aws ec2 describe-dhcp-options--region $i–output text
done
					
  • Loop through a dynamically generated list of regions
  • Print the region
  • Print DHCP options. Look for custom domains or DNS servers when auditing.

VPC Ingress/Egress Points

  • Internet Gateway (IGW)
  • VPN Gateway (VGW)
  • AWS Direct Connect
  • VPC Peering Connections
  • VPC Endpoints
  • EC2 ClassicLink

Internet Gateway

  • 2-way traffic
  • Route table
  • Public IP
  • No firewall

internet gateway

aws ec2 describe-internet-gateways --query 
'InternetGateways[].{GwId:InternetGatewayId,
VpcId:Attachments[].VpcId}'--output text --region <region>
					

Create report of IGW and attached VPCs

Print the ID of each Internet Gateway in the region and which VPC it is attached to. Use JMES query format to customize output of the command.

VPN Gateway

  • 2-way traffic
  • Route table
  • Private IP
  • No firewall
  • IPSEC Encryption

vpn gateway

AWS Direct Connect

  • 2-way traffic
  • Route table
  • Private IP
  • No firewall
  • 801.Q VLAN and BGP

vpn gateway

VPC Peering

  • 2-way traffic
  • Route table
  • Private IP
  • No firewall
  • Private, Point to Point

vpn peering

Describe VPC Peering Connections


for i in `aws ec2 describe-regions --query Regions[*].RegionName`
do
 echo “region $i”
 aws ec2 describe-vpc-peering-connections --region us-west-2 -
 -query 
'VpcPeeringConnections[].[AccepterVpcInfo.VpcId,RequesterVpcInfo.VpcId]’
done
					
  • Loop through a dynamically generated list of regions
  • Print the region
  • Print the accepter and requester VPC Ids for each connection

VPC Endpoints

  • S3 only
  • 1-way traffic
  • Route table
  • Endpoint Policies

vpn endpoints

Describe VPC Endpoints


for i in `aws ec2 describe-regions --query Regions[*].RegionName`
do
 echo “region $i”
 aws ec2 describe-vpc-endpoints --query 'VpcEndpoints[].[VpcId,ServiceName]'
done
					
  • Loop through a dynamically generated list of regions
  • Print the region
  • Print each endpoint with the VPC ID and service name.

EC2 ClassicLink

  • 1-way traffic
  • Route table
  • Private IP
  • No firewall

ec2 classiclink

Discover VPCs with Classic Link Enabled


for i in `aws ec2 describe-regions --query Regions[*].RegionName`
do
 echo “region $i”
 awse c2 describe-vpc-classic-link --output json--query Vpcs[].[VpcId,ClassicLinkEnabled]
done
					
  • Loop through a dynamically generated list of regions
  • Print the region
  • Print each VPC ID and whether ClassicLinkis enabled

Discovering Ingress/Egress points on a VPC

VPC Security

  • Route Tables
  • Network Access Control Lists (NACLs)
  • VPC Flow Logs

VPC Route Tables

Destination

Any cidr network range or endpoint object

Target

Egress point


Select from list


aws ec2 describe-route-tables --query
 'RouteTables[].[[VpcId,Routes[].GatewayId][]]’ --output text
					

List all route tables in the region with target ID

Print each VPC ID, and all targets used by route tables in the VPC.

Useful for discovering possible egress points from the VPC.

Network Access Control Lists

network access control lists

aws ec2 describe-network-acls--output table --region 
<region>
					

Print NACLs in human-readable form

Print easy-to-read text output of all NACLs in a region, including the associated VPC ID and Subnet ID

VPC Flow Logs

vpc flow logs

Explore VPC Flow Logs and Route Tables

Summary

  • VPC can be customized in many different ways
  • VPC environments have many options for ingress and egress
  • Resources in the VPC can have security mechanisms separate from the VPC itself
  • Next Up: Auditing EC2

Auditing EC2

Module Overview


    Auditing EC2

  • Security Features
  • Inventory And Best Practices

EC2 Security

  • Shared Responsibility Security Model
  • Hypervisor
  • EC2 keypairs
  • Security groups
  • Demo: Auditing Security Groups with AWS Config

Shared Responsibility Security Model

shared responsibility security model for aws

Security Model Applied to EC2

AWS

Data centers

Hardware

Hypervisor

API

Customer

OS access

OS updates

Security patches

Penetration testing

EC2 Hypervisor

  • Xen Hypervisor (heavily modified)
  • Virtualization types
    • HVM is full virtualization
    • PV is “lighter virtualization”
  • VM Isolation on all resources
    • CPU
    • Memory
    • Disk
    • Network

EC2 Keypairs

Linux

SSH using private key

Windows

Decrypt Administrator PW

Security Groups -Function

  • Stateful firewall
  • Applied to network interface
  • Multiple allowed per interface
    • Rulesetsapplied as a whole
    • Least restrictive rule wins
  • Inbound rules default deny
  • Outbound rules default allow

Security Groups –Best Practices

Security Groups –Best Practices for aws

Auditing Security Groups Script Part 1


#!/bin/bash
REGION=us-west-1
SGOUT="/tmp/sginfo”
aws ec2 describe-security-groups --region $REGION --output text > $SGOUT
IFS=$'\n'
cat $SGOUT | while read line
do
					

Auditing Security Groups Input File


SECURITYGROUPS registration frontend sg-c21df0a7 vpc1w2-reg2.0-prod-frontend 168369983848 vpc-0ef6e36c
IPPERMISSIONS    80  tcp  80
IPRANGES        0.0.0.0/0
IPPERMISSIONS     -1
IPRANGES       10.0.0.0/18
IPRANGES       10.0.128.0/18
IPRANGES       10.0.64.0/18
IPPERMISSIONS   22 tcp  22
IPRANGES       208.76.0.0/22
IPPERMISSIONS   443  tcp  443
IPRANGES       0.0.0.0/0
IPPERMISSIONSEGRESS   -1
IPRANGES       0.0.0.0/0
					

Auditing Security Groups Script Part 2


case $line in
SECURITYGROUPS*)
GID=(`echo $line | awk–F”\t” '{print $3}'`)
GNAME=(`echo $line | awk–F”\t” '{print $4}'`)
;;
IPPERMISSIONSEGRESS*)
PROTO=“EGRESS”
;;
					

Auditing Security Groups Script Part 3


IPPERMISSIONS*)
FROMPORT=(`echo $line | awk–F”\t” '{print $2}'`)
PROTO=(`echo $line | awk–F”\t” '{print $3}'`)
TOPORT=(`echo $line | awk–F”\t” '{print $4}'`)
;;
					

Auditing Security Groups Script Part 4


IPRANGES*)
CIDR=(`echo $line | awk–F”\t” '{print $2}'`)
if [[ "$CIDR" = "0.0.0.0/0" && "$PROTO" != ”EGRESS" ]]; then
  echo "$GNAME,$GID,$CIDR,$PROTO,$FROMPORT,$TOPORT"
fi
;;
esac
Done
rm $SGOUT
					

Auditing Security Groups Script Output


tu-prod-gateway,sg-3a05610a,0.0.0.0/0,tcp,22,22
vpc2w2-reg2.0-loadtest-frontend,sg-e318f586,0.0.0.0/0,tcp,80,80
vpc2w2-reg2.0-loadtest-frontend,sg-e318f586,0.0.0.0/0,tcp,443,443
vpc2w2-wp-stage-frontend,sg-c4abe6a3,0.0.0.0/0,tcp,80,80
vpc2w2-wp-stage-frontend,sg-c4abe6a3,0.0.0.0/0,tcp,443,443
					

Auditing Security Groups Script Summary

  • CSV is easy to ingest into a database
  • Python is useful for SDK work
  • Task automation will make audits easier

Using AWS Configto Audit SG

AWS Trusted Advisor SG Report

EC2 Inventory and Tagging Best Practices

  • Using CLI to inventory EC2
  • Resource tagging
  • Trusted advisor
  • Demo: EC2 dashboard and filters

Why Use CLI to Audit EC2?

why use cli to audit ec2? for aws

aws ec2 describe-instances --output text --region <region>
 --filter Name=instance-state-code,Values=16 --query 
 'Reservations[*].Instances[*].[InstanceId,PublicDnsName]'
					

Generate List of Instance ID and Public DNS

Print the Instance ID of each running instance (instance state code 16) and its associated Public DNS.

Great for inventory and loop-and-ssh functions.


aws ec2 describe-instances --output text --region <region> 
--query 
Reservations[].Instances[].[[InstanceId,SecurityGroups[].Gr
oupId,SecurityGroups[].GroupName][]]| tr'\t' ','
					

Generate CSV of Instance ID, SG ID, SG Name

Print each Instance ID in the region with its Security Group IDs and Security Group Names in CSV format.

The [] at the end of the query forces output to stay on the same line.

EC2 Tags

  • 10 tags per instance
  • Key/Value pairs, 128/255 character limit
  • Integrated with CLI and billing

Tagging Strategies

tagging strategies for aws

Audit for Missing Tag Part 1


#!/bin/bash
REGION="us-west-2"
TAG="CostCenter"
TAGFILE="/tmp/ec2tags"
aws ec2 describe-tags --region $REGION --filters=Name=resource-
type,Values=instance--output text > $TAGFILE
ec2instances=`awsec2 describe-instances --region $REGION --query 
Reservations[].Instances[].InstanceId--output text |tr'\t' '\n'`
					

Audit for Missing Tag Part 2


for i in `echo $ec2instances`; do
if [[ `grep $i $TAGFILE |grep "$TAG"` = "" ]]; then
echo "$i is missing $TAG"
fi
done
rm $TAGFILE
					

AWS Trusted Advisor

  • Free “Lite” version
  • Types of predefined reports
    • Cost optimization
    • Performance
    • Security
    • Fault tolerance

EC2 dashboard and filters

Summary

  • EC2 security involves AWS and customer
  • Security groups can require extra effort to audit from the CLI
  • Tags and filters are important
  • Next Up: Auditing EBS

Auditing EBS

Module Overview

  • EBS Basics
  • EBS Security
  • EBS Inventory
  • EBS Best Practices

EBS Basics

  • Network attached block storage
  • Availability Zone specific
  • Frequent Snapshots Decrease AFR

Volume Types

volume types for aws

EBS Security

  • NOT shared storage
  • Hardware Decommissioning
  • Volume encryption
  • Demo: Add encryption to a volume

Storage Decommissioning


https://d0.awsstatic.com/whitepapers/aws-security-whitepaper.pdf


AWS uses the techniques detailed in DoD5220.22 -M (“National Industrial Security Program Operating Manual “) or NIST 800 -88 (“Guidelines for Media Sanitization”) to destroy data as part of the decommissioning process. All decommissioned magnetic storage devices are degaussed and physically destroyed in accordance with industry -standard practices.

EBS Volume Encryption

  • Boot volumes
  • Data volumes
  • Encrypts traffic and data
  • Not available for all instance types
    • Newer generations of EC2 only
    • Not for M1, C1, T1, etc
  • Uses KMS (Key Management Service)

EBS Encryption Integration Points

ebs encryption integration points for aws

Demo: Add Encryption To A Volume

EBS Inventory

Volumes

By Type

By Size

Detached

Snapshots

Rotation

Region Copy

Orphans

Volume Inventory: Find Detached Volumes


	aws ec2 describe-volumes --region <region> \
	--query Volumes[].VolumeId\
	--filters Name=status,Values=available
					

Snapshot Inventory: Purge All But N Part 1


	#!/bin/bash
	VOLUMEID=$1
	REGION=$2
	NUMKEEP=$3
	TAIL=$(( $NUMKEEP + 1 ))
					

Snapshot Inventory: Purge All But N Part 2


SNAPIDS=`aws ec2 describe-snapshots \
--output text --region $REGION \
--query Snapshots[].[SnapshotId,StartTime] \
--filters Name=volume-id,Values=$VOLUMEID \
|sort -r -k2 |cut -f1 |tail -n +$TAIL`
					

Snapshot Inventory: Purge All But N Part 3


if [ "$SNAPIDS" != "" ]; then
  for SNAP in $SNAPIDS; do
   echo "Deleting snapshot $SNAP"
   aws ec2 delete-snapshot --region $REGION --snapshot-id $SNAP
  done
else
echo "Not enough snapshots, exiting"
fi
					

EBS Best Practices: Volume Sizing

ebs best practices: volume sizing for aws

Best Practices: Volume Performance

best practices: volume performance for aws

Demo Scenario: Compromised Instance

  • How can we perform Root Cause Analysis safely?
  • VPC, EC2, EBS

Summary

  • EBS Volumes give many choices for performance
  • Encryption is simple to deploy
  • Snapshot management is critical
  • Next Up: Auditing S3

Auditing S3

Module Overview

  • S3 Basics
  • S3 Security
  • S3 Inventory
  • S3 Best Practices

S3 Basics

  • Object Store
  • Natively Available Online
  • Fully Managed Service

S3 Hierarchy

s3 hierarchy for aws

S3 Storage Classes

s3 storage classes for aws

S3 Security

  • Access Control
    • IAM
    • ACLs
    • Bucket Policy
  • Encryption
    • Server Side
    • Client Side

Access Control

access control for aws

S3 Object Encryption –Server Side with KMS

s3 object encryption –server side with kms for aws

S3 Object Encryption –Client Side

s3 object encryption client side for aws

Demo: Uploading into S3 Using SSE-KMS

S3 Inventory

Buckets

By Region

By Name

By Size

Objects

By Prefix

By Size

By Encryption Status

S3 Inventory: Sort Buckets by Region Part 1


PRESORTFILE="/tmp/notsorted"
POSTSORTFILE="/tmp/sorted"
rm $PRESORTFILE $POSTSORTFILE
BUCKETLIST=`aws s3api list-buckets --query 'Buckets[].Name' |tr'\t' '\n'`
					

S3 Inventory: Sort Buckets by Region Part 2


for BUCKET in $BUCKETLIST; do
LOCATION=`aws s3api get-bucket-location --bucket $BUCKET`
echo "$LOCATION,$BUCKET" >> $PRESORTFILE
done
					

S3 Inventory: Sort Buckets by Region Part 3


sort -n $PRESORTFILE > $POSTSORTFILE
cat $POSTSORTFILE
					

Snapshot Inventory: List Bucket Sizes Part 1


#!/bin/bash
BUCKETLIST="/tmp/sorted"
for BUCKETINFO in `cat $BUCKETLIST`; do
REGION=`echo $BUCKETINFO |cut -f1 -d,`
if [[ $REGION == "None" ]]; then
REGION="us-east-1 --endpoint-urlhttps://s3.amazonaws.com"
fi
					

Snapshot Inventory: List Bucket Sizes Part 2


BUCKET=`echo $BUCKETINFO |cut -f2 -d,`
SIZE=`aws s3api list-objects --region $REGION --bucket $BUCKET --output json--query '[sum(Contents[].Size)]'`
echo "$BUCKET,$SIZE"
done
					

S3 Best Practices: Lifecycle Configuration

s3 best practices: lifecycle configuration for aws

Best Practices: Naming Conventions

best practices: naming conventions for aws

Summary

  • S3 is a managed service. Customer only responsible for data protection and access.
  • Several options for protecting objects
  • Bucket audit actions can be complex

Audit Results

audit results for aws

Questions?

docker-questions

Thanks for You!