Published on March 4, 2014
AWS Tips for LAUNCHing Your Infrastructure in the Cloud https://secure.flickr.com/photos/aloha75/6109626449
Y-Hack 2013 https://secure.flickr.com/photos/psd/4389135567/
LAUNCH Festival 2014 Chris Munns - @chrismunns Amazon Web Services Solutions Architect New Yorker Formerly Senior Operations @Etsy & @Meetup Little time at a Hedgefund and Xerox Rochester Institute of Technology: Applied Networking and Systems Administration ‘05 – Internet Geek – – – – –
What is AWS?
What is AWS? Deployment & Administration Application Services Compute Storage Networking AWS Global Infrastructure Database
Regions US-WEST (Oregon) EU-WEST (Ireland) AWS GovCloud (US) ASIA PAC (Tokyo) US-EAST (Virginia) ASIA PAC (Sydney) US-WEST (N. California) SOUTH AMERICA (Sao Paulo) ASIA PAC (Singapore)
Availability Zones US-WEST (Oregon) EU-WEST (Ireland) AWS GovCloud (US) ASIA PAC (Tokyo) US-EAST (Virginia) ASIA PAC (Sydney) US-WEST (N. California) SOUTH AMERICA (Sao Paulo) ASIA PAC (Singapore)
AWS OpsWorks Amazon SNS Amazon SES Amazon CloudSearch Amazon SWF Amazon SQS Amazon Amazon Elastic AWS AWS IAM CloudWatch Beanstalk CloudFormation Deployment & Administration Amazon EMR Amazon Elastic Transcoder Storage & Content Delivery Amazon Route 53 Database Amazon VPC AWS Direct Connect Amazon RDS Amazon DynamoDB Amazon RedShift Networking Amazon Kinesis Amazon ElastiCache App Services Compute Amazon EC2 AWS AWS Data CloudTrail Pipeline AWS Global Infrastructure Amazon S3 Amazon CloudFront AWS Storage Gateway Amazon Glacier
4 Easy & Basic Areas to Focus • Security • Performance • Fault Tolerance and Scale • Cost
AWS Multi-Factor Authentication Helps prevent anyone with unauthorized knowledge of your email address and password from impersonating you • Integrated into – AWS Management Console – Key pages on the AWS Portal • Forums, Support Center, and Account/Usage Activity pages – S3 (Secure Delete) • Virtual MFA – App for Android – Google Authenticator (iOS, Android, and Blackberry)
Temporary Security Credentials (sessions) • Temporary security credentials containing Identity for authentication Access Policy to control permissions Configurable Expiration (1 – 36 hours) • Supports AWS Identities (including IAM Users) Federated Identities (users customers authenticate) • Scales to millions of users – No need to create an IAM identity for every user • Use Cases Identity Federation to AWS APIs Mobile and browser-based applications Consumer applications with unlimited users
AWS Identity and Access Management (IAM) • • • • • • • • • Users and Groups within Accounts Roles for EC2 instances Unique security credentials • Access keys • Login/Password • optional MFA device Policies control access to AWS APIs Policies to restrict access to resources based on tags and other identifiers (subnet, class, AMI) API calls must be signed Deep integration into some Services • S3: policies on objects and buckets • Fine-Grained Access Control for DynamoDB AWS Management Console supports User log on Not for Operating Systems or Applications • use LDAP, Active Directory/ADFS, etc...
Multi-tier Security Approach Example Web Tier Application Tier Database Tier Ports 80 and 443 only open to the Internet Engineering staff have ssh access to the App Tier, which acts as Bastion Sync with on-premises database Amazon EC2 Security Group Firewall All other Internet ports blocked by default
Choose the right instance type • Over 25 instance types: • High CPU • High Memory • High Storage • High I/O • Bigger isn’t always better! • Going Horizontal isn’t always better either! • Don’t go with the cheapest instances because its cheapest. This laptop is several times more powerful than an m1.small
Choose the right storage
Choose the right storage 2 types of EC2 storage on AWS: • Local(ephemeral/instance based) – Regular disk – SSD • EBS – Standard – PIOPs
Choose the right storage 2 types of EC2 storage on AWS: • Local(ephemeral/instance based) – Not Persistent – RAID for increased performance • EBS – – – – Persistent Snapshots Flexible size/performance tuned by you RAID for increased performance
Choose the right storage 2 types of EC2 storage on AWS: • Local(ephemeral/instance based) – Local app/OS data – Database data that is highly replicated • EBS – Database data less replicated – Important data for your apps
Amazon Simple Storage Service • • • • Object based storage for the web 11 9s of durability Good for things like: – Static assets ( css, js, images, videos ) – Backups – Logs – Ingest of files for processing “Infinitely scalable” 5 • • • • • • • Supports fine grained permission control Ties in well with CloudFront Ties in with EMR Acts as a logging endpoint for S3/CloudFront/Billing Supports Encryption at transit and at rest Reduced Redundancy 1/3 cheaper Glacier for super long term storage 3
Use a CDN! CDN for Static CDN for Static & Content No CDN Dynamic Content • Server Load Response Time Server Load Response Time Server Load • • • • • • • Cache static content at the edge for faster delivery Helps lower load on origin infrastructure Dynamic and Static Content Streaming Video Zone Apex support Custom SSL certificates Low TTLs ( as short as 0 seconds ) Lower costs for origin fetches ( between S3/EC2 and CloudFront ) Optimized to work with EC2, S3, ELB, and Route53 Volume of Data Delivered (Gbps) • Response Time Amazon CloudFront is a web service for scalable content delivery. 80 70 60 50 40 30 20 10 0 8:00 AM 9:00 AM 10:00 11:00 12:00 AM AM PM 1:00 PM 2:00 PM 3:00 PM 4:00 PM 5:00 PM 6:00 PM 7:00 PM 8:00 PM 9:00 PM
FAULT TOLERANCE + SCALE
Spot the Difference?
MULTIPLE AVAILABILITY ZONES https://secure.flickr.com/photos/lakelou/6307404651
Your instances: Pets vs. Cattle https://secure.flickr.com/photos/81015532@N00/2192612785 vs.
MOOOO IM AN INSTANCE • No “pet” infrastructure, aka resources you’d be heartbroken if they went away • Infrastructure should be tolerable of handling failed/lost components • Have no “golden eggs” • 2+ of EVERYTHING • Automate bootstrapping + deployment • Make this painless and notification-less for your team https://secure.flickr.com/photos/anemoneprojectors/9374133369 MOOOOOOOOOOOOO….
Typical weekly traffic to Amazon.com Sunday Monday Tuesday Wednesday Thursday Friday Saturday
Typical weekly traffic to Amazon.com Provisioned capacity Sunday Monday Tuesday Wednesday Thursday Friday Saturday
November traffic to Amazon.com November
November traffic to Amazon.com Provisioned capacity November
November traffic to Amazon.com 76% Provisioned capacity November 24%
November traffic to Amazon.com November
Auto-Scaling lets you do this!
Auto-Scaling Trigger auto-scaling policy Amazon CloudWatch Automatic resizing of compute clusters based on demand Feature Details Control Define minimum and maximum instance pool sizes and when scaling and cool down occurs. Integrated to Amazon CloudWatch Use metrics gathered by CloudWatch to drive scaling. Instance types Run Auto Scaling for On-Demand and Spot Instances. Compatible with VPC. aws autoscaling create-auto-scaling-group --auto-scaling-group-name MyGroup --launch-configuration-name MyConfig --min-size 4 --max-size 200 --availability-zones us-west-2c
Leverage Elastic Load Balancing Feature Available Details Load balance across instances in multiple Availability Zones Health checks Automatically checks health of instances and takes them in or out of service Session stickiness Route requests to the same instance Elastic Load Balancer • Create highly scalable applications • Secure sockets layer Distribute load across EC2 instances in multiple availability zones • Little to no administration necessary • Automatically attach instances on bootup via API or via Auto-Scaling Monitoring Supports SSL offload from web and application servers with flexible cipher support Publishes metrics to CloudWatch
Understand Cost Models Amazon EC2 Amazo n EMR • On Demand • Reserved Instances • Spot Amazon Amazon Amazon ElastiCache RedShift RDS Amazon CloudFront • Price Classes Amazon S3 • Standard • Reduced Redundancy • Glacier* Amazon DynamoDB • Provisioned Capacity • Reserved Capacity • On Demand • Reserved Instances *Glacier isn’t a pricing model for S3, but another service part of the Storage family of services
Turn things Off! • Unused and forgotten EC2 instances • Shrink disk space if you don’t need it now • Auto-Scaling to shrink tiers during lower traffic periods • Dev/Test environments during nights • Use smaller instances if resource usage is always low (see CloudWatch data) https://secure.flickr.com/photos/93307674@N03/8548071813/
Business Support starts at 100$/month
Want to watch this again later? Sign in to add this video to a playlist. Video taken at the LAUNCH Festival 2014 on the WSGR Startup Stage ...
... launching your first VM on the “other” public cloud. ... and “completeness of vision” among cloud infrastructure ... to AWS Cloud Services ...
Launch a Cloud LaunchA Cloud ... AWS EC2 Free Tier, Windows Server, ... AWS Tips for Launching Your Infrastructure in the Cloud, ...
One of the biggest advantages of using AWS is that the API allows you to automate every part of your cloud infrastructure; from launching and provisioning ...
AWS eliminates having to predict and provision infrastructure needs before launching your marketing campaigns. We share four tips on why it's a great platform
Experience in deploying cloud only or hybrid Infrastructure ... What you need to prepare for your Amazon AWS ... (Launching soon…) AWS Exam Tips;
... highest likely hit for your cash. And EC2 is the main focus for AWS ... in the ir cloud infrastructure. Changes like launching any ... Tips ...
About AWS. Advancing the ... Your membership in AWS arms you with the latest industry news and intelligence, ... © 2016 American Welding Society ...