How to Scale to Millions of Users with AWS

40 %
60 %
Information about How to Scale to Millions of Users with AWS

Published on July 13, 2016

Author: AmazonWebServices

Source: slideshare.net

1. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Ken Payne, AWS Solutions Architecture Jean Meyer, CEO at Once Dating AG 7th July 2016 Scaling the Platform for your Startup

2. Why are you here? Building the technology platform for your startup You want to prepare for success Learn about design patterns & scalability A pragmatic approach for startups

3. Priorities for startups Racing within a window of opportunity Small team with no legacy Focus on solving a problem Avoid over-engineering & re-engineering Reduce risk of failure when you go viral

4. A scalable architecture Can support growth in users, traffic, data size Without practical limits Without a drop in performance Seamlessly - just by adding more resources Efficiently - in terms of cost per user

5. Day 1 – Dev & Private Beta

6. Single host THE server (e.g. Apache, MyS QL) Elastic IP www.example.com Amazon Route 53 DNS service S erver Image (AMI)

7. Day 2 - Public Beta

8. We need a bigger server Add larger & faster storage (EBS) Use the right instance type Easy to change instance sizes Not our long term strategy Will hit an endpoint eventually No fault tolerance X1 instances offer 1,952 GiB of DDR4 based memory, powered by four Intel® Xeon® E7 8880 v3 (Haswell) processors that offer 128 vCPUs

9. Separating web and DB More capacity Scale each tier individually Tailor instance for each tier • Instance type • Storage Security • Security groups • DB in a private VPC subnet

10. But how do I choose what DB technology I need? SQL? NoSQL?

11. Start with a Relational DB SQL is versatile & feature-rich Lots of existing code, tools, knowledge Clear patterns to scalability (for read-heavy apps) Reality: eventually you will have a polyglot data layer • There will be workloads where NoSQL is a better fit • Combination of both Relational and NoSQL • Use the right tool for each workload

12. Key Insight: Relational Databases are Complex Our experience running Amazon.com taught us that relational databases can be a pain to manage and operate with high availability Poorly managed relational databases are a leading cause of lost sleep and downtime in the IT world! Especially for startups with small teams

13. Relational Databases MySQL, Aurora, PostgreSQL, Oracle, SQL Server Fully managed; Zero Admin Amazon RDS Aurora

14. Improving Efficiency

15. Offload static content Amazon S3: highly available hosting that scales • Static files (JavaScript, CSS, images) • User uploads S3 URLs – serve directly from S3 Let the web server focus on dynamic content

16. Amazon CloudFront Worldwide network of edge locations Cache on the edge • Reduce latency • Reduce load on origin servers • Static and dynamic content • Even few seconds caching of popular content can have huge impact Connection optimizations • Optimize transfer route • Reuse connections • Benefits even non cachable content C loudFront

17. CloudFront for static & dynamic content Amazon Route 53 EC 2 instance(s) S 3 bucket Static C ontent Dynamic C ontent css/* js/* Images/* Default(*) C loudFront Distribution

18. Database Caching Faster response from RAM Reduce load on database Application server 1. If data in cache, return result 2. If not in cache, read from DB RDS database Amazon ElastiC ache 3. And store in cache “Lazy Loading”

19. Amazon ElastiCache: in-memory cache Simple to Deploy Managed • Automatically replaces failed nodes • Patch management Elastic Compatible ElastiCache

20. Day 3 – Paying Customers

21. High Availability Availability Zone a RDS DB instance Web server S 3 bucket for static assets www.example.com Amazon Route 53 DNS service Amazon C loudFront ElastiC ache node 1

22. High Availability Availability Zone a RDS DB instance Availability Zone b Web server Web server S 3 bucket for static assets www.example.com Amazon Route 53 DNS service Amazon C loudFront ElastiC ache node 1

23. High Availability Availability Zone a RDS DB instance Availability Zone b www.example.com Amazon Route 53 DNS service Elastic Load Balancing Web server Web server S 3 bucket for static assets Amazon C loudFront ElastiC ache node 1

24. Elastic Load Balancing Managed Load Balancing Service Fault tolerant Health Checks Distributes traffic across AZs Elastic – automatically scales its capacity

25. High Availability Availability Zone a RDS DB instance Availability Zone b www.example.com Amazon Route 53 DNS service Elastic Load Balancing Web server Web server S 3 bucket for static assets ElastiC ache node 1 Amazon C loudFront

26. High Availability Availability Zone a RDS DB instance Availability Zone b www.example.com Amazon Route 53 DNS service Elastic Load Balancing Web server Web server RDS DB standby S 3 bucket for static assets ElastiC ache node 1 Amazon C loudFront

27. Data layer HA Availability Zone a RDS DB instance ElastiC ache node 1 Availability Zone b S 3 bucket for static assets www.example.com Amazon Route 53 DNS service Elastic Load Balancing Web server Web server RDS DB standby

28. Data layer HA Availability Zone a RDS DB instance ElastiC ache node 1 Availability Zone b S 3 bucket for static assets www.example.com Amazon Route 53 DNS service Elastic Load Balancing Web server Web server RDS DB standby ElastiC ache node 2

29. User Sessions? Problem: Often stored on local disk (not shared) Quick-Fix: ELB Session stickiness Solution: DynamoDB Elastic Load Balancing Web server Web server Logged in Logged out

30. Amazon DynamoDB Managed document and key-value store Simple to launch and scale • To millions of IOPS • Both reads and writes Consistent, fast performance Durable: perfect for storage of session data https://github.com/aws/aws-dynamodb-session-tomcat http://docs.aws.amazon.com/aws-sdk-php/guide/latest/feature-dynamodb-session-handler.html

31. Day 4 – Let’s go viral!

32. Replace guesswork with elastic IT Startups pre-AWS Demand Unhappy Customers Waste $$$ Traditional Capacity Capacity Demand AWS Cloud

33. Scaling the web tier Availability Zone a RDS DB instance ElastiC ache node 1 Availability Zone b S 3 bucket for static assets www.example.com Amazon Route 53 DNS service Elastic Load Balancing Web server Web server RDS DB standby ElastiC ache node 2

34. Scaling the web tier Availability Zone a RDS DB instance ElastiC ache node 1 Availability Zone b S 3 bucket for static assets www.example.com Amazon Route 53 DNS service Elastic Load Balancing Web server Web server RDS DB standby ElastiC ache node 2 Web server Web server

35. Scaling the web tier Availability Zone a RDS DB instance ElastiC ache node 1 Availability Zone b S 3 bucket for static assets www.example.com Amazon Route 53 DNS service Elastic Load Balancing Web server Web server RDS DB standby ElastiC ache node 2 Web server Web server

36. Automatic resizing of compute clusters based on demand Feature Details Control Define minimum and maximum instance pool sizes and when scaling and cool down occurs. Integrated to Amazon CloudWatch Use metrics gathered by CloudWatch to drive scaling. Instance types Run Auto Scaling for on-demand and Spot Instances. Compatible with VPC. aws autoscaling create-auto-scaling-group --auto-scaling-group-name MyGroup --launch-configuration-name MyConfig --min-size 4 --max-size 200 --availability-zones us-west-2c, us-west-2b Auto Scaling Trigger auto-scaling policy Amazon CloudWatch

37. Decompose into small, loosely coupled, stateless building blocks Prerequisite

38. What does this mean in practice? Only store transient data on local disk Needs to persist beyond a single http request? • Then store it elsewhere User uploads User Sessions Amazon S3 AWS DynamoDB Application Data Amazon RDS

39. Having decomposed into small, loosely coupled, stateless building blocks You can now Scale out with ease Having done that…

40. Having decomposed into small, loosely coupled, stateless building blocks We can also Scale back with ease Having done that…

41. Take the shortcut While this architecture is simple you still need to deal with: • Configuration details • Deploying code to multiple instances • Maintaining multiple environments (Dev, Test, Prod) • Maintain different versions of the application Solution: Use AWS Elastic Beanstalk

42. AWS Elastic Beanstalk (EB) Easily deploy, monitor, and scale three-tier web applications and services. Infrastructure provisioned and managed by EB You maintain control. Preconfigured application containers Easily customizable. Support for these platforms:

43. Loose coupling with SQS Tight coupling • Place asynchronous tasks into Amazon SQS • SQS – buffer that protects backend systems • Process at own pace • Respond quickly to end users S QS Get Message Back End EC 2 Instance Put Message Front End EC 2 Instance

44. Day 5 – Add Features

45. Mobile Push Notifications Mobile Analytics Cognito Cognito Sync Analytics Kinesis Data Pipeline RedShift EMR Your Applications AWS Global Infrastructure Network VPC Direct Connect Route 53 Storage EBS S3 Glacier CloudFront Database DynamoDBRDS ElastiCache Deployment & Management Elastic Beanstalk OpsWorks Cloud Formation Code Deploy Code Pipeline Code Commit Security & Administration CloudWatch Config Cloud Trail IAM Directory KMS Application SQS SWF App Stream Elastic Transcoder SES Cloud Search SNS Enterprise Applications WorkSpaces WorkMail WorkDocs Compute EC2 ELB Auto Scaling LambdaECS

46. Stay focused as you scale your team AWS Cloud-Based Infrastructure Your Business More Time to Focus on Your Business Configuring Your Cloud Assets 70% 30%70% On-Premise Infrastructure 30% Managing All of the “Undifferentiated Heavy Lifting”

47. Day 6 – Growing Fast

48. Scaling Relational DBs Increase RDS instance specs • Larger instance type • More storage / more PIOPS Read Replicas (Master – Slave) • Scale out beyond capacity of single DB instance • Available in Amazon RDS for MySQL, PostgreSQL and Amazon Aurora • Replication lag • Writes => master • Reads with tolerance to stale data => read replica (slave) • Reads with need for most recent data => master

49. Scaling the DB Web server Web server Web server Web server Availability Zone a RDS DB instance ElastiC ache node 1 Availability Zone b S 3 bucket for static assets www.example.com Amazon Route 53 DNS service Elastic Load Balancing RDS DB standby ElastiC ache node 2

50. Scaling the DB Web server Web server Web server Web server Availability Zone a RDS DB instance ElastiC ache node 1 Availability Zone b S 3 bucket for static assets www.example.com Amazon Route 53 DNS service Elastic Load Balancing RDS DB standby ElastiC ache node 2 RDS read replica

51. Scaling the DB Web server Web server Web server Web server Availability Zone a RDS DB instance ElastiC ache node 1 Availability Zone b S 3 bucket for static assets www.example.com Amazon Route 53 DNS service Elastic Load Balancing RDS DB standby ElastiC ache node 2 RDS read replica RDS read replica

52. What if your app is write-heavy? Challenge: You will eventually hit the write throughput or storage limit of the master node Solutions: Federation (splitting into multiple DBs based on function) Sharding (splitting one data set up across multiple hosts)

53. Database federation • Split up databases by function/purpose • Harder to do cross-function queries • Essentially delaying the need for something like sharding/NoSQL until much further down the line • Won’t help with single huge functions/tables Forums DB Users DB Products DB

54. Sharded horizontal scaling • More complex at the application layer • ORM support can help • No practical limit on scalability • Operation complexity/sophistication • Shard by function or key space • RDBMS or NoSQL User ShardID 002345 A 002346 B 002347 C 002348 B 002349 A Shard C Shard B Shard A

55. NoSQL data stores Trade query & integrity features of Relational DBs for • More flexible data model • Horizontal scalability & predictable performance DynamoDB Provisioned read/write performance per table

56. Massive and Seamless Scale Distributed system that can scale both reads and writes • Sharding + Replicas Automatic partitioning: • Data set size growth • Provisioned capacity increases table

57. Amazon Route 53 DNS serviceNo limit Availability Zone a RDS DB instance ElastiC ache node 2 Availability Zone b S 3 bucket for static assets www.example.com Elastic Load Balancing RDS DB standby ElastiC ache node 3 RDS read replica RDS read replica DynamoDB RDS read replica ElastiC ache node 4 RDS read replica ElastiC ache node 1 C loudS earchLambdaS ES S QS

58. Day 7 – Rest

59. Day 8 – Go Serverless!

60. Continuous scaling No servers to manage Never pay for idle – no cold servers (only happy accountants) Benefits of AWS Lambda

61. Amazon API Gateway: Serverless APIs Internet Mobile apps Websites Services AWS Lambda functions AWS API Gateway cache Endpoints on Amazon EC2 Any other publicly accessible endpointAmazon CloudWatch Amazon CloudFront Amazon API Gateway

62. Summary

63. A quick review Keep it simple and stateless Make use of managed self-scaling services Multi-AZ and AutoScale your EC2 infrastructure Use the right DB for each workload Cache data at multiple levels Simplify operations with deployment tools

64. Next steps? READ! aws.amazon.com/documentation aws.amazon.com/architecture aws.amazon.com/start-ups ASK FOR HELP! forums.aws.amazon.com aws.amazon.com/support

Add a comment

Related pages

How to Scale to Millions of Users with AWS - embedslide.net

Embed slide. Repository for embed slide from slideshare, speakerdeck or using HTML framework like reveal.js, desk.js, ... We are not responsible ...
Read more

Scaling on AWS for the First 10 Million Users - YouTube

Cloud computing gives you a number of advantages, such as being able to scale your application on demand. As a new business looking to use the ...
Read more

Scaling on AWS (Part 4) : > One Million Users — AWS ...

Scaling on AWS (Part 4) : > One Million Users. In the fourth and final part of this series, I want to show you how to scale an architecture to serve ...
Read more

AWS re:Invent 2015 | (ARC301) Scaling Up to Your First 10 ...

AWS re:Invent 2015 | (ARC301) Scaling Up to Your First 10 Million Users ... such as the ability to scale your web application or website on demand.
Read more

A Beginner's Guide to Scaling to 11 Million+ Users on ...

If you are an advanced AWS user this ... and then tell Auto Scaling you want to scale on that metric. Users ... a million users and above ...
Read more

Scaling on AWS (Part 3): >500K Users — AWS Startup ...

Scaling on AWS (Part 3): >500K Users. ... As your environment scales, ... Scaling on AWS (Part 4) : > One Million Users.
Read more

Netflix on how to build services that scale ... - TechRepublic

Netflix on how to build services that scale beyond millions of users. ... Millions of people worldwide stream more than two billion hours ... (AWS) cloud ...
Read more

A Beginner's Guide to Scaling to 11M+ Users on Amazon's ...

A Beginner's Guide to Scaling to 11M+ Users on ... to scale past 1B users. I was trialling AWS when I ... about scale with 100s of millions of ...
Read more