Published on March 5, 2014
Store and move your data to & from the AWS cloud Markku Lepistö Senior Technology Evangelist firstname.lastname@example.org @markkulepisto
Agenda • What’s the Problem? • Data Transfer • Data Stores • Backup and DR • What’s Next?
We are constantly producing more data
Pressure on Traditional On-Premise Solutions Primary Block Storage Primary File Storage Archival Storage Disk Based Backup Storage Tape Infrastructure & Management Complex / Bri5le Siloed / Expensive Capacity Guessing Replicated Storage for Disaster Recovery Offsite Locations GeoResilience
Traditional Backups • Long/slow recovery time • Low durability (higher probability for data loss) • Requires Tape Museums • Expensive Tech Refresh
AWS Storage Services Help?
AWS Global Infrastructure 10 Regions 25 Availability Zones 51 Edge Locations
AWS Direct Connect AWS Import/Export Dedicated bandwidth between your Physical transfer of media into and site and AWS out of AWS When -‐ if ever -‐ will the bandwidth of the Internet surpass that of FedEx? —Johan Öbrink
AWS Storage Options ElasDc Block Store Simple Storage Service Glacier High performance block storage device Highly scalable object storage Long term object archive 1GB to 1TB in size, max 4000 IOPS/volume Mount as drives to instances 1 byte to 5TB in size 99.999999999% durability 1c/GB/month, $120/TB/year 99.999999999% durability with snapshot/cloning funcOonaliOes Very fast mounted volumes Fast object storage Slow, infrequent archive
Storage Tiers & Automated Life-Cycle
SpoDfy adds over 20,000 tracks a day Reduced Redundancy with ~20% savings
Raw Storage at On perm vs. Cloud 1PB Raw Storage 800TB Usable Storage 600TB Allocated Storage 400TB Wri5en ApplicaDon Storage S3 & Glacier – Only actual usage is charged
• nearmap routinely captures hi-res aerial images covering 75% of Australia’s population • nearmap provides browser-based access to up-to-date and historical aerial imagery to thousands of companies, large and small 16
“Only Amazon Glacier had the scale nearmap required to store thousands of terabytes of raw PhotoMap and digital images.” “Without Amazon Glacier’s secure and scalable archival solution, it would have been cost prohibitive for us to keep these images, which are important business IP.” - Paul Peterson VP of Engineering
Access via Console, SDKs and Third-party Solutions Via SDK Via 3rd parties Internet Web Services API HTTP(S) S3 Glacier
AWS Storage Gateway for offsite disk backups Gateway-Cached Volumes Your Datacenter iSCSI Production Systems Backup Servers AWS Storage Gateway VM On-premises Host Direct Attached or Storage Area Network Disks SSL AWS Storage Gateway Service Amazon Simple Storage Service (S3)
AWS Storage Gateway for departmental file shares Gateway-Stored Volumes Your Datacenter CIFS/ NFS iSCSI File servers AWS Storage Gateway VM On-premises Host End Users Direct Attached or Storage Area Network Disks SSL AWS Storage Gateway Service Amazon Simple Storage Service (S3)
AWS Storage Gateway - Virtual Tape Library
Partner Solutions • • • • • CIFS/NFS Local PresentaDon Appliance or Virtual Machine In-‐Line De-‐DuplicaDon Customer Retained Security Keys Local Cache for recently archived data
Partner Solutions It’s easy to backup Oracle DBs to Amazon S3 Read the case Amazon.com study Corporate data center Oracle RMAN Oracle Secure Backup Module S3
Choosing the Data Store
Structure Low High High High Low Low Amazon Glacier Amazon S3 Amazon ElasDCache Amazon DynamoDB Am Clou azon dSea rch Amazon EMR Amazon RDS Amazon Redshic Request rate Cost/GB Latency Data Volume Low Low High High
cost, volume, access pattern structure
Recovering from Disasters
Backup and restore Pilot light Warm standby Hot standby
cost speed of recovery
Backup the DB
Backup Server Images
Restore DB and App Servers Create instances from AMIs Restore data from backups
Next Generation Enterprise Storage Customer Data Center Block File AWS Direct Connect AWS Cloud Archive Backup Internet Web Services API HTTP(S) Disaster Recovery Storage Use Cases
Tradi8onal: On-‐Premise Solu8ons Primary Block Storage Primary File Storage Archival Storage Disk Based Backup Storage Tape Infrastructure & Management Replicated Storage for Disaster Recovery Oﬀsite LocaDons Geo-‐ Resilience Next Genera8on: Gateway & Cloud Storage Reduce SAN Footprint Reduce NAS Footprint Eliminate Archival Hardware & Socware Reduce Complexity. Eliminate Backup Hardware & Socware Eliminate Tape Recover in Cloud or any Data Center Reduce Cost. Eliminate Secondary Data Centers
“AWS allows us to store informaDon in a cost eﬀecDve manner. It really is a win-‐win for us and our customers.” -‐ Jeﬀ Kimsey, Associate Vice President The Story • The Global Data Products division of NASDAQ Business Beneﬁt • $50 the ﬁrst month OMX uses detailed historical stock price data for its Market Replay and Data-‐On-‐Demand AWS is a low cost soluDon that only cost • Avoided the need to buy expensive hardware products • and improved product Dme Wanted to support their growing customer to market base without worrying about infrastructure, • informaDon to the millisecond security, or performance • Needed to be able to increase storage space Can store ten years of historical stock price • Able to upload hundreds of thousands of data and keep data indeﬁnitely • ﬁles to the AWS cloud and deliver informaDon to Using Amazon S3 customers in seconds
data abandoned information actionable
Business Value From The Data Managed DBs Amazon DynamoDB Delivery Amazon CloudFront Transcoding Amazon ElasDc Transcoder Workﬂow Amazon SWF AnalyDcs Amazon ElasDc MapReduce Discovery (OR) Glacier Amazon Cloudsearch DR, Archive, Backup and Compute Amazon VPC Amazon EC2 Amazon EBS
Benefits of AWS Storage Financial – – – – Significant reduction in capital investments Pay for only what you use: Metered Utility Eliminate the need to over purchase AWS prices continue to fall over time Operational – Streamline and simplify data management operations – Reduction of systems and data centers – Focus employees on proactive and innovative tasks Business – Never stop the business as capacity is endless – Never slow the pace of innovation – Turn IT into an innovation enabler
Contact Your AWS Account Manager To discuss your use cases, opportunities to try AWS storage services And follow us on at @AWSCloudSEAsia
Thank you Markku Lepistö – Senior Technology Evangelist @markkulepisto
Backup & Recovery Store and retrieve any data, ... coaching on how to build your mobile app on the AWS Cloud. ... the AWS C-SDK for IoT by Markku Lepisto:
... "http://www.slideshare.net/AmazonWebServices/coscup-aws-markku-lepisto-v2 ... to move your collections and ... Data","Cloud & Backend technology ...
... (Infocomm Development Authority Singapore, Data ... of your Data in the Cloud. ... with AWS Lambda, at any scale. Markku Lepisto ...
Journey Through the AWS Cloud - Big Data Analysis
Journey through the cloud security best ... - slidesearch.net
Cloud Asia 2014. Cloud Asia 2014. Browse Browse. Interests. Biography & Memoir; Business & Leadership; Fiction & Literature; Politics & Economy; Health ...
... or whatever that floats your boat. ... no_free_lunch jesseva31 r_messemmore egmservices slothcore rekebra the13thvictim wayne ferry shamrockrebel ...