Amazon S3 storage engine plugin for MySQL

53 %
47 %
Information about Amazon S3 storage engine plugin for MySQL

Published on October 31, 2007

Author: kapil

Source: slideshare.net

A Storage Engine for Amazon S3 How to store a hundred billion BLOBs without buying a single disk.

Who is this guy?

http://fallenpegasus.com/code

Storage Engine

Protocol Translator

Traditional Storage Engines use the local disk

Network Storage Engines

Federated ODBC HTTP MemCacheD and...

S3

What is S3?

Petabytes?!

This can't be free

Why do I want to use it?

“An empty disk costs the same as a full one.”

Can I move my existing database over?

Translating S3 to MySQL AWS Account → CREATE SERVER • S3 Bucket → Table • S3 Item → Row • S3 Item Key → VARCHAR PRIMARY KEY • S3 Item Contents → BLOB or VARCHAR •

CREATE SERVER 'MyAWSAcct' FOREIGN DATA WRAPPER 'AWS' OPTIONS (USER 'aws id string', PASSWORD 'aws secret string');

CREATE TABLE 'bierce' ( 'word' VARCHAR(255) NOT NULL PRIMARY KEY, 'defn' BLOB) CHARSET=utf-8 ENGINE=AWSS3 CONNECTION='awss3 DevilDictionary $server MyAWSAcct';

SELECT defn FROM bierce WHERE word='WIT';

SELECT defn FROM bierce WHERE word='WIT'; WIT, n. The salt with which the American humorist spoils his intellectual cookery by leaving it out.

INSERT INTO bierce (word, defn) VALUES ('AUTHOR', 'One noted for confusing bitterness with humor.');

DELETE FROM bierce WHERE word='AUTHOR';

So, what's it good for?

“If you build it, they will come.”

Saving EC2 work • SQL CMS for S3 virtual hosting • Huge list of persistent primary keys • Big slow BLOBs, to join against fast local tables. • “The Image Server Problem” • ... and more •

Transactions?

What do you mean, “No temporal guarantees”?!

No WHERE clause means “MAKE MONEY FAST” for Amazon

The Future (watch me wave my hands)

Code Improvement • S3 & HTTP Metadata • Multiple Data Columns • Information Schema • Security & Authentication • Transfer & Storage Compression • Streaming BLOB • More Storage Engines •

Code Improvement

S3 & HTTP Metadata

Multiple Data Columns

Sharing a solution with HTTP and MemCacheD Engines

Information Schema

Security & Authentication

Transfer & Storage Compression

(new slide) Streaming BLOBs like PBXT

More storage engines

More storage engines “A Storage Engine for AWS SQS” • There are more AWS services rolling out • Replication distribution via S3 • Cluster persistent storage via S3 • ... and the EC2 holy grail: •

“A Generic Schema Storage Engine for S3”

Back to the present...

“Where do I get it?”

“Where do I get it?” MySQL 5.1 • GPL • http://fallenpegasus.com/code •

“You never write, you never call...”

Suggestions • Use Cases • Bug Reports • Patches • Feature Requests • Support Requests • Gigs & Proposals • mark@fallenpegasus.com

Questions?

Add a comment

Related pages

Amazon Simple Storage Service (S3) - Cloud Storage

Amazon RDS Managed Relational Database Service for MySQL ... 3D Game Engine, ... Amazon S3 offers a range of storage classes designed for ...
Read more

Amazon S3 Storage Engine for MySQL - Pete Freitag

Amazon S3 Storage Engine for MySQL. ... I did some quick googling but I couldn't find the plugin, ... I'm the author/presenter for the mysql s3 storage ...
Read more

Amazon S3 storage engine plugin for MySQL - Technology

1. A Storage Engine for Amazon S3How to store a hundred billion BLOBs without buying a single disk. 2. Who is this guy?
Read more

Plug-in enables MySQL plus Amazon's S3 for cheap database ...

Plug-in enables MySQL plus Amazon's S3 ... store their data on Amazon.com Inc.'s Web storage service, S3. ... to serve as the MySQL storage engine on ...
Read more

WordPress on S3 – YAPIXX | OblakSoft

The Cloud Storage Engine for MySQL ... that upload files to Amazon S3? WP2Cloud plugin for ... on Amazon cloud storage. WordPress on S3: ...
Read more

A MySQL Storage Engine for AWS v.S3 0.06 - WinSite

A MySQL Storage Engine for AWS S3 project is a plugin storage engine that allows MySQL to access Amazon Web Services Simple Storage Service (AWS S3 ...
Read more

WordPress › Support » WordPress can now run on Amazon S3

OblakSoft is pleased to enable the possibility of running WordPress websites on Amazon S3. With the Cloud Storage Engine for ... so to the MySQL plugin.
Read more

17.1 Using MySQL within an Amazon EC2 Instance

The InnoDB Storage Engine. ... 17.1 Using MySQL within an Amazon EC2 Instance ... and upload the AMI to the Amazon Simple Storage Service (S3).
Read more

MySQL on Amazon RDS - Amazon Relational Database Service

Amazon RDS fully supports the InnoDB storage engine for MySQL DB instances. Amazon RDS features such as ... //s3 .amazonaws.com/rds ... MySQL on Amazon RDS ...
Read more