2013 - Matías Paterlini: Escalando PHP con sharding y Amazon Web Services

67 %
33 %
Information about 2013 - Matías Paterlini: Escalando PHP con sharding y Amazon Web Services

Published on January 29, 2014

Author: PHPConferenceArgentina

Source: slideshare.net

Description

PHP Conference Argentina 2013

Escalando una PHP App con DB Sharding & Amazon Web Services Matias Paterlini Tuesday, November 5, 13

Objetivo de la charla • Contar quien soy y qué hago • Mostrar orígenes de la arquitectura, infraestructura y escalabilidad de TFM • Hablar de DB Sharding • Hablar de Amazon • Preguntas • Cof Cof reclutar Ejem! Tuesday, November 5, 13

• +11 años de experiencia en PHP. • 6 años desarrollando aplicaciones sobre Redes Sociales • 25 empleados, + de 250 aplicaciones desarrolladas • + 10K clientes, + 10M Usuarios • 2 compañías construidas sobre plataformas sociales Tuesday, November 5, 13

Tuesday, November 5, 13

Tuesday, November 5, 13

Tuesday, November 5, 13

Tuesday, November 5, 13

Tuesday, November 5, 13

Tuesday, November 5, 13

Tuesday, November 5, 13

Tuesday, November 5, 13

Tuesday, November 5, 13

Tuesday, November 5, 13

Tuesday, November 5, 13

Tuesday, November 5, 13

Tuesday, November 5, 13

Tuesday, November 5, 13

Tuesday, November 5, 13

Tuesday, November 5, 13

Tuesday, November 5, 13

Tuesday, November 5, 13

Tuesday, November 5, 13

Tuesday, November 5, 13

Tuesday, November 5, 13

Tuesday, November 5, 13

Tuesday, November 5, 13

Tuesday, November 5, 13

Tuesday, November 5, 13

Tuesday, November 5, 13

Tuesday, November 5, 13

Tuesday, November 5, 13

20.6 M Audiencia de nuestros primeros 8 clientes. Tenemos 250 en simultáneo. Tuesday, November 5, 13

+300.000 Fans conseguidos con solo un concurso Tuesday, November 5, 13

Todos tenemos una historia con PHP Tuesday, November 5, 13

Tuesday, November 5, 13

Tuesday, November 5, 13

<?php echo 'Hello World'; ?> Tuesday, November 5, 13

? Tuesday, November 5, 13

Apache HTTP Server Tuesday, November 5, 13

Tuesday, November 5, 13

Free Web Hosting Tuesday, November 5, 13

Tuesday, November 5, 13

Tuesday, November 5, 13

OOP Tuesday, November 5, 13

<? class Caja{     var $alto;     var $ancho;     var $largo;     var $contenido;     var $color; function introduce($cosa){     $this->contenido = $cosa; } function muestra_contenido(){     echo $this->contenido; } } ?> Tuesday, November 5, 13

Auto-complete Object class reference Tuesday, November 5, 13

Tuesday, November 5, 13

FRAMEWORKS Tuesday, November 5, 13

Tuesday, November 5, 13

2010 Tuesday, November 5, 13

The Fan Machine Tuesday, November 5, 13

Esto tiene que escalar! Tuesday, November 5, 13

¿Qué sabíamos de escalar algo en PHP? Tuesday, November 5, 13

¿Qué sabíamos de escalar algo a MILLONES de usuarios en PHP? Tuesday, November 5, 13

0 Tuesday, November 5, 13

CERO Tuesday, November 5, 13

Qué onda con el NoSQL? Tuesday, November 5, 13

Qué onda con el Sharding? Tuesday, November 5, 13

Cuales eran nuestros posibles problemas? Tuesday, November 5, 13

Escalar Rápido Tuesday, November 5, 13

Muchos writes Tuesday, November 5, 13

Mucha variación en el Tráfico Tuesday, November 5, 13

MUCHOS cambios en el modelo de datos Tuesday, November 5, 13

Mucho Storage Tuesday, November 5, 13

Todo bajo SSL Tuesday, November 5, 13

Startup = poco $$ Tuesday, November 5, 13

Como lo solucionamos??? Tuesday, November 5, 13

Possible solutions... Tuesday, November 5, 13

Web Server & Storage Tuesday, November 5, 13

HORIZONTAL SCALABILITY Tuesday, November 5, 13

Static File Caching Web Server 1 Web Server 2 Web Server 3 Data Caching Database 1 Database 1 (replicated DB) Tuesday, November 5, 13

Amazon Azure GoGrid Rackspace Vurbia - SoftLayer Akamai Tuesday, November 5, 13

AMAZON Tuesday, November 5, 13

• S3 • EC2 - Web & Memcache • CloudWatch • CloudFront • ELB - SSL termination • Route 53 Tuesday, November 5, 13

What about the DB? Tuesday, November 5, 13

MongoDB Membase Hbase Cassandra SimpleDB NoSQL BigData CouchDB Hipertable RaptorDB Tuesday, November 5, 13

MongoDB Membase Hbase Cassandra SimpleDB NoSQL BigData CouchDB Hipertable RaptorDB Tuesday, November 5, 13

NoSQL = No Trust! Tuesday, November 5, 13

Y MySQL qué onda? Tuesday, November 5, 13

Horizontal Scalability Tuesday, November 5, 13

Master/Slave Tuesday, November 5, 13

Static File Caching Web Server 1 (Transaccional, read-write) Master Database Web Server 2 Web Server 3 Data Caching Slave Database (read) Tuesday, November 5, 13 Slave Database (read)

DB scalling issues... Tuesday, November 5, 13

Static File Caching Web Server 1 (Transaccional, read-write) Master Database Web Server 2 Web Server 3 Data Caching Slave Database (read) Tuesday, November 5, 13 Slave Database (read)

Tuesday, November 5, 13

CPU Memory Disk Tuesday, November 5, 13

Static File Caching Web Server 1 (Transaccional, read-write) Master Database Web Server 2 Web Server 3 Data Caching Slave Database (read) Tuesday, November 5, 13 Slave Database (read)

Retraso Tuesday, November 5, 13

Tuesday, November 5, 13

• MAAAAANY Records - Storage issues Tuesday, November 5, 13

• MAAAAANY Records - Storage issues • Caching not enough Tuesday, November 5, 13

• MAAAAANY Records - Storage issues • Caching not enough • Alters lock tables for hours! Tuesday, November 5, 13

• MAAAAANY Records - Storage issues • Caching not enough • Alters lock tables for hours! • Maintenance Issues Tuesday, November 5, 13

• MAAAAANY Records - Storage issues • Caching not enough • Alters lock tables for hours! • Maintenance Issues • Too much response time Tuesday, November 5, 13

Cluster Computing Tuesday, November 5, 13

Tuesday, November 5, 13

Tuesday, November 5, 13

Table Partitioning Tuesday, November 5, 13

Tuesday, November 5, 13

THE approach Tuesday, November 5, 13

The Friendfeed approach Tuesday, November 5, 13

MySQL Tuesday, November 5, 13

Sharded MySQL databases Tuesday, November 5, 13

MySQL emulating NoSQL over Sharded Databases Tuesday, November 5, 13

Documents = Entities Tuesday, November 5, 13

One Table for entities, many tables for indexes. Tuesday, November 5, 13

The Model index_user index_promo entities index_participant index_vote index_fan_conversion Tuesday, November 5, 13

The Model index_user index_promo entities index_participant index_vote index_fan_conversion Tuesday, November 5, 13

entities table added_id id body class created updated deleted Tuesday, November 5, 13

Entities Table CREATE TABLE `entities` ( `added_id` int(11) NOT NULL AUTO_INCREMENT, `id` char(32) NOT NULL DEFAULT '', `updated` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, `deleted` timestamp NULL DEFAULT NULL COMMENT `body` mediumtext, `class` varchar(50) DEFAULT NULL, `created` timestamp NOT NULL DEFAULT '0000-00-00 00:00:00', PRIMARY KEY (`added_id`), UNIQUE KEY `id` (`id`), KEY `updated` (`updated`), KEY `class` (`class`), KEY `created` (`created`) ) ENGINE=InnoDB AUTO_INCREMENT=1692491 DEFAULT CHARSET=utf8 | Tuesday, November 5, 13

users index table entity_id fb_uid Tuesday, November 5, 13

user index table CREATE TABLE `index_user` ( `entity_id` char(32) NOT NULL DEFAULT '', `uid` bigint(20) NOT NULL, PRIMARY KEY (`entity_id`,`uid`), UNIQUE KEY `uid` (`uid`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8 Tuesday, November 5, 13

How do we scale it? Tuesday, November 5, 13

File Caching Web Server 1 Web Server 2 Web Server 3 Data Caching DB Server 1 Shard 1 Shard 4 Shard 7 Shard 10 Shard 2 Shard 5 Shard 8 Shard 11 Shard 3 Shard 6 Shard 9 Shard 12 Database Backups Tuesday, November 5, 13 } Databases

Static File Caching Web Server 1 Web Server 2 Web Server 3 Web Server 3 DB Server 1 DB Server 2 DB Server 3 DB Server 4 Shard 1 Shard 2 Shard 3 Shard 4 Shard 5 Shard 6 Shard 7 Shard 8 Shard 9 Shard 10 Shard 11 Shard 12 Database Backups per Shard Tuesday, November 5, 13

THE Benefits... Tuesday, November 5, 13

• Bases de datos más pequeñas son más fáciles de manejar • Bases de datos más chicas son más rápidas • El sharding puede reducir costos Tuesday, November 5, 13

• Server maintenance per db-server • Data Maintenance Per db-shard • NO ALTERS on tables • Easy recovery and replication with MultiAZ • Easy scaling • Easy migration to other No-SQL data engines Tuesday, November 5, 13

¿When is DB sharding Appropiate? Tuesday, November 5, 13

High transaction Applications Write-intensive transactions Tuesday, November 5, 13

Cómo repartimos entre los shards? Tuesday, November 5, 13

Driver Programmatic Tuesday, November 5, 13 DB

DB Driver DB Server 1 DB Server 2 DB Server 3 DB Server 4 Shard 1 Shard 2 Shard 3 Shard 4 Shard 5 Shard 6 Shard 7 Shard 8 Shard 9 Shard 10 Shard 11 Shard 12 memory cache Driver DB Tuesday, November 5, 13

DB driver table shard_id email Tuesday, November 5, 13

Progamatic Driver Tuesday, November 5, 13

Cómo usamos el driver Fetchear la entidad de un usuario: Tuesday, November 5, 13

Cómo usamos el driver Crear o actualizar una promo: Tuesday, November 5, 13

otra forma de guardar... Tuesday, November 5, 13

y se utiliza así en un helper: Tuesday, November 5, 13

Amazon Web Services Tuesday, November 5, 13

Tuesday, November 5, 13

Tuesday, November 5, 13

Tuesday, November 5, 13

Tuesday, November 5, 13

Tuesday, November 5, 13

Tuesday, November 5, 13

Tuesday, November 5, 13

Como evolucionamos en Amazon... Tuesday, November 5, 13

Un servidor LAMP, S3 DNS externo Tuesday, November 5, 13

Amazon Linux AMI Tuesday, November 5, 13

Memcache Tuesday, November 5, 13

Balancer Tuesday, November 5, 13

Route 53 Tuesday, November 5, 13

Que aprendimos?? Tuesday, November 5, 13

OJO con las public AMIs!!! Tuesday, November 5, 13

Cuando estopeas una instancia.... CHAN Tuesday, November 5, 13

PERDES LA IP ELASTICA Tuesday, November 5, 13

SE TE CAMBIA LA DNS PUBLICA Tuesday, November 5, 13

Clonar es super simple pero no super rápido Tuesday, November 5, 13

Tener stopeada una instancia sale.... casi nada!! Tuesday, November 5, 13

DNS... Tuesday, November 5, 13

SSL Termination Tuesday, November 5, 13

Límite de certificados de ssl Tuesday, November 5, 13

Questions? Matias Paterlini matias@thefanmachine.com Te estamos buscando!! jobs@thefanmachine.com Tuesday, November 5, 13

Add a comment

Related presentations

Related pages

Matías Paterlini: Escalando PHP con sharding y Amazon Web ...

Matías Paterlini: Escalando PHP con sharding y Amazon Web Services ... Autoscaling con Amazon Web Services EC2 ... Matías Paterlini ...
Read more

PHP Conference Argentina 2013 - Day 1 - YouTube

A short look at the first day of PHP Conference Argentina 2013. ... Matías Paterlini: Escalando PHP con sharding ... Una mirada a PHP 5.4 y ...
Read more

Escalando una PHP App con DB sharding - PHP Conference ...

Escalando una PHP App con DB sharding ... Escalando una PHP App con DB Sharding & Amazon Web Services Matias ... Desarrollo de aplicaciones web con PHP y ...
Read more

Argentina Services - World News

CEIBAL 2013- English Services Argentina, Córdoba, Eco-Stim: Growing Oilfield Services Company Operating in Argentina, Catering Services Miami: ...
Read more

Web Services Con PHP - Manual Completo - Documents

15/6/2015 Web Services con PHP ... Escalando PHP con sharding y Amazon Web Services PHP Conference Argentina 2013
Read more

Entendiendo Web Services con MonoDevelop - Documents

... como crear y utilizar los Web Services con Mono ... Matías Paterlini: Escalando PHP con sharding y Amazon Web Services PHP Conference Argentina 2013
Read more

C# and .NET MongoDB Driver ¶

Driver Features¶ MongoDB Driver An updated .NET driver offering a full asynchronous stack. For documentation on the update .NET driver, see MongoDB C# ...
Read more