Published on February 29, 2008
Improving the Performance of your Web Application Joe Stump, Lead Architect, Digg.com
“Web 2.0 sucks (for scaling).” Joe Stump, Lead Architect, Digg.com Users want access to all of their crap at all times. I, personally, don’t ﬁnd your dog funny or cute, but I’ll be damned if I’m the one who’ll stand in the way of you posting it and others consuming it.
Backend Considerations Language considerations Scaling out Caching strategies Content storage and delivery Parallel data requests Near time data processing Partitioning data
“PHP doesn’t scale.” Cal Henderson, Director of Development, Flickr.com Languages don’t scale Bytecode caching (PHP, Python, etc) Robust library & driver support Active developer communities
Discussion! What language do you use? Why? Does it help you or hurt to use it?
Your mom lied; don’t share. Decentralize data, storage, processing, etc. Increased redundancy Scaling becomes simple; add more boxes
How do I scale easily? 1.Caching 2.Caching 3.Caching!
What are my options? Disk based caching (e.g. Cache_Lite) In memory caching (e.g. APC, Memcached) Cloud caching (e.g. MogileFS, S3)
Disk based caching Stupid simple Cheap Fairly easy to scale out Dynamic images Slower than others Use fast disks! RAM disks are faster
APC (PHP) Bytecode caching In memory user cache Insanely fast Not centralized or shared
Memcache If you’re not using this you’re crazy Easy to set up and use Insanely fast over the network Scales to insane heights Failover, widely supported, etc. Centralized and shared across site
Mogile FS File and data store Runs over WebDAV Scales out inﬁnitely (in theory) Serialize data, store in ﬁle Centralized and shared across site
Amazon S3 File and data store Runs over HTTP Scales out inﬁnitely (in theory) Serialize data, store in ﬁle Centralized and shared across site Costs money Widely supported in all languages Check out ThruDB
Discussion! Are you using caching? Why not? If so, what’s your strategy?
Content Storage/Delivery What are your storage needs? Is it critical YOU store them? How costly is it to store in-house? Can you do it for free? (YAY! Mooching!)
i can has free storage? YouTube for video Scribd for documents Flickr for images
Cloud Services (S3) Simple to get up and running No hardware maintenance Costs money, but not as much as you think
NFS Simple to set up and get running Costs money, requires colocation, etc. Does. Not. Scale. Did I mention it doesn’t scale? Stop gap solution at best
Mogile FS Somewhat complicated to set up Costs money, requires colocation, etc. Scales exceptionally well Used at Digg, LiveJournal, others Check out File_Mogile by Digg (PEAR)
Roll Your Own File storage IS your business Highly specialized and customized Costs money, requires colocation, etc. Last resort
CDN Completely outsource it Costs a ton of money Out of your control Scales and scales and scales
Discussion! What are you using for storage? What’s worked for you? What’s failed epically?
Parallel Data Requests Access your data in parallel Make data access asynchronous (WHAT?!) Loosely couple your data access layer All for the low, low price of FREE!* *Offer only available for hardcore nerds looking for street cred.
HTTP Parallel Asynchronous Non-blocking Loosely coupled Free foot massages!
Gearman Parallel Asynchronous Scales well
Discussion! Which format to use for exchange? Anyone doing this already? Amazon, Google,Yahoo!
Near time processing Does this need to be done NOW? Ofﬂoad to background processes Ofﬂoading must be a no op Feeds, Facebook, crawling, etc.
Cron Run every minute or two Simple Great for batch jobs Not decentralized, locking issues
Gearman Fire and forget Simple Scales well Digg Images Nearly instant Decentralized No guarantees
Queues Grid Engine by Sun Starling by Twitter Others?
Amzon’s EC2 http://open.blogs.nytimes.com/2007/11/01/self-service-prorated-super-computing-fun/ Near limitless computing resources Remember; don’t share Awesome for bots, crawling, etc.
Discussion! What’s low(er) priority? Where would you implement this?
Partitioning Data Horizontal v.Vertical Not all data lives in a single place Hash records to partitions App smart / logical sharding
Horizontal 192.168.0.1 192.168.0.2 192.168.0.3 Users Users Users id int(11) id int(11) id int(11) username char(15) username char(15) username char(15) password char(15) password char(15) password char(15) email char(45) email char(45) email char(45)
Hashing your data oh hai! were’s mai dataz?!
How? Put 10,000 users per partition Partition users alphabetically Partition home listings by zip code Partition products by SKU
Vertical 192.168.0.1 192.168.0.2 192.168.0.3 Users UsersPrf UsersStg id int(11) id int(11) id int(11) username char(15) fname char(50) cmts_pg tinyint(2) password char(15) lname char(50) cmts_lvl tinyint(1) email char(45) url char(255) cmts_prf tinyint(1)
Why? Avoid altering large tables Save time during insert Many small tables v. one large table Lazy loading of rarely used data
Discussion! Natural partitions in your data? How would you hash your data?
Avoid inline JS/CSS External = Cached Inline = Not Cached
Compression / Minify Enable Gzip compression sitewide Use miniﬁcation software on JS jQuery/Prototype Miniﬁed
Learn to Love HTTP/1.1 Cache-Control: public/private Connection: close Expires: Thu, 28 Feb 2008 16:00:00 GMT
Conclusions Share nothing, decentralize, redundancy Caching, caching, caching, caching Reduce, recycle and reuse
Contact/Flame Me Joe Stump email@example.com http://joestump.net
Introduction. HTML5 gives us great tools to enhance the visual appearance of web applications. This is especially true in the realm of animations.
... the factors that contribute to your web application's performance. ... in Improving performance of ... build your next great app ...
Improving Web App Performance With the Chrome DevTools Timeline and ... Measuring and improving the performance of your applications properly will take ...
I’ll be giving a workshop at FOWA in Miami, FL titled Improving the Performance of Your Web App on February 28th. If you’re going to be at FOWA or any ...
App Development; Web; Data; ... for managing performance and scalability throughout your application life cycle ... Improving Web Services Performance;
app performance: Java Experience: 12/22 ... ( improving performance,speed etc) r rare. You have written this ... clearly says "Improve Web Application ...
Improving Your Web App's Performance with Aggressive Data Caching. ... ASP.NET pages to help boost the performance of your Web apps without adding ...
Improving .NET Application Performance and Scalability provides an ... help integrate performance throughout your ... Services, Web services, remoting ...
Web application performance depends on Internet capacity. Money flows into the networks from the first and last miles, as companies pay for hosting and end ...