Varnish Cache - step by step

50 %
50 %
Information about Varnish Cache - step by step

Published on February 25, 2014

Author: perbu



Varnish Cache Step by step

Ins e rt bu sin es sl og ic he re

About Varnish Cache ● Web app accelerator ● Fast ● Flexible

VCL • • Varnish Configuration Language • • Hook custom logic into the flow Applies to transactions that flow through the cache Main hooks used: vcl_recv and vcl_fetch

About Varnish Software ● The company behind Varnish Cache ● Offers “Varnish Plus” - consisting of: ○ Additional software ○ 24/7 support ○ Professional services

Goal Knowing what to do to speed up your website.

Designing a web app for Varnish ● Build apps from the ground up for Varnish ● Intelligent cache usage ● Proper cache invalidation ● Monitoring

Incremental implementations ● Deploy Varnish on existing apps ● Gradually increase caching ● “Low hanging fruits” first

Requirements ● Linux or FreeBSD ● Virtualization is OK, not awesome ● Tuning is only for high traffic

Step 1: Installing Varnish Difficulty level: Easy
 Payoff: Minor

Step 1: Pass through only ● Passive Varnish ○ return(pass) in vcl_recv ● Powerful load balancing ● Health checks built in ● Excellent for debugging webapps

Step 2: Cache static assets Difficulty level: Trivial
 Payoff: Minor

Step 2: Cache static assets ● Cache images, CSS, Javascript ● Backend concurrency decrease ● Backend bandwidth decrease ● Minor CPU usage decrease

Step 3: Compression Difficulty level: Trivial
 Payoff: Medium

Step 3: Compression ● GZip compress CSS/Javascript ○ JQuery standard: 263KB ○ JQuery minified: 91KB ○ JQuery gzipped: 32KB ● Compress in one place only ● Reuse compression ● Move CPU load from backend

Step 4: Semi-static caching Difficulty level: Medium
 Payoff: High

Step 4: Semi-static content caching ● Dynamically rendered content ● Expensive to render ● Almost never changes ● Set short TTL (10-120 seconds) ● Mitigates traffic peaks

Step 4: Semi static (2/2) ● Manual cache expiry ● Magic URLs ● Shift-refresh from special IP-range ● Web interface for ban or purge

Step 5: Autopurge Difficulty level: Medium
 Payoff: High

Step 5: Autopurge ● Backend generates purge requests ● Set TTL to ~forever ● Invalidate on change ● Can be difficult for composite pages ○ Map relationships between data and pages

Step 6: Grace mode Difficulty level: Easy
 Payoff: High

Step 6: Grace mode • Originally developed to mitigate pile-ups
 • Can serve “graced” content when fresh content is unavailable
 • Unavailable means • • backend down object busy

Grace mode (2) • Both req.grace and beresp.grace must accept the old object • Tip: Set req.grace to 10s normally and 1 hour when backend is down

Step 7: Content composition with ESI Difficulty level: Hard 
 Payoff: High

Edge Side Includes (ESI) Horrible Designed News Site Blah Article snippet 1 Welcome, Foobar (logout) Article snippet 2 Top 10 popular news items Article snippet 3 Article snippet 4

Edge Side Includes (ESI) TTL: 1800minutes TTL: 30min TTL: 60min, user specific TTL: 30min TTL: 1min TTL: 30min TTL: 30min

Step 7: Content composition ● Edge Side Includes (ESI) ● Split web pages into pieces ○ Individual caching policies ○ One piece = one URL ● Live stitching of content ● Works with compressed content as well

<html><body> ... <esi:include src="/frontpage/ breaking" /> ... <esi:include src="/frontpage/latest" /> ... <!-- Do we like the cache to meddle with content? --> </body><html>

Step 8: Advanced cache invalidation Difficulty level: Hard
 Payoff: High

Step 8: Advanced cache invalidation ● Sometimes a page relies on multiple “objects” for invalidation ● Finding out what object need to expire what pages can be hard ● Typically a problem in e-commerce and media

Advanced cache invalidation (2/2) ● Tag pages with custom headers ○ X-Skus: 49834, 34321, 4343 ● Execute with bans or hashninja ● Possible database driven expiry (triggers)

Conclusions ● High rewards with little work ● 3 - 15 days to reach “high” payoff ● Requires backend knowledge to do significant changes ● We talked about what to do - if you want to learn more about how please read the book or sign up for training.

Q &A

Add a comment

Related presentations

Presentación que realice en el Evento Nacional de Gobierno Abierto, realizado los ...

In this presentation we will describe our experience developing with a highly dyna...

Presentation to the LITA Forum 7th November 2014 Albuquerque, NM

Un recorrido por los cambios que nos generará el wearabletech en el futuro

Um paralelo entre as novidades & mercado em Wearable Computing e Tecnologias Assis...

Microsoft finally joins the smartwatch and fitness tracker game by introducing the...

Related pages

Webinar form - Caching with Varnish, step-by-step

... our on-demand webinar, Caching with Varnish step-by-step, is for you. ... What is Varnish Cache and what is caching? Steps to get caching; What's in it ...
Read more

Step-by-step: Speed Up Wordpress with Varnish Software

Step-by-step: Speed Up Wordpress with Varnish Software. Blog ; Contact ... For other operating systems check the latest releases on ...
Read more

How To Install and Configure Varnish with Apache on Ubuntu ...

deb lucid varnish-3.0. Save and exit. ... sudo apt-get update sudo apt-get install varnish Step Two—Configure Varnish.
Read more

Step-by-step: Setting up Project Mercury (Varnish, Apache ...

Step-by-step: Setting up Project Mercury (Varnish, Apache, APC, ... we recommend using the /mnt directory for storage of Mysql files and the Varnish cache.
Read more

Step by step guide to install and integrate Varnish with ...

Step by step guide to install and integrate Varnish with Plesk 12.x. ... I try intergate Plesk with the Varnish cache, with no luck... my problem is, ...
Read more

[4.1/master] clock-step related crash · Issue #1874 ...

Expected Behavior In general child process restart is problematic, because we are loosing whole cache, which can lead to efficiency problems under high load.
Read more

How to Install Varnish 4 on CentOS 7 | Liquid Web ...

How to Install Varnish 4 on ... Step 3: Start Varnish. ... account management apache apache2 cache caching centos centos 6 centos 7 cli command line core ...
Read more

How to Install and Configure Varnish on CentOS? | JesoX

How to Install and Configure Varnish on ... What is Varnish? Varnish is a HTTP proxy cache or an HTTP accelerator designed to cache http requests to ...
Read more