SCALE12X Build a Cloud Day: Chef: The Swiss Army Knife of Cloud Infrastructure

25 %
75 %
Information about SCALE12X Build a Cloud Day: Chef: The Swiss Army Knife of Cloud...

Published on February 23, 2014

Author: mattray



Chef is an open source configuration management and automation framework used to configure, deploy and manage infrastructure of every type. Deploying to the cloud has made it easy to run large numbers of
servers and Chef makes it even easier to deploy to nearly every public and private cloud platform as well as virtualized and physical servers. This talk will provide a quick introduction to Chef and is intended for sysadmins and developers familiar with the concepts behind managing applications and infrastructure in the cloud, without diving too deeply into technical specifics.

Chef: The Swiss Army Knife of Cloud Infrastructure Matt Ray SCALE Build a Cloud Day February 21, 2014

Introductions • Matt Ray • Director of Cloud Integrations at Chef • • mattray GitHub|IRC| Twitter

We have a problem…


Items of Manipulation (Resources) • Nodes • Networking • Files • Directories • Symlinks • Mounts • Routes • Users • Groups • Packages • Services • Filesystems

A tale of growth... Application

Add a database Application Application Database

Make database redundant Application App Databases

Application server redundancy App Servers App Databases

Add a load balancer App LB App Servers App Databases

Webscale! App LBs App Servers App Databases

Now we need a caching layer App LBs App Servers App DB Cache App DBs

Infrastructure has a Topology App LBs App Servers App DB Cache App DBs

Your Infrastructure is a Snowflake Round Robin DNS App Servers App DB Cache Floating IP? App DBs

Complexity Increases Quickly App LBs Cache App Servers DB Cache < Shiny! A we re nito mo g?? rin DB slaves DBs

The Chef Framework • Reasonability • Flexibility • Library & Primitives • TIMTOWTDI • Sane defaults

The Chef Tool(s) • ohai • chef-client • chef-shell • knife • The Ruby language Om nibu s-F Nati ull S ve P ta c k ack age s

The Chef API • HTTPS, RESTful API w/ JSON, RSA key auth • Infrastructure data store such as node data • Search Service • Derivative Services?

The Chef Community • Apache License, Version 2.0 • Thousands of Individual and Corporate contributors. • Thousands of cookbooks available from the community •

How does it work?

Chef is Infrastructure as Code • Programmatically provision and configure • Treat like any other code base • Reconstruct business from code repository, data backup, and bare metal resources.

Programs • Chef generates configurations directly on nodes from their run list • Reduce management complexity through abstraction • Store the configuration of your programs in version control

Declarative Interface to Resources • Define Policy • Say what, not how • Pull not Push

That Looks Like This package "apache2" template "/etc/apache2/apache2.conf" do! source "apache2.conf.erb"! owner "root"! group "root"! mode "0644"! variables(:allow_override => "All")! notifies :reload, "service[apache2]"! end service "apache2" do! action [:enable,:start]! supports :reload => true! end

Chef Nodes • Chef runs on nodes • Chef nodes do the heavy lifting • Authority about themselves • Stored on the server when using Chef Server • Indexed for search

Search • Search for nodes with Roles • Find Topology Data ! • IP addresses • Hostnames • FQDNs

So when this... Graphite Nagios Jboss App Memcache Postgres Slaves Postgres Master

...becomes this Graphite Nagios Jboss App Memcache Postgres Slaves Postgres Master

...this can happen automatically Graphite Nagios Jboss App ! ! ! Memcache ! ! Postgres Slaves Postgres Master

Count the Resources • Graphite Nagios • • • Jboss App ! ! • ! • Memcache ! • ! Postgres Slaves • • • • 12+ resource changes for 1 node addition • Load balancer config Nagios host ping Nagios host ssh Nagios host HTTP Nagios host app health Graphite CPU Graphite Memory Graphite Disk Graphite SNMP Memcache firewall Postgres firewall Postgres authZ config

Landscape of Chef-managed Infrastructure


knife - with the Chef Server • knife node • create / edit / delete • list • knife cookbook ... • knife role ... • knife environment ...

knife bootstrap $ knife bootstrap SERVER -r ‘role[webserver]’ • SSH to the machine given existing credentials • Install the Chef Client • Register with the Chef Server • Run the initial Run List • Now managed with Chef!

Chef & CloudStack gem install knife-cloudstack

knife-cloudstack 1/4 knife cs • ** CS COMMANDS ** • knife cs aag list (options) • knife cs account list (options) • knife cs cluster list (options) • knife cs config list (options) • knife cs disk list (options) • knife cs domain list (options) • knife cs firewallrule create hostname 8080:8090:TCP: • knife cs firewallrule list (options) • knife cs forwardrule create hostname 8080:8090:TCP • knife cs host list (options)

knife-cloudstack 2/4 knife cs • knife cs hosts • knife cs iso list (options) • knife cs keypair create KEY_NAME (options) • knife cs keypair delete KEY_NAME (options) • knife cs keypair list (options) • knife cs network list (options) • knife cs oscategory list (options) • knife cs ostype list (options) • knife cs pod list (options) • knife cs project list (options)

knife-cloudstack 3/4 knife cs • knife cs publicip list (options) • knife cs router list (options) • knife cs securitygroup list (options) • knife cs server add nic SERVERID NETWORKID (--ipaddress X.X.X.X) • knife cs server create [SERVER_NAME] (options) • knife cs server delete SERVER_NAME [SERVER_NAME ...] (options) • knife cs server list (options) • knife cs server reboot SERVER_NAME [SERVER_NAME ...] (options) • knife cs server remove nic SERVERID NICID • knife cs server start SERVER_NAME [SERVER_NAME ...] (options)

knife-cloudstack 4/4 knife cs • knife cs server stop SERVER_NAME [SERVER_NAME ...] (options) • knife cs service list (options) • knife cs stack create JSON_FILE (options) • knife cs stack delete JSON_FILE (options) • knife cs template create NAME (options) • knife cs template extract NAME (options) • knife cs template list (options) • knife cs template register NAME (options) • knife cs user list (options) • knife cs volume create NAME (options) • knife cs volume list (options) • knife cs zone list (options)

Why the Cloud? • Instant infrastructure • Unlimited capacity • Autoscaling • No commitment • Immediate replacement

Why not the Cloud? • Reliability • Performance • Security • Price

Data Gravity

Know our escape plan for every infrastructure provider

Chef for Infrastructure Portability • knife rackspace • knife openstack • knife azure • knife cloudstack • knife ec2 • knife google • knife hp • knife vsphere • ...and many others

Desktop, Virtualization & Cloud • • • • • • Vagrant VMware CloudStack Eucalyptus OpenStack bare metal • • • • • • AWS Rackspace HP Google Azure many others ™

Desktop, Virtualization & Cloud • • • • • • Vagrant VMware CloudStack Eucalyptus OpenStack bare metal • • • • • • AWS Rackspace HP Google Azure many others ™

What does this all mean? •Every infrastructure is a unique snowflake •Understand the costs associated with the features of your platform(s) of choice. •Chef enables Infrastructure Portability •Use the same infrastructure code for wherever you deploy •"Data Gravity" is a concern ™

There’s a lot more... • Attributes • Environments • Roles • Chef Handlers • LWRPs • Workflow Helpers • Testing Frameworks • and even more awesome

Thanks! Matt Ray @mattray

Add a comment

Related presentations

Presentación que realice en el Evento Nacional de Gobierno Abierto, realizado los ...

In this presentation we will describe our experience developing with a highly dyna...

Presentation to the LITA Forum 7th November 2014 Albuquerque, NM

Un recorrido por los cambios que nos generará el wearabletech en el futuro

Um paralelo entre as novidades & mercado em Wearable Computing e Tecnologias Assis...

Microsoft finally joins the smartwatch and fitness tracker game by introducing the...

Related pages

Cloud Slide decks | Lanyrd - Lanyrd - discover thousands ...

Cloud Slide decks. 95 conference ... Slides SCALE12X Build a Cloud Day: Chef: The Swiss Army Knife of Cloud ... The Swiss Army Knife of Cloud ...
Read more

Chef and Google Compute Platform: Ready to Scale | Chef

Chef and Google Compute Platform ... infrastructure. With Chef Delivery and ... I needed and at least a day to add a new developer. With Chef, ...
Read more Victorinox Swiss Classic 8" Chef's Knife ...

Victorinox Swiss Classic 8" Chef's Knife ... Comparing the 40520 and the Swiss Army Chef's side by ... This knife does the trick Published 17 days ...
Read more

Software Developer Job at Rackspace, the #1 managed cloud ...

Posted 2 days ago May 11 ... With the largest OpenStack public cloud in the business and our commitment to build the best private cloud offering in ...
Read more

Chef | IT Automation for speed and awesomeness | Chef

Cloud Management ... Build security and compliance checks into your software ... and report on any operational events within your Chef-managed infrastructure.
Read more

Next at Microsoft -

These are the things that you use every day for work and play, ... one ‘Swiss Army knife ... Next at Microsoft; Official Microsoft Blog;
Read more

Case knives from xxCutlery-Case Engraving Specialist

About, Your Case Knife Master Dealer. ... Free Shipping Special on this knife! Memorial Day ... Smooth Blue Cloud Bone Tiny Trapper with ...
Read more

Is Office 365 Overshadowing SharePoint? --

Is Office 365 Overshadowing SharePoint? ... the idea of SharePoint as a Swiss army knife, ... need platform model and into a built-for-the-cloud ...
Read more

Build 2016

Everything you need to know about Build 2016. ... Reasons day 2 rocked for developers; ... one ‘Swiss Army knife ...
Read more