Published on March 12, 2014
IEEEFutureofPaas2014 Virtualization vs Containerization to support PaaS RajdeepDua VMWare Email:email@example.com AReddyRaja IIITHyderabad,India DharmeshKakadia IIITEmail:firstname.lastname@example.org
IEEEFutureofPaas2014 About Myself • Working on Distributed Systems / Server side platforms for more than 15 years • Some companies I worked for VMware Google Microsoft Amdocs • vSphere, OpenStack, Cloud Foundry, Google App Engine… • Twitter : @rajdeepdua
IEEEFutureofPaas2014 Agenda • Introduction • Definition : VMs and Containers • Paas Requirements • Understanding Containment Principles • Existing Container Implementation and Comparison • Choosing the right container for a Paas • Conclusion
IEEEFutureofPaas2014 Introduction • Platform as a Service has brought developer productivity to the forefront • Infrastructure as a Service is masked from a developer • Opportunity for more efficient use of resources • In this paper, we explore various container technologies and how they are used by Paas Implementation
IEEEFutureofPaas2014 Comparing VMs and Containers
IEEEFutureofPaas2014 Virtual Machine and Containers • Virtual Machine • Allow Multiple Guest OS to run together on a single machine. • Each Guest OS abstracts Compute, Storage and Network Components. • Hypervisor itself could run on bare-metal (ESXi) or be part of an OS (KVM). • Guest ISA is translated to Host ISA using multiple techniques like Hardware Virtualization or Binary Translation.
IEEEFutureofPaas2014 Virtual Machine and Containers Container • light weight containment system running • Runs instructions native to the core CPU : • Container shares the Kernel and Process scheduler with the Host OS • No Binary Translation or VM exits • Containers can be run either directly on the Host OS or in a Guest VM :
IEEEFutureofPaas2014 Virtual Machine and Containers Parameter Virtual Machines Containers Guest OS Each VM runs on a hypervisor and Kernel is loaded into in its own memory region All the guests share same OS and Kernel. Kernel image is loaded into the physical memory Networking Can be linked to virtual or Physical switches. Hypervisors have buffers for IO performance improvement, NIC bonding etc Leverages standard IPC mechanisms like Signals, pipes, sockets etc. Advanced features like NIC bonding etc still not available. Security Complete Isolation Isolation using techniques like namespaces
IEEEFutureofPaas2014 Virtual Machine and Containers Parameter Virtual Machines Containers Performance Suffer from a small overhead as the Machine instructions are translated from Guest to Host OS. Provide near native performance as compared to the underlying Host OS. Isolation Higher level of Isolation –Need special techniques for file sharing Subdirectories can be transparently mounted and can be shared Startup time VMs take a few minutes to boot up Containers can be booted up in a few seconds as compared to VMs Storage VMs take much more storage as the whole OS kernel and its associated programs have to installed and run Containers take lower amount of storage as the base OS is shared
IEEEFutureofPaas2014 Paas Requirements
IEEEFutureofPaas2014 Paas Requirements • Paas focusses on developer productivity and abstracts outs the underlying infrastructure • 3 Key requirements of Paas from Underlying Infrastructure are •  Provision and manage lifecycle of Networking, compute and storage programatically •  Provide Highly reliable infrastructure which can be efficicently used •  Ability to add applications/services and bind them to external routers/ dns servers
IEEEFutureofPaas2014 Paas Requirements : VM • VMs are very suitable for requirements  and  • Applications could be hosted either directly in VMs or on Containers inside VMs. • Containers are more efficient for  as they are light weight and allow better leverage of the hardware and resources
IEEEFutureofPaas2014 Container Overview • Containers leverage the underlying features in the Linux Kernel to implement Containment • chroot : Unix command which changes the root directory for the existing process and its child • Control Groups (cgroup) : • cgroup is a method to put processes into groups. • Implemented as pseudo filesystem. • Grouping can be done by a unit of thread. • Many functions are implemented as “subsystem” • e.g CPU Sets , Fake NUMA, Freezer, device
IEEEFutureofPaas2014 Container Overview : Kernel Namespaces • The purpose of each namespace is to wrap a particular global system resource in an abstraction • Makes it appear to the processes within the namespace that they have their own isolated instance of the global resource • Namespaces and cgroups are used in conjunction Pic reference : http://www.linuxfoundation.jp/jp_uploads/seminar20081119/CgroupMemcgMaster.pdf
IEEEFutureofPaas2014 Namespaces Implemented in Linux • Mount namespace : Isolate the set of file system mount points seen by a group of processes • PID namespace : Isolate the process ID number space. In other words, processes in different PID namespaces can have the same PID • UTS namespace : Isolate two system identifiers—node- name and domain-name • Network namespace : Each network namespace has its own network devices, IP addresses, IP routing tables, /proc/net directory, port numbers,
IEEEFutureofPaas2014 Containers Current State
IEEEFutureofPaas2014 Linux Containers • Linux Containers (LXC) are light weight kernel containment implementation • Use cgroups for resource allocation and namespaces for isolation • Key Characteristics of the Linux Containers are, • Resource Allocation Uses cgroups for CPU, Memory and Device allocation • Process and Network Isolation Using namespaces : PID and net • File System Isolation Each container gets a private file system by leveraging chroot
IEEEFutureofPaas2014 Linux Containers
IEEEFutureofPaas2014 Warden Container • Warden container provides a kernel independent containment implementation on top of LXC • Used by Cloud Foundry project to host applications. • Key attributes of Warden container : • Uses namespaces for isolation of Process Ids and Network namespaces • Uses cgroups concept from Linux to allocate resources • Each container is managed by a Warden Daemon
IEEEFutureofPaas2014 Docker • Docker is a daemon which provides ability to manage Linux containers as self contained images. • Utilizes LXC (Linux Containers) for the container implementation and adds image management and Union File System capability to it. • Key attribute of Docker Containers are • Resource Isolation - Same as LXC • Network and Process Isolation - Leverages LXC functionality • File System Isolation Leverages LXC functionality • Container Lifecycle - Managed using a daemon and command line tool • Container State - Docker allows ability to store and retrieve container state
IEEEFutureofPaas2014 Docker – Improving Security • High level Goals of Docker project to improve security (which are limitations of Linux Containers) • Map root user of the container to non-root user of docker • Make docker daemon run as a non root user
IEEEFutureofPaas2014 Google lmctfy • Google lmctfy provides a resource configuration API which simplifies container management as compared to LXC • Provides intent based configuration without the need to understand cgroups and removes the difficulties of unstable LXC APIs. • Improves the resource sharing and can support performance guarantees. • Key attribute of lmctfy Containers are Uses only cgroups and customer kernel patches for resource isolation • Resource Isolation - Resources isolation using cgroups. • Network Isolation - Plan to use net namespace in next version CL2 • File System Isolation - Plan to use mnt namespace in next version CL2 • Container Lifecycle - Plan to support freeze, chck- point/restore and disk import/export. • Monitoring - To be supported through higher level c APIs
IEEEFutureofPaas2014 Open VZ • OpenVZ uses a modified Linux Kernel with a set of extensions. • very similar conceptually, its functionality is not part of the core Linux Kernel • It additionally provides templates that help in pre- created Virtual environments. • Resource Management - Manages resource sharing using Bean Counters, Fair Share CPU Scheduler .. • Network Isolation - Uses net namespace File System Isolation - Provides isolation to Application Files, System Libraries etc • Container Lifecycle – managed using vzctl tool • Container State - Provides Check pointing feature for storing and recovering the last known state.
IEEEFutureofPaas2014 Container Comparison
IEEEFutureofPaas2014 Container Comparison Parameter LXC Warden Docker OpenVZ Process Iso- lation Uses pid names- pace Uses pid names- pace Uses pid names- pace Uses pid names- pace Resource Isolation Uses cgroups Uses cgroups Uses cgroups Uses cgroups Network Isolation Uses net names- pace Uses net names- pace Uses net names- pace Uses net names- pace Filesystem Isolation Using chroot Overlay File system using overlayfs Using chroot Using chroot Container Lifecycle Tools lxc-create, lxc-stop, lxc-start to create, start, stop a container Containers are managed by running com- mands on a warden client which talks to warden server Uses Docker daemon and a client to manage the containers uses vzctl to manage container lifecycle
IEEEFutureofPaas2014 Container usage in a Paas Example
IEEEFutureofPaas2014 Container adoption in a Open Source Paas : Cloud Foundry • Cloud Foundry uses Warden Container to isolate each application. • Multiple containers are run on a single VM for efficient Resource usage • Currently tested on on Ubuntu. Supports AWS, vSphere and OpenStack Clouds
IEEEFutureofPaas2014 Choosing the Right Container for your Paas • Choosing the right container for most of the PaaS implementations will be based on the following factors: • Ecosystem for prebuilt containers • Hardened layer for isolation of Process, Network, CPU and File system • Tools to manage lifecycle of a container • Ability to migrate containers between hosts • Support for multiple OS and kernels
IEEEFutureofPaas2014 Next Generation Paas Support • Some of the features listed below would help improve adoption of containers in PaaS platforms • Standardization : • Need for a standard container file format – similar to OVF • Security : • Containers need to be more secure both from the perspective of file system, network and memory isolation. • Most popular container implementations like LXC and Docker seem to be lacking in this regard. • OS Independence • Containers should have abstraction layer so that they are not tied to a particular Kernel or its user space. • Warden Container seems to be in the right direction in this regard, but we doubt it will get wide adoption beyond Cloud Foundry.
IEEEFutureofPaas2014 Conclusion • Containers have an inherent advantage over VMs for PAAS use case • Multiple flavors of container implementations available, all of them open source. • Common Linux containment principles like chroot, cgroups or namespaces are being used by all of them. • Containers have a bright future specially in the PaaS use case provided there is more standardization and abstraction from the underlying kernel and Host OS.