MINCS - containers in the shell script (Eng. ver.)

67 %
33 %
Information about MINCS - containers in the shell script (Eng. ver.)

Published on October 27, 2015

Author: mhiramat

Source: slideshare.net

1. 1 MINCS – containers in the shell script @mhiramat Github.com/mhiramat/

2. 2 Who @mhiramat A linux kernel hacker but less chance to coding (><)o Maintain perf-probe and kprobes

3. 3 At First This presentation is almost 100% about shell script. Not about kernel 'C' source code.

4. 4 What is the container? Container == Docker? There are other OSS implementations! LXC Runc OpenVZ etc… So what the container is ...?

5. 5 What is the Docker ? Docker provides many container related features. Containerize Packaging software Managing Layers and its catalog REST API etc… How does it work??

6. 6 Docker is Great, but... It seems a bit .. too BIG All the features are hidden in one binary It is hard to know how it works Remember the Unix philosophy Keep It Simple, Stupid We can do it with existing tools

7. 7 Let's mimic it! Let's try to make a minimal container How to use the namespaces How to bind the devices How to change the rootfs with chroot/pivot_root How to use Capabilities and CPUSET etc. Let's try to overlay the layers Now we have the overlayfs! How to manage layers

8. 8   MINCS Minimum Container Shell-scripts https://github.com/mhiramat/mincs Basic functions Use PID/Net/UTS/Mount namespaces Layering with overlayfs Capabilities, CPUSET and more POSIX shell script (not bash script) This can work with busybox shell/dash

9. 9 The MINCS Frontend minc marten polecat Backend minc-exec minc-coat minc-leash minc-farm minc-trapper

10. 10 Frontend Scripts Frontends of MINCS Minc : run a command in a container Marten : manage layered container images Polecat : make a self executable containerized command Frontend == parsing options Set options to environment vars and call backend scripts The pair of marten/minc-farm is exception

11. 11 minc The main tool of MINCS Run a command in a container Works as chroot (Or Docker run? :) Setup namespaces and workspaces by overlayfs Do not need any container images like Docker No need rootfs dir as chroot (we can reuse current rootfs) Netns is not enabled by default [mhiramat@localhost mincs]$ sudo ./minc ps -ef UID PID PPID C STIME TTY TIME CMD root 1 0 0 10:58 ? 00:00:00 ps -ef

12. 12 minc: Usage Usage: minc [options] [command] Options: -r/--root ROOTDIR Specify a directory as a rootfs. If omitted, use “/”. -t/--temp TEMPDIR Specify a working directory. If omitted, use a tmpdir by mkdir. -k Do not remove working directory --name UTSNAME Specify the host name in the container --debug Show the debug log If the command is omitted, run $SHELL.

13. 13 Dive into the shell script Let's look into the minc command Phase 1: Parse the command line and setup env-vars. Phase 2: Invoke minc-exec Setup netns and cpumask (if needed) Move to the new namespaces Get correct PID and setup UTSNAME Setup rootfs for container Bind device files Unmount original mounts Chroot to new rootfs and setup capabilities

14. 14 Minc: command line parsing Case and while loop Getopts is not used (not so flexible) While { case & shift } loop Mainly setup the environment value After loop Call minc-farm to get image based on UUID Post-scripting by trap command Call minc-exec

15. 15 Minc-exec(1) : Overview Self execution shellscript Unshare requires some other command to execute, so call the script itself This is a historical reason – previously minc-exec was chns – 1 script The first execution is outside a container Setup netns and cpuset Call unshare to make a container (namespace) The second is inside of the container Switch the script by checking PID == 1 Hide something from the program running in the container Device files / unused mount points

16. 16 Minc-exec(2) : netns/cpuset netns Use “ip netns” to create new network namespace if needed Use trap command to remove when the shell exits Just create an eth pair on the namespaces Do not assign IP address We can use “pipework” for more networking options CPUSET Just setup a CPUSET bitmask by using taskset. Still not using cgroups

17. 17 Intermission: Trap command Trap is great :) We can handle signal interrupts and exit Able to call shell script functions Minc usually use trap for... Remove temporary files/PID file Show the information messages when exits Suppress ^C

18. 18 Minc-exec(3): Change namespace Use unshare to change namespaces Run unshare by passing $0 Pid, mount, ipc, uts namespaces are unshared unshare -iumpf $0 “$@” For the netns, we use ip netns exec ip netns exec $MINC_NETNS unshare -iumpf $0 “$@”

19. 19 Minc-exec(4): Setup PID and utsname Get the original PID (PID in parent namespace) The PID outside container is good to send signal Since unshare command forks, we can know the PID inside the container. Even if we separate mount namespace, /proc is still same until remount it. This means we can see /proc/self. Set up utsname Use hostname command to setup utsname

20. 20 Minc-exec(5):Mount namespace Setup mount namespace In some environment (with systemd?), mount information propagates to other namespaces Mount --make-rprivate / Do not propagate all the mount operations Overlaying workspace via minc-coat Minc-coat backend does overlay on rootfs image. Do not change rootfs afterwords. If the rootfs can be changed, use --direct option

21. 21 Minc-coat: Implement overlays Make root/, storage/, work/ under tempdir Root/: The mountpoint for overlayfs → $RD Storage/: Overlayfs top directory →$UD work: a workdir for overlayfs → $WD Build a new rootfs via Overlayfs Not only using mount namespaces, but also layering for storage isolation Some differences are there depends on the version Overlayfs for upstream kernel mount -t overlay -o upperdir=$UD,lowerdir=$BASEDIR,workdir=$WD overlayfs $RD Overlayfs for Ubuntu14.10 (out-of-tree) mount -t overlayfs -o upperdir=$UD,lowerdir=$BASEDIR overlayfs $RD

22. 22 Minc-exec(6): Special Files Special files and directories Make /etc, /dev, /sys and /proc on new rootfs Bind mounts under /dev Touch dummy files and bind it (like symlink) /dev/console, /dev/null, /dev/zero, /dev/random, /dev/urandom, /dev/mqueue (and others, if you need) /dev/pts are mounted with newinstance Mount /proc for new PID namespace Old /proc should be ro remount. Some files to be readonly (/proc/sys etc.), should be bind-mounted the ro /proc. Bind mounts /sys This could be skipped or be read only

23. 23 Intermission: Debug How to debug it? Just for checking the commands, run it with --debug This option enables “set -x” If you want to break into it, write “bash”(or other shell you like) You can do anything :) Or write a command what you run MINCS is just a set of shell scripts You can change it as you want.

24. 24 Minc-exec(7): Post-process Mountpoint Remove old mountpoints If we keep it, it can still be visible after chroot Use pivot_root to unmount somethings Let's monitor it with “df -h” At last, call minc-leash to chroot.

25. 25 Minc-exec(7): Post-process Mountpoint Remove old mountpoints If we keep it, it can still be visible after chroot Use pivot_root to unmount somethings Let's monitor it with “df -h” Filesystem Size Used Avail Use% Mounted on devtmpfs 740M 0 740M 0% /dev tmpfs 748M 0 748M 0% /dev/shm tmpfs 748M 8.5M 740M 2% /run tmpfs 748M 0 748M 0% /sys/fs/cgroup /dev/sda2 15G 8.6G 6.5G 58% / Before minc

26. 26 Minc-exec(7): Post-process Mountpoint Remove old mountpoints If we keep it, it can still be visible after chroot Use pivot_root to unmount somethings Let's monitor it with “df -h”Filesystem Size Used Avail Use% Mounted on /dev/sda2 15G 8.6G 6.5G 58% / devtmpfs 740M 0 740M 0% /dev tmpfs 748M 0 748M 0% /dev/shm tmpfs 748M 0 748M 0% /sys/fs/cgroup tmpfs 748M 8.5M 740M 2% /run overlayfs 15G 8.6G 6.5G 58% /tmp/minc1012-NpuyIA/root tmpfs 748M 0 748M 0% /tmp/minc1012-NpuyIA/root/dev devtmpfs 740M 0 740M 0% /tmp/minc1012-NpuyIA/root/dev/console devtmpfs 740M 0 740M 0% /tmp/minc1012-NpuyIA/root/dev/null devtmpfs 740M 0 740M 0% /tmp/minc1012-NpuyIA/root/dev/zero devtmpfs 740M 0 740M 0% /tmp/minc1012-NpuyIA/root/dev/random devtmpfs 740M 0 740M 0% /tmp/minc1012-NpuyIA/root/dev/urandom Special files

27. 27 Minc-exec(7): Post-process Mountpoint Remove old mountpoints If we keep it, it can still be visible after chroot Use pivot_root to unmount somethings Let's monitor it with “df -h”Filesystem Size Used Avail Use% Mounted on /dev/sda2 15G 8.6G 6.5G 58% /.orig devtmpfs 740M 0 740M 0% /.orig/dev tmpfs 748M 0 748M 0% /.orig/dev/shm tmpfs 748M 0 748M 0% /.orig/sys/fs/cgroup tmpfs 748M 8.5M 740M 2% /.orig/run overlayfs 15G 8.6G 6.5G 58% / tmpfs 748M 0 748M 0% /dev devtmpfs 740M 0 740M 0% /dev/console devtmpfs 740M 0 740M 0% /dev/null devtmpfs 740M 0 740M 0% /dev/zero devtmpfs 740M 0 740M 0% /dev/random devtmpfs 740M 0 740M 0% /dev/urandom After the first pivot_root

28. 28 Minc-exec(7): Post-process Mountpoint Remove old mountpoints If we keep it, it can still be visible after chroot Use pivot_root to unmount somethings Let's monitor it with “df -h” Filesystem Size Used Avail Use% Mounted on /dev/sda2 15G 8.6G 6.5G 58% /.orig overlayfs 15G 8.6G 6.5G 58% / tmpfs 748M 0 748M 0% /dev devtmpfs 740M 0 740M 0% /dev/console devtmpfs 740M 0 740M 0% /dev/null devtmpfs 740M 0 740M 0% /dev/zero devtmpfs 740M 0 740M 0% /dev/random devtmpfs 740M 0 740M 0% /dev/urandom Remove old Procfs, etc.

29. 29 Minc-exec(7): Post-process Mountpoint Remove old mountpoints If we keep it, it can still be visible after chroot Use pivot_root to unmount somethings Let's monitor it with “df -h” Filesystem Size Used Avail Use% Mounted on overlayfs 15G 8.6G 6.5G 58% / tmpfs 748M 0 748M 0% /dev devtmpfs 740M 0 740M 0% /dev/console devtmpfs 740M 0 740M 0% /dev/null devtmpfs 740M 0 740M 0% /dev/zero devtmpfs 740M 0 740M 0% /dev/random devtmpfs 740M 0 740M 0% /dev/urandom 2nd pivot_root and Chroot to new rootfs

30. 30 Minc-leash: capabilities and chroot Leash() = “Least capabilities shell” Limits capabilities and chroot by using capsh(libcap) Change UID/GID too If we skip capabilities setting, just do chroot Wash() = “Wash out the environment variables” MINCS use environment variables internally, clean it up Unset all the vars start with MINC_*

31. 31 Use cases of MINCS Good learning material for containers If you hits some limitations on docker, you can try it, and understand. Prototyping new features Containers for embedded devices Is it wrong to desire running applications in containers on embedded device? :) Docker(>14MB, docker only) vs MINCS+Busybox(<4MB, +shell and tools) → Boot2MINC

32. 32 Boot2minc Minimal ISO image + MINCS https://github.com/mhiramat/boot2minc Forked from minimal Linux Live (https://github.com/ivandavidov/minimal  ) Including Linux kernel Busybox(+unshare patch) MINCS 8MB image including kernel (can run on Qemu-kvm) Able to reduce the size if we optimize the configuration

33. 33 Marten: Manage container images Minc provides only container feature Should we prepare rootfs via debootstrap? How to get the rootfs of Fedora/CentOS etc.? Want to reuse the result of previous container easily Overlayfs-based container image manager Identify container images by Docker-like UUID Track the dependency between images Import Docker export/saved images

34. 34 Demonstration Minc Marten Boot2minc

35. 35 TODO minc Work with pipework Correct TTY support via tmux/screen Use cgroups to limit cpu/memory/io usage (minc-cage?) Plugin support of btrfs and dm-thin Marten Container execution command (like docker run) Support OCI compatible container export/import and signing

36. 36 Known Issues Testcases Well, we can make it by shell script too :) Capsh Capsh only accepts “sh -c” type command It doesn't accept escape characters…

37. 37 Conclusion What I'd like to say is “We can run a container by combining commands” Docker etc. is not a special, we've already have fundamental tools. And “Shell script is great!”

38. 38 END Thank you very much! :) https://github.com/mhiramat/mincs

39. 39 Example: Import image from Docker # docker save centos | gzip - > centos.tar.gz # marten import centos.tar.gz Importing image: centos 511136ea3c5a64f264b78b5433614aec563103b4d4702f3ba7d4d2698e22c158 5b12ef8fd57065237a6833039acc0e7f68e363c15d8abb5cacce7143a1f7de8a 8efe422e6104930bd0975c199faa15da985b6694513d2e873aa2da9ee402174c # marten images ID SIZE NAME 511136ea3c5a 4.0K (noname) 5b12ef8fd570 4.0K (noname) 8efe422e6104 224M centos # minc -r centos /bin/bash

Add a comment

Related pages

Shell Script | LinkedIn

Shell Script. Articles, experts, jobs, and more: get all the professional insights you need on LinkedIn. Sign up Get more personalized results when you ...
Read more

Full text of "Pacific service magazine" - Internet Archive

Full text of "Pacific service magazine" See other formats ...
Read more