Running VM in container, and then container in VM ...

Everyone of us.. (okay fine, most of us) have seen inception movie, you know dreams in dreams in dream. How how about container in vm in container in vm .. and so on? Would this even work? How much feasible this thought can be?

Since, we can run virtualization in ubuntu, which is essentially linux. So i guess this is possible. Now the question is how to run vm in container.. no no no i can't think about virtual box or vmware kind of thing.. i want something quite native to linux, something which utilizes existing kernel implementations like scheduler or memory management. Yes, after a couple of google hits you will find "kvm"

How kvm works?

In short, KVM (Kernel based Virtual Machine) is a virtualization solution for linux as part of linux kernel itself (since 2006).

It uses various kernel modules like (kvm.ko and kvm-intel.ko or kvm-amd.ko etc etc, which helps to modify kernel functionality just by loading and unloading). Read more about kvm here https://www.linux-kvm.org/page/Main_Page

(image taken from https://upload.wikimedia.org/wikipedia/commons/4/40/Kernel-based_Virtual_Machine.svg)

VM in container

So if we need to run kvm in container what we need? If you are familier with what cgroups and namespaces are, then you know that you need to put limits on what devices the container can use.

That points me that i would need "devices" cgroup of kernel. You may read more about that here https://www.kernel.org/doc/Documentation/cgroup-v1/devices.txt

Requirements

The very first requirement is to have hardware of x86 hardware with intel or amd processor. Well thats what kvm is built for.

And if you have that, you need to run a container that has kvm loaded. A good shortcut to go for is

docker run --privileged --net=host -it ulexus/qemu bash

Go into running container and run below and if you see something around kvm, you are good to go

$ lsmod | grep kvm

Then you might want to install a few tools that gives you handy commands to start or stop vm

$ apt-get -y install wget qemu-kvm libvirt-bin virtinst bridge-utils cpu-checker

And verify kvm installation

$ kvm-ok
INFO: /dev/kvm exists
KVM acceleration can be used

You would need an iso to start vm. Let's download that first.

$ wget https://dl-cdn.alpinelinux.org/alpine/v3.12/releases/x86_64/alpine-virt-3.12.0-x86_64.iso

Behind the scene

what is happening behind the scene

Start a vm

Let's use virt-installl to start a vm with the iso just downloaded

$ virt-install --memory=512 --vcpus=2 --cpu=host --name=alpinevm \
 --cdrom=alpine-virt-3.12.0-x86\_64.iso \
 --os-variant=alpinelinux3.12 \
 --disk size=10

Exec into vm

Let's use virsh to connect inside the vm.

$ virsh connect alpinevm

Recursive work

Now as you have vm running, you may install docker again and then start the container and repeat the work untill you feel okay you can come back to original host.

Cleaning up.

Most of the cleanup you can do, but for removing vms started via kvm, there m

$ virsh shutdown alpinevm
$ virsh destroy alpinevm
$ # other docker stop/rm

References

• https://www.cyberciti.biz/faq/installing-kvm-on-ubuntu-16-04-lts-server
• https://www.polarsparc.com/xhtml/Linux-KVM.html