Creating a disposable dev environment using libvirt

When AWS Elastic Compute Cloud was first launched in 2006 [1] it was lauded as the future of scalable, on-demand web services. The first use cases explained that up to 20 instances per account could be launched, each with access to a virtualised 1.7 GHz Xeon processor, 1.75 GB of RAM, 160 GB (!) of disk space, and a sustained 250 Mb/second of bandwidth.
Fast-forward to 2019 and it’s never been easier to create your own instance on your server at home (or at work). This is thanks to libvirt and associated tools to make provisioning and partitioning the resources simple and reproducible.
In this guide I’ll be showing you how to create a base image on your machine that you can use to run a disposable dev environment for testing new packages or kernels.

We’ll be using Debian 9 “Stretch” and because libvirt is common to many distros the instructions should carry over nicely.

To get started, install the necessary tools.
$ sudo apt install libvirt-daemon virtinst libguestfs-tools virt-manager

Next, download the Debian 9 ISO [2]. I used /data/virt/img

Next we’ll use virsh, the libvirt CLI, to configure and install the VM. For this VM we’ll want the following:

256MB RAM
20GB disk
1 vCPU
network bridged using macvtap

$ virt-install --connect qemu:///system --os-variant debian9 --vcpus 1 --memory 256 --name debian-dev-env-2 --disk size=10,path=/data/virt/vhd/debian-dev-env.qcow2,format=qcow2 --cdrom /data/virt/img/debian-9.7.0-amd64-netinst.iso --network type=direct,source=enp4s0 --input tablet --graphics vnc,password=virt,port=5900 --transient

Connect via VNC port 5900 on the host if the VNC viewer didn’t already launch via virt-viewer.

After the installation is done shutdown the guest and you should now have a virtual image you can boot from. In my case, this image is stored at /data/virt/vhd/debian-dev-env.qcow2.

Now let’s copy our SSH key into the base image. This assumes the user you created is called local and the user running the virt-sysprep command is called user

$ virt-sysprep --ssh-inject local:file:/home/user/.ssh/denv-rsa.pub --add /data/virt/vhd/debian-dev-env.qcow2

Finally, we’ll enable the serial console in the event that we no longer have VNC. Notice the new “–import” option.

$ virt-install --connect qemu:///system --os-variant debian9 --import --vcpus 1 --memory 256 --name debian --disk /data/virt/vhd/debian-dev-env.qcow2,format=qcow2 --network type=direct,source=enp4s0 --input tablet --graphics vnc,password=virt,port=5900 --transient

# systemctl enable serial-getty@ttyS0.service

# poweroff

Finally, our development environment is ready. We can launch it whenever we need one by creating a snapshot of the base volume and using that to launch virt-install.

 $ chmod 400 /data/virt/vhd/debian-dev-env.qcow2
 $ qemu-img create -f qcow2 -o backing_file=/data/virt/vhd/debian-dev-env.qcow2 /data/virt/snap/debian-9.qcow2_snapshot
 $ virt-install --connect qemu:///system --os-variant debian9 --import --vcpus 1 --memory 256 --name debian --disk /data/virt/snap/debian-9.qcow2_snapshot,format=qcow2 --network type=direct,source=enp4s0 --graphics none --transient
 $ ssh user@debian

When you are done, just shutdown.

If you want a fresh installation, create a new snapshot of the base image and use that as the new base.

I haven’t figured out a good way to automate this in a fail-safe manner with one command line yet. I’ll update this post when I have.

[1]: https://aws.amazon.com/blogs/aws/amazon_ec2_beta/

[2]: https://www.debian.org/CD