Annotated HOWTO for creating an SELinux enabled UML system

---

Look here (2.6.8.1) and here (2.6.9) for older version of this document, which refer to older kernel versions.

Compiling the host system

There is little to be done here, except to apply the SKAS patches to the host kernel. These patches allow the UML kernel to run in an entirely different host address space from its processes. This solves the security and honey pot fingerprinting problems by making the UML kernel totally inaccessible to UML processes. Their address spaces are identical to what they would be on the host. A given version of UML guest will look for a specific SKAS patch in the host and fall back to thread tracing mode if it's not there. The boot-up message will tell you which version it's looking for in case you're not sure. The steps are as follows:

1. Download the original kernel sources from kernel.org - selecting the latest version that seems to work well with UML, which is, at the time of writing, 2.6.10:
   % cd /usr/local/src/kernel/
   % wget ftp://ftp.us.kernel.org/pub/linux/kernel/v2.6/linux-2.6.10.tar.bz2
   % wget ftp://ftp.us.kernel.org/pub/linux/kernel/v2.6/linux-2.6.10.tar.bz2.sign
   % gpg --verify linux-2.6.10.tar.bz2.sign linux-2.6.10.tar.bz2
2. Untar the sources somewhere (/usr/local/src/kernel/ is what I use). It is really immaterial where the kernel is unpacked, but I tend to avoid /usr/src since I do not want to be root when compiling the kernels, and so the working directory I use has write permissions for an unprivileged user.
   % tar jvvfx linux-2.6.10.tar.bz2
3. Create a dir for compiling the host kernel (2.6.10, for example). It is nice to compile the kernel in a separate directory tree from the place it has been unpacked, using a symbolic link farm, since it allows one to apply patches to the build tree while retaining the pristine source tree. Once can then use incremental patches when the next upstream release of the kernel comes out. Additionally, this allows us to build a host and a guest kernel (which needs different patch sets) without having to duplicate all the kernel source tree .
   % mkdir 2.6.10
4. Populate the build dir with symbolic links.
   % cd 2.6.10
   % lndir ../linux-2.6.10
5. Get the SKAS host patch. You can find it on the download page for user mode linux. It is also a good idea to check out the download page of the author , which has updates. At the time of writing, I got the host-skas3-2.6.9-v7.patch file (which replaces the host-skas3-2.6.9-rc4-v6.patch). Also, you need skas-update-2.6.10.patch which applies a simple correction for the 2.6.10 kernel.
   % cd ..
   % wget http://www.user-mode-linux.org/~blaisorblade/patches/skas3-2.6/host-skas3-2.6.9-v7.patch
   % wget http://www.user-mode-linux.org/~blaisorblade/patches/skas3-2.6/skas-update-2.6.10.patch
   
6. Apply the SKAS patch. Unfortunately, the former patch does not apply cleanly to the 2.6.10 sources, two hunks to the arch/i386/kernel/ptrace.c need to be applied manually (The missing hunks just need to moved down lower in the file).
   % cd 2.6.10
   % patch -p1 --dry-run < ../host-skas3-2.6.9-v7.patch
   % patch -p1 < ../host-skas3-2.6.9-v7.patch
   % patch -p1 --dry-run < ../skas-update-2.6.10.patch
   % patch -p1 < ../skas-update-2.6.10.patch
   
7. Configure the host kernel (without ARCH=um, this is a host kernel), enable /proc/mm under "Processor type and features" menu if needed, save the new configuration and build it (the config file I use can be seen as an example.)
   % make xconfig
   % make-kpkg --rootcmd fakeroot kernel-image
8. Install the resulting package, tweak your boot loader as needed, and you are good to go.
   # dpkg -i ../kernel-image-2.6.10-skas3-v7_2.6.10_i386.deb

Compiling the UML

The good news is that both SELinux and uml patches are mostly incorporated into the mainline kernel. However, there is a little glitch: the UML folks were working off Andrew Mortons tree, and the four-level page table patch is slightly different in the final 2.6.10 tree. So, we need to download a patch to the vanilla kernel to make the tree into an rc-mm tree.

1. Download the kernel sources from kernel.org:
   % cd /usr/local/src/kernel/
   % wget ftp://ftp.us.kernel.org/pub/linux/kernel/v2.6/testing/linux-2.6.10-rc3.tar.bz2
   % wget ftp://ftp.us.kernel.org/pub/linux/kernel/v2.6/testing/linux-2.6.10-rc3.tar.bz2.sign
   % gpg --verify linux-2.6.10-rc3.tar.bz2.sign linux-2.6.10-rc3.tar.bz2
2. Untar the sources.
   % tar jvvfx linux-2.6.10-rc3.tar.bz2
3. Create a dir for compiling the UML kernel (uml-2.6.9, for example), and populate it with symbolic links
   % cp -lr linux-2.6.10-rc3 uml-2.6.10-rc3
4. Download Andrew Morton's patch from ftp.kernel.org.
   % wget ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.10-rc3/2.6.10-rc3-mm1/2.6.10-rc3-mm1.bz2
   % wget ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.10-rc3/2.6.10-rc3-mm1/2.6.10-rc3-mm1.bz2.sign
   % gpg --verify 2.6.10-rc3-mm1.bz2.sign 2.6.10-rc3-mm1.bz2
5. Apply the patch:
   % cd uml-2.6.10-rc3
   % bzcat ../2.6.10-rc3-mm1.bz2 | patch -p1 --dry-run
   % bzcat ../2.6.10-rc3-mm1.bz2 | patch -p1

6. Please note that if you configure something as a module, an extra step would be required to install the modules into the UML

   Configure the kernel (don't forget ARCH=um). Since we have patched the host kernel with SKAS, we may turn of the thread tracing mode. Also, hostfs is a nice option to have, unless you are very concerned about security and leaking information from the host system into the UML. Configure the character, block, and network devices to include all the features you shall need in the UML. I strongly suggest using the honey-pot proc pseudo file-system, as well as logging. DO NOT turn on SMP and highmem support; that is known to be broken for these versions. (Oh, don't forget to turn on SELinux -- which also needs AUDIT to be turned on, amongst other things). Save the new configuration and build it (the config file I use can be seen as an example.)
   % make ARCH=um xconfig
7. Starting with recent versions of kernel-package have the functionality to build kernel-uml packages natively, so, instead of doing make ARCH=um linux, we can build a kernel-uml debian package instead.
   % make-kpkg --arch=um --rootcmd=fakeroot kernel_image
8. This results in a ../kernel-uml-2.6.10-rc3-mm1_10Custom_i386.deb, for example, which can then be installed anywhere using dpkg:
   % dpkg -i ../kernel-uml-2.6.10-rc3-mm1_10Custom_i386.deb

Preparing to network the UML's

The networking page at the UML home defines a number of ways to network the resulting UML's. Since the kernels used are way beyond 2.2.X, ethertap was out. Multicast, slip, slirp, and pcap were also out -- one should be able to use the UML to provide real world services. That leaves TUN/TAP and the Daemon protocols. The root_fs created below primarily supports the Daemon (since that makes the root_fs more portable, however, tuntap can also be used by just editing /etc/network/interfaces on the root_fs and uncommenting the tuntap stanza.

Using TUN/TAP

If you go the tun/tap route, you may either setup a fixed tap device as detailed below, or you may use the uml_net command. To do that, make sure that the user that runs the UML is in the group uml-net.

Next, make sure /usr/lib/uml is in the path for the user who runs the UML

After this, you just need to specify that the eth0 interface in the UML needs to use the tun/tap transport, set up /etc/network/interfaces inside the UML, and then sit back and let uml_net do the rest (including proxy arp and all)

Using the Daemon transport

If you just want a bunch of UML's to talk to each other, then you are all set -- uml_switch comes set up out of the box for you. However, if you choose to have the UML communicate to the external world, you need to set up a tap device for the uml_switch network to talk to. All you have to do is add something like this to /etc/network/interfaces (I chose to use tap2 since I sometimes use a vpnc client that likes to take up tap0; and 192.168.3.X is close enough to my internal network that I would have to make minimal changes to firewall rules).

Next, make sure that the tap2 interface comes up, tell uml_switch to listen to tap2, and restart uml_switch.

Again, you may set up a static network interface inside the UML, or you can set up a DHCP server on the host, and setup your UML to be a DHCP client. The advantage of the latter is that I can then share root file systems across machines, since there is nothing that is site-specific inside the root_fs.

To set up DHCP, first one needs to setup DHCP; here is an example you may use to set up dhcp on the host. You may need to edit /etc/default/dhcp in order to tell dhcp to listen on tap2, by adding the following line (if you already have an interfaces line, add tap2 to it).:

Then just restart dhcpd to have it start listening on the tap2 line:

Then mount the UML root_fs, and change the /etc/network/interfaces to the following, and you should be more or less good to go.(In my case, I also had to add tap2 to /etc/shorewall/interfaces, and also add tap2 to /etc/shorewall/masq, as well as allowing the IP address 192.168.3.X to access my local bind daemon).

Setting up the root_fs

The first thing we need to be running a UML on a machine is to install the uml-utilities package.

Though there are already mechanisms in Debian for creating a user mode linux root_fs system (notably, rootstrap), they did not work for me. For example, rootstrap fails currently on a 2.6.x host, since rootstrap tries to build the root_fs inside a UML that uses hostfs to mount the current / as the initial file system - and proceeds to fall flat on its face, since libc assumes that it can use the native posix thread library (since the UML kernel version is 2.6.8.1), and /lib/tls exists -- but UML has not yet ported NPTL over, so things fail.

I decided to write a simple shell script based around debootstrap, which also happens to be a simple shell script with very few dependencies, and hence fewer things that can go wrong. This shell script sets up a root_fs, based on a few variables at the top (you may also drop these variables in ~/.creatfsrc), and sets it up with a simple uml_net based networking.

First, select a dir on a file system that has at least a GB of space available, and run the the creatfs.sh script (after suitable modification).

At this point, you should have a root file system that can bootup, and while it is not yet running SELinux, it is ready to be taken there. There is a script written into /root/post-install.sh that needs to be run inside the UML to complete the process.

If you had configured anything in the UML as a module, this is the time to install them. If not, you may skip this step.

Russel Coker has created a very useful site that has tweaks required to make a system work properly with SELinux; creatfs.sh tries very hard to incorporate all the relevant tweaks in the root_fs created. You should be able to fire up your UML like so if you are using the TUN/TAp interface and not the persistent tap2 device (just tweak the IP address of the host):

If, on the other hand, you are using the daemon transport, use:

Working on SELinux

As mentioned before, please look at the tweaks page and make any changes that creatfs.sh may have missed. The next step would be to fire up the UML, and install the SELinux packages. To finish the process, do the following:

1. Fire up the UML. This shall be running in permissive mode, and as yet, there is no policy installed. Expect to see a lot of warnings in this run; but after we reboot the UML, it should be running with no warnings. So, as before, if using TUN/TAP transports, use:
   % /usr/bin/linux mem=256M \
   > con=xterm con0=fd:0,fd:1 con1=xterm devfs=nomount \
   > tty_log_fd=3 3>tty_log_file \
   > eth0=tuntap,,,192.168.1.10
   Else, use:
   % /usr/bin/linux mem=256M \
   > con=xterm con0=fd:0,fd:1 con1=xterm devfs=nomount \
   > tty_log_fd=3 3>tty_log_file \
   > eth0=daemon,,unix,/var/run/uml-utilities/uml_switch.ctl
2. Next, login as root, and run the final step inside the UML. Please note that installing selinux-policy-default fails initially, and generates error messages, but the script should clean all that up at the end.
   % /bin/bash /root/post-install.sh
3. Now, halt the UML
   % shutdown -h now
4. Fire up the UML a last time. This time around, there should be no warnings as you boot into an SELinux enabled UML. Enjoy.
   % /usr/bin/linux mem=256M \
   > con=xterm con0=fd:0,fd:1 con1=xterm devfs=nomount \
   > tty_log_fd=3 3>tty_log_file \
   > eth0=tuntap,,,192.168.1.10
   Or
   % /usr/bin/linux mem=256M \
   > con=xterm con0=fd:0,fd:1 con1=xterm devfs=nomount \
   > tty_log_fd=3 3>tty_log_file \
   > eth0=daemon,,unix,/var/run/uml-utilities/uml_switch.ctl

---

Manoj Srivastava <srivasta@debian.org>