Running a 2.6.6 UML Inside a 2.6.6 Host

About This Document

This is a list of steps I went through to get a Linux 2.6.6 UML kernel working in a Linux 2.6.6 host kernel. I initially tried a pre-built Fedora Core 1 host kernel, but abandoned it quickly after realizing that the separate kernel address space (skas) patch is vital for performance.

To make things clear, this is what I did and it worked for me. (Just for sanity, there has to be a better way) As we all know, even the smallest change in hardware or software may render some steps invalid for your system. I used the documentation on the UML website as the basis for all of these steps.

I would love to improve my installation and/or this documentation, so if you have any suggestions or better a way of doing things, feel free to send me an email.

Last Updated

May 28, 2004

My System

Host Kernel

If you want your UMLs to run fast, you should consider recompiling your host kernel using the SKAS patch available at the User-mode Linux site.More information about SKAS can be found here: http://user-mode-linux.sourceforge.net/skas.html The patch can be found here

Now configure the kernel (without ARCH=um, this is a host kernel) enable:

UML Utilities

Many features of the UML kernel require a user-space helper program, so a uml_utilities package is distributed separately from the kernel patch which provides these helpers. Download it Included within this is: The uml_utilities tree is compiled with:
host# make && make install

File Systems

Once running UML, you can access the host file system as well as all other file systems that are passed in as command line arguments. Each file system file is considered a unique device/partition in UML.

Swap Partition

The following commands will create a 128M swap partition that will be used by the UML instance.
$ dd if=/dev/zero of=swap_fs bs=1M count=1 seek=127
$ /sbin/mkswap swap_fs

tmpfs

UMLs will create a temporary memory file with the size of the UML's memory ( mem option) in the directory denoted by $TMPDIR . It is recommended to use a tmpfs file system for $TMPDIR as it reduces system load significantly and thus improves overall performance.

To set up a tmpfs directory, create a directory and mount it as type tmpfs . The start script shown later in this document will set the TMPDIR environment variable before booting the kernel. Note, that it is not a good idea to use /tmp with tmpfs as this might break some applications.

You must be root to mount the tmpfs directory. Adjust the size option as needed (Should be as large as the sum of memory assigned to UMLs).

# mkdir umltmpfs
# mount -t tmpfs -o mode=1777,size=512M none /usr/src/uml/temp_fs

It is also possible to add the tmpfs directory to the host's fstab to mount it during host system boot.

none  /usr/src/uml/temp_fs  tmpfs  mode=1777,size=512M     0  0

Root File System

I used a redhat 9 file system located here. I would have liked to use a Fedora Core 1 (since that is my host) but I couldn't find one. A fedora core 2 client exists here, but I was worried about being on the bleeding edge. I've heard that it isn't too hard to simply make a file system of the system you are running and import the packages/rpm's into it but I've never tried. Having the same root file system as the host is extremely useful if for no other reason to ensure the gcc compiler is the same between them. You will be compiling inside UML and referencing the Linux source tree built by the host gcc, causing module version loading problems.

UML Networking Support

There are several different approaches to allow UMLs to access the network. A description of the possibilites can be found at http://user-mode-linux.sourceforge.net/networking.html. I found the best way to get your host and UML both on the net is to use a bridge and use TUN/TAP. I'm sure the others work, but I had less success. I also found this less confusing as you do not need separate IP addresses for each side of the TUN/TAP setup. I find enabling your UML to access the outside world useful if only to be able to use apt to update the bare bones file systems that are available for download with UML.

On the host where:

The first step is configure your host with a bridge and a tap device.  
By doing this, neither your ethX or tapX device has an IP address, 
only the bridge.  The commands to execute are:
tunctl -u dhildebz
brctl addbr br0
ifconfig eth0 0.0.0.0 promisc up
ifconfig tap1 0.0.0.0 promisc up
ifconfig br0 132.175.21.58 netmask 255.255.255.0 broadcast 132.175.21.255 up
brctl stp br0 off
brctl setfd br0 1
brctl sethello br0 1
brctl addif br0 eth0
brctl addif br0 tap1
chmod 666 /dev/net/tun

Before you change your routing tables, you may need to delete some old entries.
route add -net 132.175.21.0 netmask 255.255.255.0 dev br0
route add default gw 132.175.21.1 dev br0

You will need to edit the /etc/sysconfig/network-scripts/ifcfg-eth0 
to no longer set the IP address.  Here is mine:
DEVICE=eth0
BOOTPROTO=static
ONBOOT=yes
TYPE=Ethernet

One thing I haven't done yet is get all of this stuff to automatically 
load on a reboot of the host (right now I manually run a script, I need 
to get it into /etc/rc.d or something)

If you don't want that tap device any more, you can make it non-persistent with
host#  tunctl -d tap device

Finally, tunctl has a -b (for brief mode) switch which causes it to output 
only the name of the tap device it created. This makes it suitable 
for capture by a script:
host#  TAP=`tunctl -u 1000 -b`

In UML

Once you have succesfully booted UML (as described below), eth0 
may be initially configured to some random incorrect value.

Edit /etc/sysconfig/network-scripts/ifcfg-eth0 with values for
the UML instance.  Here is mine:
DEVICE=eth0
ONBOOT=YES
IPADDR=132.175.21.63
GATEWAY=132.175.21.1
NETMASK=255.255.255.0
NETWORK=132.175.21.0
BROADCAST=132.175.21.255
TYPE=Ethernet

Edit /etc/resolv.conf so that it looks like the version on the host.

Restart the network (/etc/init.d/network restart)

Execute the following to get the routing tables correct.
(rebooting the UML kernel will probably also correct the routing 
table automatically)  Essentially it should be the same as the host.  
route add -net 132.175.21.0 netmask 255.255.255.0 dev eth0
route add default gw 132.175.21.1 dev eth0

User-Mode Linux Kernel Compile

First, unpack the kernel and apply both UML-patch's listed here. Then configure the kernel and compile the linux binary and the modules. Do not forget the ARCH=um! I heard rumours of having to perform a make distclean if you forget it.

$ make menuconfig ARCH=um
Now you must fix the config as the default doesn't even include networking. Here are some tips:
$ make linux modules ARCH=um

You now want to mount the root file system and install the modules into it.

mount root_fs mnt -o loop
make modules_install INSTALL_MOD_PATH=/path/to/uml/mnt ARCH=um 
umount mnt

Final Pre-Boot Steps

Bugs with 2.6.6

Booting

The root file system is now prepared for the first boot. If you intend to create multiple UMLs, I would first get 1 fully working and then copy the root file system for the others. You can use the same temp_fs for all UML instances, but its size must equal the sum of all 'mem=' params set on the command line. (I'm currently only running 1 instance)

Below is an example script to start the UML. The script requires the root_fs and swap_fs files, the configured TUN device and the linux kernel binary.

#!/bin/sh
 
mem=128M
fs=/usr/src/uml/root_fs.rh-9-full.pristine.20030724
swapfs=/usr/src/uml/swap_fs
 
# change this path to point to correct tmpfs dir
export TMPDIR=/usr/src/uml/temp_fs
 
echo "Checking root file system..."
/sbin/e2fsck -C -p $fs
if [ $? -gt 1 ]; then
    echo "*** An error occured during the root filesystem check. ***"
    exit 1
fi
 
echo "Starting kernel..."
./linux-2.6.6-uml-nfsv4-build/linux \
    ubd0=$fs ubd1=$swapfs \
    con=pty con0=fd:0,fd:1  \
    mem=$mem \
    eth0=tuntap,tap1

# If you want an xterm to appear instead of the console, chang the console line to 
# con0=xterm con1=xterm con=/dev/null \

Post Boot Steps

  1. Complete UML network steps as described above.
  2. Most likely your swap partition failed to mount. This is due to incorrect minor numbers on the /dev/ubd/X nodes. UML handles ubd-drives in /dev with major number 98 and minor number 0, 16, 32, etc for the harddisks. I created a new node
    mknod 8 b 98 16
    with the correct minor number. I guess ubd 2-7 can be deleted...
  3. Replace the swap config in /etc/fstab with
    /dev/ubd/8      swap            swap    sw
  4. Make the directory /mnt/host and add the following to /etc/fstab to mount the host file system
    none       /mnt/host  hostfs  defaults  0  0
  5. I copied all of my bash shell config files to the UML root fs.

References

  1. The User-mode Linux Kernel Home Page: http://user-mode-linux.sourceforge.net/
  2. Linux ethernet bridging: http://bridge.sourceforge.net/
  3. User-mode Linux: skas mode http://user-mode-linux.sourceforge.net/skas.html
  4. User-mode Linux: Setting up the network http://user-mode-linux.sourceforge.net/networking.html
  5. 2.6 Host and UML patches
  6. http://www.user-mode-linux.org/~blaisorblade/
  7. Some setup info http://www.muhlesteins.com/jsp/Wiki?UserModeLinux
  8. The most vital resource, the UML email archives http://marc.theaimsgroup.com/?l=user-mode-linux-user&r=1&w=2
  9. Even more info http://www.theshore.net/~caker/uml/tips_and_notes.html
  10. Yikes, even more. http://uml.openconsultancy.com/
  11. More and more http://www.tkn.tu-berlin.de/~posch/resoa-uml.html
  12. A useful but slightly outdated redhat UML setup guide http://www.nrh-up2date.org/howto/uml/
  13. A great UML Bridging tutorial http://edeca.net/articles/bridging/index.html
dhildebz at eecs dot umich dot edu