Set up Xen 4.1-unstable & pvops Dom0 on top of Ubuntu 10.04 Server

Due to presence CSs 21556 (blktap2: Fix tapdisk disktype issues) , 21167 (Port latest grub zfs boot code to pygrub) Xen instance been created bellow would provide an option to test NexentaStor-Community-3.0.2-xen.tar.bz2 . In meantime new service xencommons should be configured for successful xend startup and network bridge configuration has to be performed manually vs 4.0.1. Service xencommons
actually starts xenstored and xenconsoled services before xend.

apt-get install libcurl4-openssl-dev \
xserver-xorg-dev \
mercurial gitk \
build-essential \
libncurses5-dev \
uuid-dev gawk \
gettext texinfo bcc \
libncurses5-dev dpkg-dev debhelper \
kernel-package fakeroot \
iasl texinfo bridge-utils bison flex
apt-get build-dep xen-3.3

Now git clone and checkout branch xen/stable-2.6.32.x

cd /usr/src
git clone
git:// linux-2.6-xen
cd linux-2.6-xen
git checkout -b xen/stable-2.6.32.x origin/xen/stable-2.6.32.x

Build kernel in Debian way

# export CONCURRENCY_LEVEL=(number_of_cores) + 1
# make menuconfig # Enabled the dom0 support and the Xen backend as usual
# make-kpkg clean
# fakeroot make-kpkg --initrd --append-to-version=-xen kernel-image kernel-headers
# dpkg -i ../linux-image**.deb
# update-initramfs -c -k

In case configuring kernel (1)
<M> Xen /dev/xen/evtchn device
echo “xen-evtchn” >> /etc/modules
In case configuring kernel (2)
<*> Xen /dev/xen/evtchn device
/etc/modules shouldn’t be touched.

With config bellow there is no need to touch /etc/modules at all.
Option “userspace grant access device driver is set to” <*> to make virt-install handy at Xen guests

Set up Xen 4.1-unstable Hypervisor
As of July 11 2010 exeCooler’s blkpat2_pygrub.patch
should be applied before build until corresponding CS will be committed to xen-unstable.hg

# cd /usr/src
# hg clone
# cd xen-unstable.hg
# make xen
# make tools
# make install-xen
# make install-tools PYTHON_PREFIX_ARG=

Set up xencommons,xend,xendomains,watchdog-xen to run as services

update-rc.d xencommons defaults 19 18
update-rc.d xend defaults 20 21
update-rc.d xendomains defaults 21 20
update-rc.d xen-watchdog defaults 22 23

You should also have configure bridge for xen DomUs manually via /etc/network/intefaces :-

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).
# The loopback network interface
auto lo
iface lo inet loopback
# The primary network interface
# auto eth0
# iface eth0 inet dhcp
auto br0
iface br0 inet static
bridge_ports eth0
bridge_stp on
bridge_maxwait 0

Reboot in Xen with grub entry

menuentry "Xen 4.1 / Ubuntu 10.04 kernel" {
insmod ext2
set root='(hd1,9)'
multiboot (hd1,9)/boot/xen.gz dummy=dummy
module (hd1,9)/boot/vmlinuz- dummy=dummy nopat root=/dev/sdb9 ro console=tty0
module (hd1,9)/boot/initrd.img-

Verify Xen environment :-

root@ServerLSX:~# xm info
host : ServerLSX
release :
version : #10 SMP Wed Jul 7 22:17:57 MSD 2010
machine : x86_64
nr_cpus : 4
nr_nodes : 1
cores_per_socket : 4
threads_per_core : 1
cpu_mhz : 2833
hw_caps : bfebfbff:20100800:00000000:00000940:0408e3fd:00000000:00000001:00000000
virt_caps : hvm
total_memory : 8190
free_memory : 2048
free_cpus : 0
xen_major : 4
xen_minor : 1
xen_extra : -unstable
xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64
xen_scheduler : credit
xen_pagesize : 4096
platform_params : virt_start=0xffff800000000000
xen_changeset : Mon Jul 05 08:40:26 2010 +0100 21719:9d965ac1b0db
xen_commandline :
cc_compiler : gcc version 4.4.3 (Ubuntu 4.4.3-4ubuntu5)
cc_compile_by : root
cc_compile_domain :
cc_compile_date : Wed Jul 7 21:21:02 MSD 2010
xend_config_format : 4

root@ServerLSX:~# brctl show
bridge name bridge id STP enabled interfaces
br0 8000.90e6ba2d11eb yes eth0
virbr0 8000.000000000000 yes

Test Xen installation via creation NexentaStor 3.0.2 Xen PV Guest. Download sample via
# wget
and uncompress tar.bz2 archive :-

root@ServerLSX:~/NexentaStor-Community-3.0.2# ls -l
total 2439888
-rw-r–r– 1 root root 21017088 2010-07-08 16:48 data_disk1.img
-rw-r–r– 1 root root 5120 2010-05-24 15:36 data_disk2.img
-rw-r–r– 1 root root 5120 2010-05-24 15:36 data_disk3.img
-rw-r–r– 1 root root 2573 2010-07-08 16:17 nexentastor-community-3.0.2-xen.cfg
-rw-r–r– 1 root root 1262 2010-05-24 16:15 readme.txt
-rw-r–r– 1 root root 2477388288 2010-07-08 16:48 syspool.img

# cat nexentastor-community-3.0.2-xen.cfg

work_dir = '/root/NexentaStor-Community-3.0.2'
# Names of disks
disk_1_name = 'syspool.img'
disk_2_name = 'data_disk1.img'
disk_3_name = 'data_disk2.img'
disk_4_name = 'data_disk3.img'
# Bootloader for boot DomU
bootloader = "/usr/local/bin/pygrub"
# Memory for DomU, in MB
memory = 1024
# Name of DomU
name = "NexentaStor-3.0.2"
# Network interfaces for DomU
# You can change the mac address of your own needs.
# mac=00:16:3E:xx:xx:xx
vif = [
disk = [
'tap2:tapdisk:vhd:' + work_dir + '/' + disk_1_name + ',xvda,w',
'tap2:tapdisk:vhd:' + work_dir + '/' + disk_2_name + ',xvdb,w',
'tap2:tapdisk:vhd:' + work_dir + '/' + disk_3_name + ',xvdc,w',
'tap2:tapdisk:vhd:' + work_dir + '/' + disk_4_name + ',xvde,w',
on_reboot = 'destroy'
on_crash = 'destroy'
on_poweroff = 'destroy'

Start Nexenta DomU :-

root@ServerLSX:~/NexentaStor-Community-3.0.2# xm create -c nexentastor-community-3.0.2-xen.cfg
Using config file “./nexentastor-community-3.0.2-xen.cfg”.
Started domain NexentaStor-3.0.2 (id=1)
WARNING: Cannot extract revision on this hypervisor version: v-unstable, unexpected version format
v4.1-unstable chgset ‘Mon Jul 05 08:40:26 2010 +0100 21719:9d965ac1b0db’
SunOS Release 5.11 Version NexentaOS_134d 64-bit
Loading Kernel…
Hostname: myhost
Reading ZFS config: done.
Mounting ZFS filesystems: (6/6)

ZFS Open Storage Appliance (v3.0.2)

myhost console login: root
Last login: Sat Jul 10 00:31:02 on console
nmc@myhost:/$ show
Option ? network
Option ? interface
==== Interfaces ====
lo0: flags=2001000849 mtu 8232 index 1
inet netmask ff000000
xnf0: flags=1004843 mtu 1500 index 2
inet netmask ffffff00 broadcast
ether 0:16:3e:0:0:1
xnf1: flags=1000842 mtu 1500 index 3
inet netmask 0
ether 0:16:3e:0:0:2
lo0: flags=2002000849 mtu 8252 index 1
inet6 ::1/128

Set up Libvirt 0.8.1 along with with all related packages via Daniel Baumann PPA It seems to provide virt-install utility working OK under Xen 4.1-unstable with pvops kernel, regardless “virsh” fails to work properly with xenstore and consequently virt-manager 0.8.4 is useless for 4.1

root@ServerLX:~# apt-key adv --keyserver --recv-keys D18C7A1DF6E6BED2
Executing: gpg --ignore-time-conflict --no-options --no-default-keyring --secret-keyring /etc/apt/secring.gpg --trustdb-name /etc/apt/trustdb.gpg --keyring /etc/apt/trusted.gpg --primary-keyring /etc/apt/trusted.gpg --keyserver --recv-keys D18C7A1DF6E6BED2
gpg: requesting key F6E6BED2 from hkp server
gpg: key F6E6BED2: public key "Launchpad ppa" imported
gpg: Total number processed: 1
gpg: imported: 1 (RSA: 1)
root@ServerLX:~# apt-get update
Hit lucid Release.gpg
Get:1 lucid Release.gpg [307B]
Ign lucid/main Translation-en_US
Hit lucid Release.gpg
Hit lucid-security Release.gpg
Ign lucid/main Translation-en_US
Ign lucid-security/main Translation-en_US
Ign lucid-security/restricted Translation-en_US
Ign lucid/restricted Translation-en_US
Ign lucid/main Translation-en_US
Get:2 lucid Release [57.3kB]
Ign lucid-security/universe Translation-en_US
Ign lucid-security/multiverse Translation-en_US
Hit lucid-security Release
Ign lucid/universe Translation-en_US
Hit lucid Release
Ign lucid/multiverse Translation-en_US
Hit lucid-security/main Packages
Hit lucid/main Packages
Hit lucid/main Sources
Hit lucid-updates Release.gpg
Hit lucid-security/restricted Packages
Hit lucid-security/main Sources
Hit lucid-security/restricted Sources
Hit lucid-security/universe Packages
Ign lucid-updates/main Translation-en_US
Hit lucid/main Packages
Hit lucid-security/universe Sources
Hit lucid-security/multiverse Packages
Hit lucid-security/multiverse Sources
Ign lucid-updates/restricted Translation-en_US
Ign lucid-updates/universe Translation-en_US
Ign lucid-updates/multiverse Translation-en_US
Hit lucid Release
Hit lucid-updates Release
Hit lucid/main Packages
Hit lucid/restricted Packages
Hit lucid/main Sources
Hit lucid/restricted Sources
Hit lucid/universe Packages
Hit lucid/universe Sources
Hit lucid/multiverse Packages
Hit lucid/multiverse Sources
Hit lucid-updates/main Packages
Hit lucid-updates/restricted Packages
Hit lucid-updates/main Sources
Hit lucid-updates/restricted Sources
Hit lucid-updates/universe Packages
Hit lucid-updates/universe Sources
Hit lucid-updates/multiverse Packages
Hit lucid-updates/multiverse Sources
Fetched 308B in 5s (55B/s)
Reading package lists... Done

root@ServerLX:~# apt-get install virtinst python-virtinst virt-viewer
It’s important to specify “–connect xen:///” inside virt-install command line to succeed with VNC console launching (vs 9.10) . Variable VIRSH_DEFAULT_CONNECT_URI exported via .bashrc in my case caused virt-install to crash.Comment out (xend-unix-server yes) in /etc/xen/xend-config.sxp and export variable


in root’s .bashrc. Then reboot Xen Host.

root@ServerLX:~# virsh version
Compiled against library: libvir 0.8.1
Using library: libvir 0.8.1
Using API: Xen 3.0.1
Running hypervisor: Xen 4.1.0

6 Responses to Set up Xen 4.1-unstable & pvops Dom0 on top of Ubuntu 10.04 Server

  1. harley says:

    why do you have a virbr0 and a br0 if you only created br0 with /etc/network/interfaces ? i have tried to reproduce this instead of using static and an address field i’ve used dhcp on br0 , in my win7 domU with vif = [bridge=br0] i am unable to get an ip address (even setting up interfaces with a static ip address i am unable to use networking with the win7 domu)


  2. harley says:

    oh ok someone helped me and pointed out the libvirt networking part of virbr0 aside from br0

  3. Ron Arenas says:

    Hi there! I got a new kernel on Lucid 10.04 LTS compiled with the requirement specifications from Xen relative to Paravirt-Ops kernels with Xen 4.1.1-rc1-pre freshly baked, installed and running. For now, I experience reboots or system restart when I try to shutdown the server. I also see a slew of messages from dmesg relative to the following:
    a) “XENBUS: Unable to read cpu state”
    b) radeon 0000:08:09.0: VGA-1: EDID invalid

    Hope you can shed some light on this, as I am still doing alot of other tests with this build.

    Many thanks in advance for any help.


  4. webmin handy says:

    We’re a group of volunteers and opening a new scheme in our community. Your website provided us with valuable info to work on. You’ve done a formidable job and
    our entire community will be grateful to you.

  5. Katherin says:

    I don’t know whether it’s just me or if perhaps everyone else encountering issues with your
    website. It appears as if some of the written text in your content are running off the screen. Can somebody else please comment and let
    me know if this is happening to them too? This may be a problem with my web browser because I’ve had this happen previously.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: