Set up Xen 4.1-unstable & 2.6.32.16 pvops Dom0 on top of Ubuntu 10.04 Server

Due to presence CSs 21556 (blktap2: Fix tapdisk disktype issues) , 21167 (Port latest grub zfs boot code to pygrub) Xen instance been created bellow would provide an option to test NexentaStor-Community-3.0.2-xen.tar.bz2 . In meantime new service xencommons should be configured for successful xend startup and network bridge configuration has to be performed manually vs 4.0.1. Service xencommons
actually starts xenstored and xenconsoled services before xend.

apt-get install libcurl4-openssl-dev \
xserver-xorg-dev \
mercurial gitk \
build-essential \
libncurses5-dev \
uuid-dev gawk \
gettext texinfo bcc \
libncurses5-dev dpkg-dev debhelper \
kernel-package fakeroot \
iasl texinfo bridge-utils bison flex
apt-get build-dep xen-3.3

Now git clone and checkout branch xen/stable-2.6.32.x

cd /usr/src
git clone
git://git.kernel.org/pub/scm/linux/kernel/git/jeremy/xen.git linux-2.6-xen
cd linux-2.6-xen
git checkout -b xen/stable-2.6.32.x origin/xen/stable-2.6.32.x

Build kernel in Debian way

# export CONCURRENCY_LEVEL=(number_of_cores) + 1
# make menuconfig # Enabled the dom0 support and the Xen backend as usual
# make-kpkg clean
# fakeroot make-kpkg --initrd --append-to-version=-xen kernel-image kernel-headers
# dpkg -i ../linux-image*2.6.32.16-xen*.deb
# update-initramfs -c -k 2.6.32.16-xen

In case configuring kernel (1)
<M> Xen /dev/xen/evtchn device
echo “xen-evtchn” >> /etc/modules
In case configuring kernel (2)
<*> Xen /dev/xen/evtchn device
/etc/modules shouldn’t be touched.

With config bellow there is no need to touch /etc/modules at all.
Option “userspace grant access device driver is set to” <*> to make virt-install handy at Xen guests
installs



Set up Xen 4.1-unstable Hypervisor
************************************************************************************************************
As of July 11 2010 exeCooler’s blkpat2_pygrub.patch
should be applied before build until corresponding CS will be committed to xen-unstable.hg
*************************************************************************************************************

# cd /usr/src
# hg clone http://xenbits.xensource.com/xen-unstable.hg
# cd xen-unstable.hg
# make xen
# make tools
# make install-xen
# make install-tools PYTHON_PREFIX_ARG=

Set up xencommons,xend,xendomains,watchdog-xen to run as services

update-rc.d xencommons defaults 19 18
update-rc.d xend defaults 20 21
update-rc.d xendomains defaults 21 20
update-rc.d xen-watchdog defaults 22 23

You should also have configure bridge for xen DomUs manually via /etc/network/intefaces :-

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).
# The loopback network interface
auto lo
iface lo inet loopback
# The primary network interface
# auto eth0
# iface eth0 inet dhcp
auto br0
iface br0 inet static
address 192.168.1.7
netmask 255.255.255.0
network 192.168.1.0
broadcast 192.168.1.255
gateway 192.168.1.1
bridge_ports eth0
bridge_stp on
bridge_maxwait 0

Reboot in Xen with grub entry

menuentry "Xen 4.1 / Ubuntu 10.04 kernel 2.6.32.16-xen" {
insmod ext2
set root='(hd1,9)'
multiboot (hd1,9)/boot/xen.gz dummy=dummy
module (hd1,9)/boot/vmlinuz-2.6.32.16-xen dummy=dummy nopat root=/dev/sdb9 ro console=tty0
module (hd1,9)/boot/initrd.img-2.6.32.16-xen
}

Verify Xen environment :-

root@ServerLSX:~# xm info
host : ServerLSX
release : 2.6.32.16-xen
version : #10 SMP Wed Jul 7 22:17:57 MSD 2010
machine : x86_64
nr_cpus : 4
nr_nodes : 1
cores_per_socket : 4
threads_per_core : 1
cpu_mhz : 2833
hw_caps : bfebfbff:20100800:00000000:00000940:0408e3fd:00000000:00000001:00000000
virt_caps : hvm
total_memory : 8190
free_memory : 2048
free_cpus : 0
xen_major : 4
xen_minor : 1
xen_extra : -unstable
xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64
xen_scheduler : credit
xen_pagesize : 4096
platform_params : virt_start=0xffff800000000000
xen_changeset : Mon Jul 05 08:40:26 2010 +0100 21719:9d965ac1b0db
xen_commandline :
cc_compiler : gcc version 4.4.3 (Ubuntu 4.4.3-4ubuntu5)
cc_compile_by : root
cc_compile_domain :
cc_compile_date : Wed Jul 7 21:21:02 MSD 2010
xend_config_format : 4

root@ServerLSX:~# brctl show
bridge name bridge id STP enabled interfaces
br0 8000.90e6ba2d11eb yes eth0
virbr0 8000.000000000000 yes

Test Xen installation via creation NexentaStor 3.0.2 Xen PV Guest. Download sample via
# wget http://genunix.org/dist/nexenta/NexentaStor-Community-3.0.2-xen.tar.bz2
and uncompress tar.bz2 archive :-

root@ServerLSX:~/NexentaStor-Community-3.0.2# ls -l
total 2439888
-rw-r–r– 1 root root 21017088 2010-07-08 16:48 data_disk1.img
-rw-r–r– 1 root root 5120 2010-05-24 15:36 data_disk2.img
-rw-r–r– 1 root root 5120 2010-05-24 15:36 data_disk3.img
-rw-r–r– 1 root root 2573 2010-07-08 16:17 nexentastor-community-3.0.2-xen.cfg
-rw-r–r– 1 root root 1262 2010-05-24 16:15 readme.txt
-rw-r–r– 1 root root 2477388288 2010-07-08 16:48 syspool.img

# cat nexentastor-community-3.0.2-xen.cfg

work_dir = '/root/NexentaStor-Community-3.0.2'
# Names of disks
disk_1_name = 'syspool.img'
disk_2_name = 'data_disk1.img'
disk_3_name = 'data_disk2.img'
disk_4_name = 'data_disk3.img'
# Bootloader for boot DomU
bootloader = "/usr/local/bin/pygrub"
# Memory for DomU, in MB
memory = 1024
# Name of DomU
name = "NexentaStor-3.0.2"
# Network interfaces for DomU
# You can change the mac address of your own needs.
# mac=00:16:3E:xx:xx:xx
vif = [
'mac=00:16:3E:00:00:01',
'mac=00:16:3E:00:00:02'
]
disk = [
'tap2:tapdisk:vhd:' + work_dir + '/' + disk_1_name + ',xvda,w',
'tap2:tapdisk:vhd:' + work_dir + '/' + disk_2_name + ',xvdb,w',
'tap2:tapdisk:vhd:' + work_dir + '/' + disk_3_name + ',xvdc,w',
'tap2:tapdisk:vhd:' + work_dir + '/' + disk_4_name + ',xvde,w',
]
on_reboot = 'destroy'
on_crash = 'destroy'
on_poweroff = 'destroy'

Start Nexenta DomU :-

root@ServerLSX:~/NexentaStor-Community-3.0.2# xm create -c nexentastor-community-3.0.2-xen.cfg
Using config file “./nexentastor-community-3.0.2-xen.cfg”.
Started domain NexentaStor-3.0.2 (id=1)
WARNING: Cannot extract revision on this hypervisor version: v-unstable, unexpected version format
v4.1-unstable chgset ‘Mon Jul 05 08:40:26 2010 +0100 21719:9d965ac1b0db’
SunOS Release 5.11 Version NexentaOS_134d 64-bit
Loading Kernel…
Hostname: myhost
Reading ZFS config: done.
Mounting ZFS filesystems: (6/6)

ZFS Open Storage Appliance (v3.0.2)

myhost console login: root
Password:
Last login: Sat Jul 10 00:31:02 on console
nmc@myhost:/$ show
Option ? network
Option ? interface
==== Interfaces ====
lo0: flags=2001000849 mtu 8232 index 1
inet 127.0.0.1 netmask ff000000
xnf0: flags=1004843 mtu 1500 index 2
inet 192.168.1.4 netmask ffffff00 broadcast 192.168.1.255
ether 0:16:3e:0:0:1
xnf1: flags=1000842 mtu 1500 index 3
inet 0.0.0.0 netmask 0
ether 0:16:3e:0:0:2
lo0: flags=2002000849 mtu 8252 index 1
inet6 ::1/128
nmc@myhost:/$









Set up Libvirt 0.8.1 along with with all related packages via Daniel Baumann PPA https://launchpad.net/~dnjl/+archive/virtualization. It seems to provide virt-install utility working OK under Xen 4.1-unstable with 2.6.32.16 pvops kernel, regardless “virsh” fails to work properly with xenstore and consequently virt-manager 0.8.4 is useless for 4.1

root@ServerLX:~# apt-key adv --keyserver keyserver.ubuntu.com --recv-keys D18C7A1DF6E6BED2
Executing: gpg --ignore-time-conflict --no-options --no-default-keyring --secret-keyring /etc/apt/secring.gpg --trustdb-name /etc/apt/trustdb.gpg --keyring /etc/apt/trusted.gpg --primary-keyring /etc/apt/trusted.gpg --keyserver keyserver.ubuntu.com --recv-keys D18C7A1DF6E6BED2
gpg: requesting key F6E6BED2 from hkp server keyserver.ubuntu.com
gpg: key F6E6BED2: public key "Launchpad ppa" imported
gpg: Total number processed: 1
gpg: imported: 1 (RSA: 1)
root@ServerLX:~# apt-get update
Hit http://us.archive.ubuntu.com lucid Release.gpg
Get:1 http://ppa.launchpad.net lucid Release.gpg [307B]
Ign http://ppa.launchpad.net/dnjl/virtualization/ubuntu/ lucid/main Translation-en_US
Hit http://ppa.launchpad.net lucid Release.gpg
Hit http://security.ubuntu.com lucid-security Release.gpg
Ign http://us.archive.ubuntu.com/ubuntu/ lucid/main Translation-en_US
Ign http://security.ubuntu.com/ubuntu/ lucid-security/main Translation-en_US
Ign http://security.ubuntu.com/ubuntu/ lucid-security/restricted Translation-en_US
Ign http://us.archive.ubuntu.com/ubuntu/ lucid/restricted Translation-en_US
Ign http://ppa.launchpad.net/gwibber-daily/ppa/ubuntu/ lucid/main Translation-en_US
Get:2 http://ppa.launchpad.net lucid Release [57.3kB]
Ign http://security.ubuntu.com/ubuntu/ lucid-security/universe Translation-en_US
Ign http://security.ubuntu.com/ubuntu/ lucid-security/multiverse Translation-en_US
Hit http://security.ubuntu.com lucid-security Release
Ign http://us.archive.ubuntu.com/ubuntu/ lucid/universe Translation-en_US
Hit http://ppa.launchpad.net lucid Release
Ign http://us.archive.ubuntu.com/ubuntu/ lucid/multiverse Translation-en_US
Hit http://security.ubuntu.com lucid-security/main Packages
Hit http://ppa.launchpad.net lucid/main Packages
Hit http://ppa.launchpad.net lucid/main Sources
Hit http://us.archive.ubuntu.com lucid-updates Release.gpg
Hit http://security.ubuntu.com lucid-security/restricted Packages
Hit http://security.ubuntu.com lucid-security/main Sources
Hit http://security.ubuntu.com lucid-security/restricted Sources
Hit http://security.ubuntu.com lucid-security/universe Packages
Ign http://us.archive.ubuntu.com/ubuntu/ lucid-updates/main Translation-en_US
Hit http://ppa.launchpad.net lucid/main Packages
Hit http://security.ubuntu.com lucid-security/universe Sources
Hit http://security.ubuntu.com lucid-security/multiverse Packages
Hit http://security.ubuntu.com lucid-security/multiverse Sources
Ign http://us.archive.ubuntu.com/ubuntu/ lucid-updates/restricted Translation-en_US
Ign http://us.archive.ubuntu.com/ubuntu/ lucid-updates/universe Translation-en_US
Ign http://us.archive.ubuntu.com/ubuntu/ lucid-updates/multiverse Translation-en_US
Hit http://us.archive.ubuntu.com lucid Release
Hit http://us.archive.ubuntu.com lucid-updates Release
Hit http://us.archive.ubuntu.com lucid/main Packages
Hit http://us.archive.ubuntu.com lucid/restricted Packages
Hit http://us.archive.ubuntu.com lucid/main Sources
Hit http://us.archive.ubuntu.com lucid/restricted Sources
Hit http://us.archive.ubuntu.com lucid/universe Packages
Hit http://us.archive.ubuntu.com lucid/universe Sources
Hit http://us.archive.ubuntu.com lucid/multiverse Packages
Hit http://us.archive.ubuntu.com lucid/multiverse Sources
Hit http://us.archive.ubuntu.com lucid-updates/main Packages
Hit http://us.archive.ubuntu.com lucid-updates/restricted Packages
Hit http://us.archive.ubuntu.com lucid-updates/main Sources
Hit http://us.archive.ubuntu.com lucid-updates/restricted Sources
Hit http://us.archive.ubuntu.com lucid-updates/universe Packages
Hit http://us.archive.ubuntu.com lucid-updates/universe Sources
Hit http://us.archive.ubuntu.com lucid-updates/multiverse Packages
Hit http://us.archive.ubuntu.com lucid-updates/multiverse Sources
Fetched 308B in 5s (55B/s)
Reading package lists... Done

root@ServerLX:~# apt-get install virtinst python-virtinst virt-viewer
It’s important to specify “–connect xen:///” inside virt-install command line to succeed with VNC console launching (vs 9.10) . Variable VIRSH_DEFAULT_CONNECT_URI exported via .bashrc in my case caused virt-install to crash.Comment out (xend-unix-server yes) in /etc/xen/xend-config.sxp and export variable

export VIRSH_DEFAULT_CONNECT_URI="xen:///"

in root’s .bashrc. Then reboot Xen Host.

root@ServerLX:~# virsh version
Compiled against library: libvir 0.8.1
Using library: libvir 0.8.1
Using API: Xen 3.0.1
Running hypervisor: Xen 4.1.0

This entry was posted on Thursday, July 8th, 2010 at 1:34 pm and is filed under Uncategorized. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.

6 Responses to Set up Xen 4.1-unstable & 2.6.32.16 pvops Dom0 on top of Ubuntu 10.04 Server

  1. harley says:
    March 30, 2011 at 5:21 am

    why do you have a virbr0 and a br0 if you only created br0 with /etc/network/interfaces ? i have tried to reproduce this instead of using static and an address field i’ve used dhcp on br0 , in my win7 domU with vif = [bridge=br0] i am unable to get an ip address (even setting up interfaces with a static ip address i am unable to use networking with the win7 domu)

    😦

    Reply
  2. harley says:
    March 30, 2011 at 5:59 am

    oh ok someone helped me and pointed out the libvirt networking part of virbr0 aside from br0 http://wiki.libvirt.org/page/Networking#NAT_forwarding_.28aka_.22virtual_networks.22.29

    Reply
  3. Ron Arenas says:
    April 14, 2011 at 3:58 am

    Hi there! I got a new kernel 2.6.32.36 on Lucid 10.04 LTS compiled with the requirement specifications from Xen relative to Paravirt-Ops kernels with Xen 4.1.1-rc1-pre freshly baked, installed and running. For now, I experience reboots or system restart when I try to shutdown the server. I also see a slew of messages from dmesg relative to the following:
    a) “XENBUS: Unable to read cpu state”
    b) radeon 0000:08:09.0: VGA-1: EDID invalid

    Hope you can shed some light on this, as I am still doing alot of other tests with this build.

    Many thanks in advance for any help.

    Ron

    Reply
  4. webmin handy says:
    August 23, 2012 at 6:52 am

    We’re a group of volunteers and opening a new scheme in our community. Your website provided us with valuable info to work on. You’ve done a formidable job and
    our entire community will be grateful to you.

    Reply
  5. Katherin says:
    July 22, 2014 at 11:57 am

    I don’t know whether it’s just me or if perhaps everyone else encountering issues with your
    website. It appears as if some of the written text in your content are running off the screen. Can somebody else please comment and let
    me know if this is happening to them too? This may be a problem with my web browser because I’ve had this happen previously.
    Kudos

    Reply

Leave a comment