Setup OSOL PV Guests (build<=132) via virsh on Xen 3.4.3 Dom0 on top of Ubuntu 9.10 Server

This posting is actually responding recent entry in Martin’s Blog
OpenSolaris 2009.06 domU on opensuse 11.2 dom0
Martin states:-
Then I tried out a number of current linux distributions, but except for openSuSE none had a dom0 kernel out of the box which really is a shame. Seems I need to look more closely into KVM with virtio support.

Article bellow tries to explain that due to efforts of Jan Beulich and Andy Lyon xenified aka Suse Kernel may be built on any Linux and along with the most recent stable Xen Hypervisor (3.4.2 and higher) port provide Xen Environment supporting OpenSolaris PV Guests , including the most recent unstable builds like 129,130,131 ( i mean vncserver behavior on OSOL PV DomU and GDM/VNC setup ). I also choosed Ubuntu Karmic Koala Server with Libvirt 0.7.0 capabilities, actually virsh capabilities, been connected to Xen 3.4.3 Hypervisor one more time reproduce nice schema of John Levon of OSOL PV Guest install at Linux Dom0.

Install Libvirt 0.7.0 along with KVM creates environment with default Hypervisor QEMU . Two steps in procedure bellow make Xen default Hypervisor on Ubuntu 9.10 . First is commenting out (xend-unix-server yes) in /etc/xen/xend-config.sxp , second export variable


in root’s .bashrc.
1. Then Install KVM with no intend to work with it
# apt-get install ubuntu-virt-server ubuntu-virt-mgmt
# adduser $USER kvm
2. Build Xen 3.4.3-rc2-pre.
First – install on Uubuntu 9.10 Server all packages required for Xen build:-

apt-get install libcurl4-openssl-dev \
xserver-xorg-dev \
python2.6-dev \
mercurial gitk \
build-essential \
libncurses5-dev \
uuid-dev gawk \
gettext texinfo bcc

Second step :-

# cd /usr/src
# hg clone
# cd xen-3.4-testing.hg
Set in
PYTHON = python
# make xen
# make tools
# make install-xen
# make install-tools

Tuning results Xen packages to be placed into /usr/local/lib/python2.6/dist-packages due to Changeset 19594 in xen-3.4-testing.hg. Otherwise, Xen packages would go to /usr/lib/python2.6/site-packages, which is not default location for python 2.6 on Ubuntu 9.10

3. Edit xend-config.sxp set (xend-unix-server yes) .
4. Patch kernel via
Untar linux- and apply patches.

# cd /usr/src/build/
# bzcat linux- | tar xf -
# mkdir -p xen-patches-2.6.31-10
# cd xen-patches-2.6.31-10
# bzcat ../xen-patches-2.6.31-10.tar.bz2 | tar xf -
# cd ../linux-

Run script

for P in `ls ../xen-patches-2.6.31-10/6*.patch1 | sort `
patch -p1 -s -i $P
if [ $? = 0 ]; then
echo $P applied
echo "Error processing "$P
exit 1

5.Edit Makefile and set EXTRAVERSION=.9-xen. Build kernel , previously tuned to support Xen at Dom0 , with built-in static command line (root=/dev/sdb11 ro console=tty0) and install.
Create /boot/grub/grub.cfg entry:-

menuentry "Xen 3.4 / Ubuntu 9.10 kernel xenified" {
insmod ext2
set root=(hd1,10)
multiboot (hd1,10)/xen-3.4.gz
module (hd1,10)/vmlinuz-
module (hd1,10)/initrd-

Reboot the system

root@ServerKoala:~# export VIRSH_DEFAULT_CONNECT_URI="xen:///"
root@ServerKoala:~# virsh version
Connecting to uri: xen:///
Compiled against library: libvir 0.7.0
Using library: libvir 0.7.0
Using API: Xen 3.0.1
Running hypervisor: Xen 3.4.0

Create XML startup file for OSOL 131 PV Guest:

<domain type='xen'>
<bootloader_args>--kernel=/platform/i86xpv/kernel/amd64/unix --ramdisk=/platform/i86pc/amd64/boot_archive </bootloader_args>
<clock offset='utc'/>
<disk type='block' device='cdrom'>
<driver name='phy'/>
<source dev='/dev/loop0'/>
<target dev='xvda' bus='xen'/>
<disk type='block' device='disk'>
<driver name='phy'/>
<source dev='/dev/sdb8'/>
<target dev='xvdb' bus='xen'/>
<interface type='bridge'>
<mac address='00:16:3e:34:0b:18'/>
<source bridge='eth0'/>
<script path='/etc/xen/scripts/vif-bridge'/>
<target dev='vif6.0'/>
<console type='pty' tty='/dev/pts/2'>
<source path='/dev/pts/2'/>
<target port='0'/>

Then issue :-
# virsh create OSOL131.xml
# virsh console OSOL131
Now follow John Levon run script adopted for Xen 3.4.3 to open VNC connection to DomU

domid=`virsh domid OSOL131`
ip=`/usr/bin/xenstore-read /local/domain/$domid/guest/ipv4/0/address`
echo $ip
port=`/usr/bin/xenstore-read /local/domain/$domid/guest/vnc/port`
echo $port
/usr/bin/xenstore-read /local/domain/$domid/guest/vnc/passwd
vncviewer $ip:$port

Start Caiman installer and complete PV Guest installation.

Following XML is used to define created OSOL 131 DomU

<domain type='xen'>
<clock offset='utc'/>
<disk type='block' device='disk'>
<driver name='phy'/>
<source dev='/dev/sdb8'/>
<target dev='xvdb' bus='xen'/>
<interface type='bridge'>
<mac address='00:16:3e:10:93:61'/>
<source bridge='eth0'/>
<script path='/etc/xen/scripts/vif-bridge'/>
<target dev='vif7.0'/>
<console type='pty' tty='/dev/pts/2'>
<source path='/dev/pts/2'/>
<target port='0'/>

Now issue :-
# virsh define OSOL-131-DEF.xml
# virsh start OSOL131
# virsh console OSOL131
Login into DomU
# vncserver
and run script bellow to open vnc connection

domid=`virsh domid OSOL131`
ip=`/usr/bin/xenstore-read /local/domain/$domid/guest/ipv4/0/address`
echo $ip
vncviewer $ip:1

Setup VNC via GDM configuration file at OSOL 132 DomU

root@ServerKoala:/home/boris/132# virsh define os132-def.xml
Connecting to uri: xen:///
Domain OS-132 defined from os132-def.xml

root@ServerKoala:/home/boris/132# virsh start OS-132
Connecting to uri: xen:///
Domain OS-132 started

root@ServerKoala:/home/boris/132# virsh console OS-132
Connecting to uri: xen:///
Connected to domain OS-132
Escape character is ^]
v3.4.3-rc2-pre chgset 'Wed Feb 03 09:53:37 2010 +0000 19878:35a62fbdb74d'
SunOS Release 5.11 Version snv_132 64-bit
Copyright 1983-2010 Sun Microsystems, Inc. All rights reserved.
Use is subject to license terms.
Hostname: opensolaris
Configuring devices.
Loading smf(5) service descriptions: 165/165
svccfg import warnings. See /var/svc/log/system-manifest-import:default.log .
Reading ZFS config: done.
Mounting ZFS filesystems: (6/6)
Creating new rsa public/private host key pair
Creating new dsa public/private host key pair

opensolaris console login: boris
Sun Microsystems Inc. SunOS 5.11 snv_132 January 2010

root@opensolaris:~# cat /etc/gdm/custom.conf
# GDM configuration storage
# AllowRoot=true
# AllowRemoteRoot=true

root@opensolaris:~# svcadm disable xvnc-inetd gdm
root@opensolaris:~# svcadm enable xvnc-inetd gdm


5 Responses to Setup OSOL PV Guests (build<=132) via virsh on Xen 3.4.3 Dom0 on top of Ubuntu 9.10 Server

  1. Martin says:

    Great post, and I should try this again. I built a patched kernel, xen 3.4.2 on ubuntu 9.10 but to my dismay the radeon driver only produced garbled output. I then ran out of time … so there’s my feeble excuse!

  2. dbaxps says:

    Martin, it works for Radeon HD 4650.
    I believe,that for the most recent models it would work
    as well, but i am not an expert in Radeon Graphical CPU architecture. Actually, card was selected for PVOps kernel, that was primary concern. JF doesn’t want to support NVIDIA via kernel driver under Xen. Xenified aka Suse kernels always supported NVIDIA cards (like inexpensive GeForce 8500 GT).Personally, been concerned about , i would look for in GeForce 8X00 (not lower). It comes out to be just a question of money (wasting money).

  3. Max says:

    If you wrote a movie, what would the title be?

  4. dbaxps says:

    Virsh and his friends on top Ubuntu Karmic Server 🙂

  5. Hmm is anyone else experiencing problems with the
    pictures on this blog loading? I’m trying to find out if its a problem on my end or if it’s
    the blog. Any responses would be greatly appreciated.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: