Set up Xen 4.0.1 & Libvirt 0.8.3 on top of Ubuntu 10.10 Desktop

Following bellow is brief instruction of mentioned set up loading xen back-end, evtchn, gntdev devices separately from pvops kernel 2.6.32.24 under Xen 4.0.1 on top Ubuntu Maverick Meerkat Desktop. Due to Desktop environment we have first to disable Network Manager and restart networking manually updating /etc/network/interfaces. This step is not required on the Ubuntu 10.10 Server. Libvirt 0.8.3 is available via Maverick Meerkat repositories. Two patches have been applied to official 4.0.1 tar’s ball. First one updates xen back-end rules, second one allows to load Ubuntu 10.10 PV Guest via standard trick with install same HVM first and using a ready image to load PV DomU via pygrub.

Disable Network manager

# chkconfig network-manager off
# gedit /etc/network/interfaces
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet dhcp

Restart netwoking :-

# /etc/init.d/networking restart

Install required packages

apt-get install libcurl4-openssl-dev \
xserver-xorg-dev \
mercurial gitk \
build-essential \
libncurses5-dev \
uuid-dev gawk \
gettext texinfo bcc \
libncurses5-dev dpkg-dev debhelper \
iasl texinfo bridge-utils bison flex
apt-get build-dep xen-3.3

Now git clone and checkout branch xen/stable

cd /usr/src
git clone
git://git.kernel.org/pub/scm/linux/kernel/git/jeremy/xen.git linux-2.6-xen
cd linux-2.6-xen
git checkout -b xen/stable-2.6.32.x origin/xen/stable-2.6.32.x

Build kernel in Debian way



Get .config file

aptitude install kernel-package fakeroot
export CONCURRENCY_LEVEL=(number_of_cores) +1
make-kpkg clean
fakeroot make-kpkg --initrd --append-to-version=-pv kernel-image kernel-headers
dpkg -i ../linux-image-2.6.32.24-pv_2.6.32.24-pv-10.00.Custom_amd64.deb

Edit /etc/modules

root@boris-System-P5Q3:~# cat /etc/modules
# /etc/modules: kernel modules to load at boot time.
# This file contains the names of kernel modules that should be loaded
# at boot time, one per line. Lines beginning with "#" are ignored.
lp
rtc
xen-evtchn
xen-gntdev
xen-netback
xen-blkback
xenfs
blktap

Extract Xen 4.0.1 archive :-

cd /usr/src
gunzip xen-4.0.1.tar.gz
tar -xvf xen-4.0.1.tar
cd xen-4.0.1

Apply following patch to xen-4.0.1 folder

--- .orig/tools/hotplug/Linux/xen-backend.rules 2010-10-05 14:08:56.436012895 +0400
+++ /tools/hotplug/Linux/xen-backend.rules 2010-10-05 11:48:38.694087211 +0400
@@ -9,3 +9,5 @@
KERNEL=="evtchn", NAME="xen/%k"
KERNEL=="blktap[0-9]*", NAME="xen/%k"
KERNEL=="pci_iomul", NAME="xen/%k"
+KERNEL=="gntdev", NAME="xen/%k"
+KERNELS=="xen", KERNEL=="xvd*", SUBSYSTEM=="block", OPTIONS+="last_rule"

At this point create grub2-fix.patch1 as raw content of http://xenbits.xensource.com/xen-unstable.hg?rev/6eaab8297681 Changeset 22238 pygrub: support grub2 “(hdX,msdosY)” partition syntax

patch -p1 -s -i ../grub2-fix.patch1
make xen
make tools
make install-xen
make install-tools PYTHON_PREFIX_ARG=

Set up xend,xendomains to run as services

update-rc.d xend defaults 20 21
update-rc.d xendomains defaults 21 20

Reboot in Xen with grub entry

menuentry "Xen 4.0.1 / Ubuntu 10.10 kernel 2.6.32.24" {
insmod ext2
set root='(hd1,5)'
multiboot (hd1,5)/boot/xen.gz dummy=dummy dom0_mem=1024M
module (hd1,5)/boot/vmlinuz-2.6.32.24-pv dummy=dummy root=/dev/sdb5 ro console=tty0
module (hd1,5)/boot/initrd.img-2.6.32.24-pv
}

Verify Xen environment :-

root@boris-System-P5Q3:~# xm info
host : boris-System-P5Q3
release : 2.6.32.24-pv
version : #1 SMP Thu Oct 14 20:47:56 MSD 2010
machine : x86_64
nr_cpus : 4
nr_nodes : 1
cores_per_socket : 4
threads_per_core : 1
cpu_mhz : 2833
hw_caps : bfebfbff:20100800:00000000:00000940:0408e3fd:00000000:00000001:00000000
virt_caps : hvm
total_memory : 8190
free_memory : 7070
node_to_cpu : node0:0-3
node_to_memory : node0:7070
node_to_dma32_mem : node0:3259
max_node_id : 0
xen_major : 4
xen_minor : 0
xen_extra : .1
xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64
xen_scheduler : credit
xen_pagesize : 4096
platform_params : virt_start=0xffff800000000000
xen_changeset : unavailable
xen_commandline : dummy=dummy dom0_mem=1024M
cc_compiler : gcc version 4.4.5 (Ubuntu/Linaro 4.4.4-14ubuntu5)
cc_compile_by : root
cc_compile_domain :
cc_compile_date : Fri Oct 15 10:17:36 MSD 2010
xend_config_format : 4

root@boris-System-P5Q3:~# brctl show
bridge name bridge id STP enabled interfaces
eth0 8000.90e6ba2d11eb no peth0
virbr0 8000.000000000000 yes

Install libvirt 0.8.3 :-

# apt-get install virtinst python-libvirt \
virt-viewer virt-manager

Comment out (xend-unix-server yes) in /etc/xen/xend-config.sxp and export variable VIRSH_DEFAULT_CONNECT_URI via root’s .bashrc

export VIRSH_DEFAULT_CONNECT_URI="xen:///"

Then reboot Xen Host.

root@boris-System-P5Q3:~# virsh version

Compiled against library: libvir 0.8.3
Using library: libvir 0.8.3
Using API: Xen 3.0.1
Running hypervisor: Xen 4.0.0





References
1.http://wiki.xen.org/xenwiki/Xen4.0
Tutorials to install Xen 4.0
* Xen 4.0 on Ubuntu 10.04: http://bderzhavets.wordpress.com/2010/04/24/set-up-ubuntu-10-04-server-pv-domu-at-xen-4-0-dom0-pvops-2-6-32-10-kernel-dom0-on-top-of-ubuntu-10-04-server/

About these ads

36 Responses to Set up Xen 4.0.1 & Libvirt 0.8.3 on top of Ubuntu 10.10 Desktop

  1. Srujan says:

    Hi Boris,

    I am trying to install xen-4.0.1 on Ubuntu Desktop 10.10 following the above tutorial. I tried to apply the hotplug patch (mentioned above). It gave error saying “malformed patch at line 4″. I edited the patch to the following, it still gives me error “malformed patch at line 8″

    — .orig/tools/hotplug/Linux/xen-backend.rules 2010-10-05 14:08:56.436012895 +0400
    +++ /tools/hotplug/Linux/xen-backend.rules 2010-10-05 11:48:38.694087211 +0400
    @@ -9,3 +9,5 @@
    +KERNEL==”evtchn”, NAME=”xen/%k”
    +KERNEL==”blktap[0-9]*”, NAME=”xen/%k”
    +KERNEL==”pci_iomul”, NAME=”xen/%k”
    +KERNEL==”gntdev”, NAME=”xen/%k”
    +KERNELS==”xen”, KERNEL==”xvd*”, SUBSYSTEM==”block”, OPTIONS+=”last_rule”

  2. dbaxps says:

    Add two lines to tools/hotplug/Linux/xen-backend.rules
    manually.

  3. Jonathan D. Lettvin says:

    Hi Boris,

    Your instructions are clear and easy to follow. Thank you. But hit a snag and hope you can help me. The error shown below is where I have stopped.

    I just did a fresh install of ubuntu 10.10, then your instructions up to “Build kernel in Debian way” (I recommend changing that line to “make menuconfig” and then what actions to take when in the menu).

    I did “make menuconfig”, exited, then “make”. Make failed after almost half an hour with the following:

    CC drivers/pci/xen-pcifront.o
    drivers/pci/xen-pcifront.c:372: error: variable ‘pci_frontend_ops’ has initializer but incomplete type
    drivers/pci/xen-pcifront.c:373: error: unknown field ‘enable_msi’ specified in initializer
    drivers/pci/xen-pcifront.c:373: warning: excess elements in struct initializer
    drivers/pci/xen-pcifront.c:373: warning: (near initialization for ‘pci_frontend_ops’)
    drivers/pci/xen-pcifront.c:374: error: unknown field ‘disable_msi’ specified in initializer
    drivers/pci/xen-pcifront.c:374: warning: excess elements in struct initializer
    drivers/pci/xen-pcifront.c:374: warning: (near initialization for ‘pci_frontend_ops’)
    drivers/pci/xen-pcifront.c:375: error: unknown field ‘enable_msix’ specified in initializer
    drivers/pci/xen-pcifront.c:375: warning: excess elements in struct initializer
    drivers/pci/xen-pcifront.c:375: warning: (near initialization for ‘pci_frontend_ops’)
    drivers/pci/xen-pcifront.c:376: error: unknown field ‘disable_msix’ specified in initializer
    drivers/pci/xen-pcifront.c:376: warning: excess elements in struct initializer
    drivers/pci/xen-pcifront.c:376: warning: (near initialization for ‘pci_frontend_ops’)
    drivers/pci/xen-pcifront.c: In function ‘pci_frontend_registrar’:
    drivers/pci/xen-pcifront.c:382: error: ‘xen_pci_frontend’ undeclared (first use in this function)
    drivers/pci/xen-pcifront.c:382: error: (Each undeclared identifier is reported only once
    drivers/pci/xen-pcifront.c:382: error: for each function it appears in.)
    drivers/pci/xen-pcifront.c: In function ‘pcifront_scan_bus’:
    drivers/pci/xen-pcifront.c:421: warning: unused variable ‘err’

    How do I proceed?

  4. dbaxps says:

    I’ve just forwarded to you ready .config file via xen-users
    mailing list.

  5. Jonathan D. Lettvin says:

    Thank you.

    I am trying it out now.
    Won’t know if it fixes the problem for a while.
    Nevertheless, thank you for responding quickly.

  6. dbaxps says:

    I also updated post with link to
    http://pastebin.com/mzn8WQXa
    right after snapshot. That’s working 64-bit config

  7. Jonathan D. Lettvin says:

    Sorry to say, but there is another error:

    LD init/built-in.o
    LD .tmp_vmlinux1
    drivers/built-in.o: In function `xen_acpi_processor_get_info’:
    /usr/src/linux-2.6-xen/drivers/acpi/processor_xen.c:204: undefined reference to `xen_pcpu_index’

  8. dbaxps says:

    Please, escalate to xen-devel mailing list.

  9. dbaxps says:

    Try
    # make oldconfig
    first

  10. dbaxps says:

    “Howto” you tried to follow did have been moderated.
    Please, view :-
    http://www.linuxtoday.com/infrastructure/2010101502135OSSV
    I would also suggest another one :-
    http://www.linuxtoday.com/infrastructure/2010101902635OSSV
    Per my opinion 2.6.34.7 xenified aka Suse is pretty stable
    and fast.

  11. Brian says:

    nice instructions
    Great post!
    Thanks for the info

  12. npopravka says:

    Well…
    The kernel boots fine.
    But I can’t get a working Grub menu entry :
    dpkg: version ‘/boot/xen.gz’ has bad syntax: invalid character in version number

    Any idea ?

  13. Ron says:

    Nice work. I’m still waiting patiently for XEN to be brought into the mainstream Ubuntu server installation.

    Is there a quick way to only install the necessary grub entry? As the majority reference symlinks back to xen-4.0.1.gz, why have them?

  14. Craig says:

    Thanks for the info, all was going fine until I tried to build XEN tools it exits with the following error.

    /usr/include/gnu/stubs.h:7: fatal error: gnu/stubs-32.h: No such file or directory
    compilation terminated.
    make[10]: *** [tcgbios.o] Error 1
    make[10]: Leaving directory `/usr/src/xen-4.0.1/tools/firmware/rombios/32bit/tcgbios’
    make[9]: *** [subdir-all-tcgbios] Error 2
    make[9]: Leaving directory `/usr/src/xen-4.0.1/tools/firmware/rombios/32bit’
    make[8]: *** [subdirs-all] Error 2
    make[8]: Leaving directory `/usr/src/xen-4.0.1/tools/firmware/rombios/32bit’
    make[7]: *** [subdir-all-32bit] Error 2
    make[7]: Leaving directory `/usr/src/xen-4.0.1/tools/firmware/rombios’
    make[6]: *** [subdirs-all] Error 2
    make[6]: Leaving directory `/usr/src/xen-4.0.1/tools/firmware/rombios’
    make[5]: *** [subdir-all-rombios] Error 2
    make[5]: Leaving directory `/usr/src/xen-4.0.1/tools/firmware’
    make[4]: *** [subdirs-all] Error 2

  15. dbaxps says:

    Are you on 32 bit system ?

  16. Craig says:

    no currently using 64 bit system.
    Linux virtual 2.6.35-22-generic #35-Ubuntu SMP Sat Oct 16 20:45:36 UTC 2010 x86_64 GNU/Linux

  17. Craig says:

    Found the answer at http://wiki.xensource.com/xenwiki/Xen4.0

    needed gcc-multilib

  18. Dnyanesh says:

    — .orig/tools/hotplug/Linux/xen-backend.rules 2010-10-05 14:08:56.436012895 +0400
    +++ /tools/hotplug/Linux/xen-backend.rules 2010-10-05 11:48:38.694087211 +0400
    @@ -9,3 +9,5 @@
    +KERNEL==”evtchn”, NAME=”xen/%k”
    +KERNEL==”blktap[0-9]*”, NAME=”xen/%k”
    +KERNEL==”pci_iomul”, NAME=”xen/%k”
    +KERNEL==”gntdev”, NAME=”xen/%k”
    +KERNELS==”xen”, KERNEL==”xvd*”, SUBSYSTEM==”block”, OPTIONS+=”last_rule”

    cant apply this patch

    dnyanesh@dnyanesh-desktop:~/Downloads/xen-4.0.1$ patch -p1 -s -i ../grub2-fix.patch1
    patch: **** malformed patch at line 8:

    dnyanesh@dnyanesh-desktop:~/Downloads/xen-4.0.1$

    what is missing?
    please help me

  19. Dnyanesh says:

    Hi,
    I solved the problem of patching but i got a new problem.
    When i boot into xen it shows only console output
    It doesn’t show gnome environment.
    What is the problem ?
    this is my grub menu entry

    menuentry “Xen 4.0.1 / Ubuntu 10.10 kernel 2.6.32.26″ {
    insmod ext2
    set root=’(hd0,msdos4)’
    multiboot (hd0,msdos4)/boot/xen.gz dummy=dummy dom0_mem=1024M
    module (hd0,msdos4)/boot/vmlinuz-2.6.32.26-pv dummy=dummy root=/dev/sda4 ro console=tty0
    module (hd0,msdos4)/boot/initrd.img-2.6.32.26-pv
    }

    please help me…..

  20. Dnyanesh says:

    hey,
    please help me.
    after booting in xen it doesn’t start gnome/gdm.
    It only shows consol tty0
    what i have to do to start gnome-session.
    please please help me….

  21. Clabman says:

    I have follow all the instructions in this guide.

    But when i boot on Xen, in grub menu, Xen looks like to load, but it stop at Dom0 has maximum 24 VCPUs. And i can’t do anything.

    Can you help me ?

    Thanks

    Regards

  22. Dnyanesh says:

    I am again building new xen-kernel using above instructions.
    whether can i use provided .config file on my PC.
    My PC is 32-bit PC.
    Before this i have build the xen-kernel successfully, but it doesn’t start the gnome-session ,It directly login to tty1 console.
    what should i do to fix this at this time.
    Please help me.
    thank you.

  23. Bklynnykid says:

    What are you supposed to do with the .config file after it is downloaded? I am new to this so please bear with me.

  24. dbaxps says:

    Place it into build directory and run :-
    # make oldconfig

  25. dbaxps says:

    > In the linux-2.6-xen directory?

    Yes.

    > It probably would make sense to add a step that explains
    > what to do with the .config file and which folder to
    > copy it to.

    I am sorry, it’s just a blog been written on my own.
    Detail manuals are written in another places by another
    persons :)

  26. Disco says:

    Followed the guide but I always come up to a device not found error.

    Described here : http://serverfault.com/questions/208777/installing-xen-4-0-1-from-source-on-ubuntu-10-10

  27. [...] 参考:http://bderzhavets.wordpress.com/2010/10/15/set-up-xen-4-0-1-libvirt-0-8-3-on-top-of-ubuntu-10-10-de…    本作品采用知识共享署名 2.5 中国大陆许可协议进行许可。 [...]

  28. my says:

    Do you have the 2.6.32.27-pv’s .config file?http://git.kernel.org/pub/scm/linux/kernel/git/jeremy/xen.git is already update their kernel from 2.6.32.24-pv to 2.6.32.27-pv, so there are a lot mistakes.

  29. my says:

    Or do you have linux-2.6.32.24-pv ops?please,give me a copy~

  30. Phucvdb says:

    Hi , I have implemented Your Instruction and don’t see any error while installation.When I reboot again with Xen Kernel and type:
    $ xm info
    Then Have the following error:
    Error: Unable to connect to xend: No such file or directory. Is xend running?
    More information :
    /boot/grub/grub.cfg:
    menuentry ‘Ubuntu, with Linux 2.6.32.43-pv’ –class ubuntu –class gnu-linux –class gnu –class os {
    recordfail
    insmod part_msdos
    insmod ext2
    set root=’(hd0,msdos1)’
    search –no-floppy –fs-uuid –set 353ccbf3-2a35-4c62-bcff-492ad7cb0a23
    multiboot /boot/xen.gz dummy=dummy dom0_mem=1024M
    module /boot/vmlinuz-2.6.32.43-pv dummy=dummy root=UUID=353ccbf3-2a35-4c62-bcff-492ad7cb0a23 ro quiet splash
    module /boot/initrd.img-2.6.32.43-pv
    }
    Please help me.
    thank you.

  31. Excellent post. I used to be checking constantly this blog and I’m inspired! Extremely useful info specifically the closing part :) I take care of such info a lot. I used to be looking for this particular information for a very long time. Thanks and good luck.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.