Configuring Xen 4.0.1 & 2.6.32.23 pvops kernel for stable networking with OpenIndiana/OpenSolaris PV Guests on top of Ubuntu 10.04.1 Server

This post is targeting pvops kernel 2.6.32.23 configuration close to 2.6.34.7 aka Suse 11.3 Xenified kernel. This config by some reasons allows OpenSolaris134/OpenIndiana147 PV Guest obtain IP address pretty stable on systems Xen 4.0.1 & PVOPS like it happens on OpenSuse 11.3 with Xen Hypervisor upgraded up to 4.0.1 and the most recent XenLinux 2.6.34.7 kernel. PVOPS systems with mentioned bellow drivers hard linked to the kernel have permanent trouble with providing
IP via DHCP request,issued by OSOL/OI_147 PV guest, to this guest at least through my experience. Actually , only xen-netback module loaded separately from kernel is important. I just tried to change traditional PVOPS config , working perfectly for Linux guests, a bit more then usual and see what happens. The last step obviously requires changing xen-backend.rules. Testing has been done on top of Ubuntu 10.04.1 Server. On Fedoras 13,14 in case of using Michael Young’s Dom0 kernel and xen-4.0.1 via Fedora repos there is nothing to wary about, in case of git checkout via Jeremy Fitzhardinge repository you are responsible for .config tuning 2.6.32.23 pvops kernel.

Apply following patch to xen-4.0.1 folder

--- .orig/tools/hotplug/Linux/xen-backend.rules 2010-10-05 14:08:56.436012895 +0400
+++ /tools/hotplug/Linux/xen-backend.rules 2010-10-05 11:48:38.694087211 +0400
@@ -9,3 +9,5 @@
KERNEL=="evtchn", NAME="xen/%k"
KERNEL=="blktap[0-9]*", NAME="xen/%k"
KERNEL=="pci_iomul", NAME="xen/%k"
+KERNEL=="gntdev", NAME="xen/%k"
+KERNELS=="xen", KERNEL=="xvd*", SUBSYSTEM=="block", OPTIONS+="last_rule"

Rebuild and reinstall xen and tools
Change standard tuning of pvops kernel as follows:-

CONFIG_XEN_BALLOON=y
CONFIG_XEN_SCRUB_PAGES=y
CONFIG_XEN_DEV_EVTCHN=m
CONFIG_XEN_BACKEND=y
CONFIG_XEN_NETDEV_BACKEND=m
CONFIG_XEN_BLKDEV_BACKEND=m
CONFIG_XEN_BLKDEV_TAP=m
CONFIG_XEN_BLKBACK_PAGEMAP=m
CONFIG_XEN_PCIDEV_BACKEND=y
CONFIG_XEN_PCIDEV_BACKEND_VPCI=y
# CONFIG_XEN_PCIDEV_BACKEND_PASS is not set
# CONFIG_XEN_PCIDEV_BACKEND_SLOT is not set
# CONFIG_XEN_PCIDEV_BACKEND_CONTROLLER is not set
CONFIG_XEN_PCIDEV_BE_DEBUG=y
CONFIG_XENFS=m
CONFIG_XEN_COMPAT_XENFS=y
CONFIG_XEN_SYS_HYPERVISOR=y
CONFIG_XEN_MCE=y
CONFIG_XEN_XENBUS_FRONTEND=y
CONFIG_XEN_GNTDEV=m

Build and install

cd /usr/src/linux-2.6-xen
aptitude install kernel-package fakeroot
export CONCURRENCY_LEVEL=(number_of_cores) +1
make-kpkg clean
fakeroot make-kpkg --initrd --append-to-version=-pv kernel-image kernel-headers
dpkg -i ../linux-image-2.6.32.23-pv_2.6.32.23-pv-10.00.Custom_amd64.deb



Apply following patch to /etc/init.d/xend

--- xend.orig 2010-10-06 18:44:35.055245544 +0400
+++ xend 2010-10-06 18:46:26.056620024 +0400
@@ -21,7 +21,7 @@

shopt -s extglob
test -f /etc/sysconfig/xend && . /etc/sysconfig/xend
-
+/sbin/modprobe -v xenfs
if test "x$1" = xstart && \
test -d /proc/xen && \
! test -d /proc/xen/capabilities && \
@@ -57,6 +57,11 @@
test -z "$XENCONSOLED_TRACE" || export XENCONSOLED_TRACE
[[ "$XENSTORED_TRACE" == @(yes|on|1) ]] && export XENSTORED_TRACE
[[ "$XENBACKENDD_DEBUG" == @(yes|on|1) ]] && export XENBACKENDD_DEBUG
+ /sbin/modprobe -v xen-evtchn
+ /sbin/modprobe -v xen-gntdev
+ /sbin/modprobe -v xen-blkback
+ /sbin/modprobe -v xen-netback
+ /sbin/modprobe -v blktap
xend start
await_daemons_up
;;

Another option on Ubuntu 10.04.1 Server configure /etc/modules as follows

root@ServerLLX:~# cat /etc/modules
# /etc/modules: kernel modules to load at boot time.
#
# This file contains the names of kernel modules that should be loaded
# at boot time, one per line. Lines beginning with "#" are ignored.
loop
lp
rtc
xen-evtchn
xen-gntdev
xen-netback
xen-blkback
xenfs
blktap

Reboot the system and make sure :-

root@ServerLLX:~# ls -l /dev/xen
total 0
crw-rw---- 1 root root 10, 57 2010-10-05 13:29 evtchn
crw-rw---- 1 root root 10, 56 2010-10-05 13:29 gntdev

root@ServerLLX:~# lsmod|grep xen
xenfs 10142 1
xen_blkback 19942 0 [permanent]
blkback_pagemap 2592 2 blktap,xen_blkback
xen_netback 28778 0 [permanent]
xen_gntdev 7172 0
xen_evtchn 4806 1

root@ServerLLX:~# lsmod|grep blktap
blktap 22229 0
blkback_pagemap 2592 2 blktap,xen_blkback




I have to notice that the most recent pvops kernels 2.6.32.23,(24) don’t require patching xen-backend to create device /dev/xen/gntdev. It appears to be created automatically , even when xen-gntdev has been created as a module. However, attempt to disable xen hotplug patch affects starting domains, created by virt-manager, right away.