The most impressive F11 Xen related features seem to be the nice Xen 3.4.1 build with python 2.6 coming as default with F11 and graphical installer behavior during pygrub based PV DomU installation phase. F11 is supposed to be installed without libvirt to avoid conflict during Xen 3.4.1 port to Fedora 11 instance. Xen 3.3.1 Hypervisor working with libvirt may be installed on F11 via xen-3.3.1-11.fc11.src.rpm. Notice, that mentioned version of Xen 3.3.1 is already patched to work with pvops kernels and may be also patched for pygrub ZFS support. View [1] for details . As appears setting default path for python packages to /usr/lib/python2.6/site-packages resolves issues with Xen build raising up on Ubuntu 9.04 Server (/usr/local/lib/python2.6/dist-packages).
*****************************************************************************
Update on 08/24/2009 . View the most recent post :-
Fedora 11 as the best target for Xen 3.4.1 & Libvirt 0.7.0-6 deployment
I have to notice that Libvirt 0.7.0-6 (in other words virt-install,virt-manager) been able to work with Xen 3.4.1 Hypervisor is obvious advantage F11 vs Ubuntu Carmic,Jaunty,Intrepid,Hardy Servers. Virt-install was broken in Hardy and afterwards was not supposed to work with Xen at all.
*****************************************************************************
Dependencies unacceptable for Xen 3.4.1
yum install python-virtinst
. . . . . .
Dependencies Resolved
===================================================
Package Arch Version Repository Size
===================================================
Installing:
python-virtinst noarch 0.400.3-8.fc11 fedora 401 k
Installing for dependencies:
iscsi-initiator-utils x86_64 6.2.0.870-8.fc11 fedora 750 k
libvirt x86_64 0.6.2-11.fc11 updates 1.8 M
libvirt-python x86_64 0.6.2-11.fc11 updates 116 k
qemu-img x86_64 2:0.10.4-4.fc11 updates 100 k
-> xen-libs x86_64 3.3.1-11.fc11 fedora 176 k
Transaction Summary
=====================================================
Install 6 Package(s)
Update 0 Package(s)
Remove 0 Package(s)
Total download size: 3.4 M
Proceed with building Xen 3.4.1 Dom0 on top of F11.
# yum install gitk dev86 vnc-server bridge-utils
# cd /usr/src
# hg clone http://xenbits.xensource.com/xen-3.4-testing.hg
# cd xen-3.4-testing.hg
# make xen
# make install-xen
# make tools
# make install-tools
Building pvops enabled kernel.
1.To checkout master branch:-
# git clone git://git.kernel.org/pub/scm/linux/kernel/git/jeremy/xen.git linux-2.6-xen
# cd linux-2.6-xen
# git checkout origin/xen-tip/master -b xen-tip/master
2.To checkout the most recent branch:-
# git clone git://git.kernel.org/pub/scm/linux/kernel/git/jeremy/xen.git linux-2.6-xen
# cd linux-2.6-xen
# git checkout origin/rebase/master -b rebase/master
To setup Xen Dom0 support :-
Activating Xen Dom0 Support for pvops kernel:-
1. Processor Type and features -> Paravirtualized guest support->Enable Xen Priveleged Domain Support <*>
2.Device Drivers -> Block Devices->
Xen Virtual Block Device Support <*>
3.Device Drivers -> [*] Backend driver support
<*>Block-device backend driver
<*> Xen backend network device
<*> Xen filesystem
[*] Create compatibility mount point /proc/xen
[*] Create xen entries under /sys/hypervisor
[*] userspace grant access device driver
[*] Staging drivers --->
[*] X86 Platform Specific Device Drivers --->
# make menuconfig
# make
# make modules_install install
Install 2.6.29.4 xen-ified kernel :-
# wget http://x17.eu/linux-2.6.29-xen-r4-aka-suse-xenified-2.6.29-62.1.tar.gz
# tar -zxvf linux-2.6.29-xen-r4-aka-suse-xenified-2.6.29-62.1.tar.gz
# cd linux-2.6.29-xen-r4-aka-suse-xenified-2.6.29-62.1
# make O=~user1/build menuconfig
# make O=~user1/build
# make O=~user1/build modules_install install
Tuning 2.6.29.4 xen-ified kernel :-
Subarchitecture Type (Enable Xen compatible kernel)
( ) PC-compatible
(X) Enable Xen compatible kernel
( ) Support for ScaleMP vSMP
Device Drivers --->
XEN --->
[*] Privileged Guest (domain 0)
<*>Backend driver support
<*>Block-device backend driver
<*>Block-device tap backend driver
<*> Network-device backend driver
Add to /etc/fstab :-
none /proc/xen xenfs defaults 0 0
Create a grub entry:-
title Xen 3.4 / Fedora kernel 2.6.30-rc6-tip
kernel /boot/xen-3.4.gz
module /boot/vmlinuz-2.6.30-rc6-tip root=/dev/mapper/vg_fedora11-LogVol00 ro console=tty0
module /boot/initrd-2.6.30-rc6-tip.img
Set initdefault to 3 in /etc/inittab and reboot in Xen environment,
having /etc/rc.local to start:-
export HOME=/root
vncserver :1 -geometry 1280x1024 -depth 16
/etc/init.d/xend start
/etc/init.d/xendomains start
Xend and xendomains would be better to setup running as services :-
# chkconfig xend on
# chkconfig xendomains on
View also :-
Remote Login with GDM and VNC on Fedora 11 regarding standard setup resumable VNC session. It seemed to me too much unstable. Several actions required root authorization caused VNC session to interrupt.
File /etc/gdm/custom.conf didn’t contain [daemon] section. There was no any instruction regarding RemoteGreeter settings.
Connect to Xen Host remotely via vncviewer
Bring up local Apache Server to create HTTP installation source.
# chkconfig httpd on
# service httpd start
# mount -o loop f11.iso /var/www/html/f11
# wget http://192.168.1.36/f11/images/pxeboot/vmlinuz
# wget http://192.168.1.36/f11/images/pxeboot/initrd.img
Create installation profile:-
name="F11PV"
memory=2048
disk = ['phy:/dev/sdc7,xvda,w' ]
vif = [ 'bridge=eth0' ]
vfb = [ 'type=vnc,vncunused=1']
kernel = "/home/boris/fedora/vmlinuz"
ramdisk = "/home/boris/fedora/initrd.img"
vcpus=2
on_reboot = 'restart'
on_crash = 'restart'
# xm create f11.install
# vncviewer localhost:0
This time graphical installer will be brought up with no issues (vs F10). Been prompted by installer about source : choose URL and submit http://IP-Dom0/f11
Load DomU via profile:-
name="F11PV"
memory=2048
disk = ['phy:/dev/sdc7,xvda,w' ]
vif = [ 'bridge=eth0' ]
vfb = [ 'type=vnc,vncunused=1']
bootloader = "/usr/bin/pygrub"
vcpus=2
on_reboot = 'restart'
on_crash = 'restart'
# xm create f11.pyrun
# vncviewer localhost:0
OpenSolaris 2009.06 PV DomU at the same Xen Host:-
References
1. Backport ZFS support to Xen 3.3.1 F10 Dom0 (kernel 2.6.30-rc3-tip)
- Comment
- Reblog
-
Subscribe
Subscribed
Already have a WordPress.com account? Log in now.
Xen Virtualization on Linux and Solaris 
I do not see the config images you display. Then, after I “make,” I get an error swiotlb_bus_to_phys undefined reference. Any ideas?
Post your problem to xen-devel mailing list and mention the branch you’ve checked out. xen-tip/next may be updated any time. Personally, i’ve never had problems with building Jeremy’s kernel with the most recent xen-tip/next. You may try xen-tip/master
I’ve just ran
# cd /usr/src/linux-2.6-xen
# git pull
All is up to date
# make
No problems. I’ve also tried to send you complete .config via e-mail and
got Mail Delivery Failure.
Thanks for the help. It seems that my linux version is 2.6.29.4, would that cause a problem? I still cannot see the “xen” configuration menu that you show screen shots of. I compared your conf file and mine, and I adapted your settings when appropriate, but they differ significantly.
FYI: I installed the newest version of FC11. My email is derek.riley (at) vanderbilt(dot)edu
Thanks!
What kernel you attempt to tune 2.6.30-rc6-tip (pvops) or
2.6.29.4 (xenified) ?
If the last one view better :-
I am somewhat confused by the kernel line in your grub.conf: xen-3.4.gz <– is this the same as vmlinuz-2.6.29-xen-r4 ? I have been following your guide and that kinda came from nowhere. I am following this on a 32bit version of Fedora Release 11. Please help!
# Xen Hypervisor loading first
kernel /xen-3.4.gz
# Xenified kernel and it’s initramdisk follow as modules
module /vmlinuz-2.6.29-xen-r4
module /initrd-2.6.29-xen-r4.img
thanks for clearing that up, for some reason make install did not copy that over to /boot. Weird eh? I copied it over and I can boot from grub, but got a new problem, the boot chokes on a stack trace of some sort, and i cannot scroll up to see where it started from. 😦 I know that is a little vague (ok it is REALLY vague), should I wait a week or so for the sources to get updated and start from scratch? I am open for any ideas at this point, I am so close!
>thanks for clearing that up, for some reason make install >did not copy that over to /boot. Weird eh?
“make install” fails only when something wrong happened to
“make”. It’s just “cp”.
> i cannot scroll up to see where it started from.
Setup serial console
>should I wait a week or so for the sources to get updated
You have nothing to wait. Fix your mistakes right now.
vmlinuz-2.6.29-xen-r4 seems to be broken in meantime.
Build xenified kenel via applying Suse’s rebased patches :-
wget http://gentoo-xen-kernel.googlecode.com/files/xen-patches-2.6.29-6.tar.bz2
to wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.29.4.tar.bz2
This appears to be a fix for an F11 dom0 + F11 domU. What of another distro as dom0 + f11 domU? I get these nice disk errors on boot up.
screenshot here:- http://jayeola.org/blog/wp-content/uploads/2009/07/ss-f11_0.png
I have F11 DomU at Xen 3.4.1 Dom0 on top of Ubuntu 9.04 Server with xenified 2.6.29.4 and 2.6.30-rc6-tip pvops enabled.
DomU has been installed vis Local Apache Mirror (same way).
What is your Xen Dom0 in details ?
DRDY ERROR attempting to connect to Dom0 block backend driver, that’s what i see.
The best option – please, post your detailed installation procedure for Dom0 and DomU to xen-users mailing list ( subscription is free), then i’ll respond you in public.
OK ?
Hello I am trying this, I successfully compiled and installed the Xen 3.4.1 and linux-2.6-xen kernel which works fine .
Then why this xenified kernel(linux-2.6.29-xen-r4-aka-suse-xenified-2.6.29-62.1) is required. I am able to compile it and install but how it will help? i am not able to figure out why it is required?
please help
thanks
-Dipak
It’s not required.
Hi dbaxps:
Thank you for this post. I was beginning to think that on my XPS720 (Dell) I would never get xen running – and I tried
several approaches/disctributions/Live CD (which worked, but
its not HDD based). Following your steps to the letter
(making only directory/path specific changes), things worked
well.
A couple of points you might want to consider adding/revising
for even greater clarity of your post:
– The “/etc/init.d/xend start” and
“/etc/init.d/xendomains start” were not mentioned.
– I didn’t understand the purpose, nor made use of, the
“Install/Tune 2.6.29.4 xen-ified kernel” section.
Perhaps you can elaborate for my edification, and for
others too – as it may be significant.
– Finally, when starting the “xendomains” service, I get
a [FAILED] status, though, it doesn’t say the cause.
I suspect it’s because I don’t have any domU domains
configured. I’ll have to visually single-step the
“/etc/init.d/xendomains” script to see
In any event, my gratitude for this post. Thank you very
much.
Noel
I forgot one bullet…
Recognizing that the xen “make install” part of the above procedure would clobber xen tool binaries that were part of FC RPM’s, I removed numerous stock xen-??-?? FC11 stock RPM packages. The only relevant ones I have left are:
xen-libs
Dependencies:
libvirt
libvirt-python
python-virtinst
I could not remove xen-libs or libvirt because
python-virtinst depend on them. I removed other xen/virt
stock tools too.
The most recent Xen 3.4.1 will be definitely followed by
xen-3.4.1-(X).fc12.src.rpm. It will provide an option for libvirt stuff working Xen 3.4.1 Hypervisor.
View for details :-
https://bugzilla.redhat.com/show_bug.cgi?id=511611
I mentioned 2.6.29.6 (.30.2) xenified (aka Suse) kernels because mentioned kernels include opensource Nvidia drivers, so Xen Hosts boots right away into Gnome or KDE desktop. VNC setup and LAN to work remotely are not required in this case. PVOPS kernels ( including the most recent 2.6.31-rc5) does require Xvnc Server setup to utilize remote X-Server for GDM running locally.
Regarding your statement above: “I mentioned 2.6.29.6 (.30.2) xenified (aka Suse) kernels”… If I read you correctly, one would opt to boot a SUSE xen-ified kernel for dom0 (not domU) due to the integration of Nvidia drivers (thus avoiding the need to set up VNC).
(1) Correct?
If I stated the above right, then for dom0 I would definitely be interested in SUSE xen-ified for that reason, so…
(2) How are you reliably determining what the latest stable (and unstable) SUSE xen-ified kernels are? I went to “http://x17.eu” to browse, but the listing is forbidden. Poking around randomly, I did find a link for 2.6.29.62.1 (because that’s only because I googled that very term – I wouldn’t have known it otherwise). So what site are you using?
Thanks & Regards,
Noel
>(1) Correct?
Yes
>If I stated the above right, then for dom0 I would >definitely be interested in SUSE xen-ified for that >reason, so…
>(2) How are you reliably determining what the latest >stable (and unstable) SUSE xen-ified kernels are? I >went to “http://x17.eu” to browse, but the listing is >forbidden. Poking around randomly, I did find a link >for 2.6.29.62.1 (because that’s only because I googled >that very term – I wouldn’t have known it otherwise). >So what site are you using?
Please, view for details:-
Actually, i prefer to apply Andy Lyon’s patch sets myself:-
http://code.google.com/p/gentoo-xen-kernel/downloads/list
Andy states that all 2.6.30.X patch sets are buggy,
just 2.6.29.6 is good. However, i have some things working with 2.6.30.X patch sets and failing with 2.6.29.6. Would you have any more questions my e-mail is bderzhavets@yahoo.com.
There is also a simple way for navigating in my blog:-
http://lxer.com/module/newswire/byuser.php?user=dba477
I have installed xen as per above procedure, i am getting
“(XEN) I/O virtualisation disabled” when i do “xm dmesg”
I have VT-d in BIOS, which is detected by xen kernel as follows
(XEN) VMX: Supported advanced features:
(XEN) – APIC TPR shadow
(XEN) – MSR direct-access bitmap
(XEN) HVM: VMX enabled
then why pnly I/O Virtulization is disabled?
thanks
-Dipak
.
.
.
The best board i have is ASUS P5Q-E, so i never had concern
like yours. I even cannot work on this issue due to no VT-d support by BIOS.Post it to xen-users@list.xensouce.com.
Thanks for the writeup.
But booting up XEN following your steps results in the kernel
seeing only 4Gig memory. My system has 28Gigs & the CPU has PAE support :
[XEN ~]$ grep pae /proc/cpuinfo
flags : fpu de tsc msr pae cx8 apic mtrr cmov pat clflush acpi mmx fxsr sse sse2 ss ht nx constant_tsc up xtopology tsc_reliable nonstop_tsc pni est ssse3 sse4_1 sse4_2 hypervisor ida
I was hoping to see all the 28 gigs… Is it because of the kernel limitation ?
After following all the steps, the final Grub looks as follows. Though i see a grub entry for the kernel with pae support, the XEN grub entry (last one)’s kernel module do
not seem to have the PAE support. Any suggestions ?
Grub entries :
title Fedora (2.6.29-xen-r4)
root (hd0,0)
kernel /vmlinuz-2.6.29-xen-r4 ro root=/dev/mapper/vg_trifecta-lv_root nomodeset rhgb quiet
initrd /initrd-2.6.29-xen-r4.img
title Fedora (2.6.30-rc3-tip)
root (hd0,0)
kernel /vmlinuz-2.6.30-rc3-tip ro root=/dev/mapper/vg_trifecta-lv_root nomodeset rhgb quiet
initrd /initrd-2.6.30-rc3-tip.img
title Fedora (2.6.29.4-167.fc11.i686.PAE)
root (hd0,0)
kernel /vmlinuz-2.6.29.4-167.fc11.i686.PAE ro root=/dev/mapper/vg_trifecta-lv_root nomodeset rhgb quiet
initrd /initrd-2.6.29.4-167.fc11.i686.PAE.img
title Xen 3.4 / Fedora kernel 2.6.30-rc3-tip
root (hd0,0)
kernel /xen-3.4.gz vtd=1 iommu=1
module /vmlinuz-2.6.30-rc3-tip root=/dev/mapper/vg_trifecta-lv_root ro console=tty0
module /initrd-2.6.30-rc3-tip.img
Sorry, i never was concerned about PAE, working on 64-bit
hardware (Core2Duo, Core2Quad boxes)
I followed your steps, everything compiles, but I get following error while booting at grub loader
Error 11: Unrecognized device string
My grub entries
# grub.conf generated by anaconda
#
# Note that you do not have to rerun grub after making changes to this file
# NOTICE: You have a /boot partition. This means that
# all kernel and initrd paths are relative to /boot/, eg.
# root (hd0,0)
# kernel /vmlinuz-version ro root=/dev/mapper/lvmvolume-fedora–root
# initrd /initrd-version.img
#boot=/dev/sda
default=1
timeout=10
splashimage=(hd0,0)/grub/splash.xpm.gz
hiddenmenu
title Xen 3.4 / Fedora kernel 2.6.31
root (hd0, 0)
kernel /boot/xen-3.4.gz
module /boot/vmlinuz-2.6.31 root=/dev/mapper/lvmvolume-fedora–root ro
module /boot/initrd-2.6.31.img
title Fedora (2.6.31)
root (hd0,0)
kernel /vmlinuz-2.6.31 ro root=/dev/mapper/lvmvolume-fedora–root rhgb quiet
initrd /initrd-2.6.31.img
title Fedora (2.6.30.9-90.fc11.x86_64)
root (hd0,0)
kernel /vmlinuz-2.6.30.9-90.fc11.x86_64 ro root=/dev/mapper/lvmvolume-fedora–root rhgb quiet
initrd /initrd-2.6.30.9-90.fc11.x86_64.img
Excellent post!
But when I choose URL to install domU, it says “there was an error configuring your network interface”
can u help please!
Submit your URL.
It was
http://dom0IP/f11
FC 11 wasn’t able to configure my network,
so I removed the bridge = eth0 from vif and made it –
vif = [”]
and it worked
My other issue is with ‘Xenoprof’, my domU’s report processor type as not supported when I do
opcontrol –init
ophelp -r
do u have any experience with Xenoprof on domU?
>FC 11 wasn’t able to configure my network,
>so I removed the bridge = eth0 from vif and made it –
>vif = [”]
Please, post
# xm info
# brctl show.
Something is wrong with your xen-bridge.
[root@nishikant nishikant]# xm info
host : nishikant
release : 2.6.31
version : #4 SMP Mon Nov 2 15:51:44 EST 2009
machine : x86_64
nr_cpus : 8
nr_nodes : 1
cores_per_socket : 4
threads_per_core : 1
cpu_mhz : 1995
hw_caps : bfebfbff:20100800:00000000:00000140:040ce33d:00000000:00000001:00000000
virt_caps : hvm
total_memory : 8189
free_memory : 8
node_to_cpu : node0:0-7
node_to_memory : node0:8
xen_major : 3
xen_minor : 4
xen_extra : .0
xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64
xen_scheduler : credit
xen_pagesize : 4096
platform_params : virt_start=0xffff800000000000
xen_changeset : unavailable
cc_compiler : gcc version 4.4.1 20090725 (Red Hat 4.4.1-2) (GCC)
cc_compile_by : nishikant
cc_compile_domain : (none)
cc_compile_date : Mon Nov 2 10:15:55 EST 2009
xend_config_format : 4
[root@nishikant nishikant]# brctl show
bridge name bridge id STP enabled interfaces
eth0 8000.0023ae95d38d no peth0
vif13.0
pan0 8000.000000000000 no
virbr0 8000.000000000000 yes
[…] https://bderzhavets.wordpress.com/2009/06/10/setup-fedora-11-pv-domu-at-xen-3-4-1-dom0-kernel-2-6-30-… […]