Virsh building Fedora 12 & CentOS 5.4 PV Guests at xVM 3.4.2 (b130)

December 27, 2009

Setup bellow might seem strange unless you will become aware of some ongoing issues with VNC-Console behavior during virt-install running on the most recent OSOL builds 130 and 129. View [1],[2] at xen-discuss forum@opensolaris.org. Actually , it clearly demonstrates that Xen Guest might be created and managed via virsh in graphical mode , regardless of VNC issues ( at Dom0) affecting virt-install for particular OpenSolaris (b130,129) xVM 3.4.2 implementation. Virt-install in nographics mode would bring us to absolutely same results , but virsh scripting in this case would be silently skipped. Create virsh installation profile for F12 DomU , previously downloaded vmlinuz and and initrd.img from local Apache Mirror to current directory :-

root@Opensolaris130:/home/boris/fedora# cat f12.virsh-install
<domain type='xen'>
<name>VM12S</name>
<os>
<type>linux</type>
<kernel>/export/home/boris/fedora/vmlinuz</kernel>
<initrd>/export/home/boris/fedora/initrd.img</initrd>
</os>
<memory>1048576</memory>
<vcpu>1</vcpu>
<devices>
<disk type='block' device='disk'>
<driver name='phy' type='zvol'/>
<source dev='/tank2/guest-pv'/>
<target dev='xvda' bus='xen'/>
</disk>
<interface type='bridge'>
<source bridge='rge0'/>
<mac address='00:16:36:43:2a:72'/>
</interface>
<input type='mouse' bus='xen'/>
</devices>
</domain>

Start install
# virsh create f12.virsh-install
# virsh console VM12S





Now F12 DomU will bring up vncserver to perform graphical install due to Dom0 doesn’t
have one properly initialized. VNC Console cannot be brought up via virt-install as it supposed to happen.





Notice, that

virt-install --paravirt --name VF12S --ram 1024 --nographics --os-type=linux --os-variant=fedora11 --network bridge -f /dev/zvol/dsk/tank3/guest-pv -l http://192.168.1.33/f12 --debug

will also suggest to bring up VNCSERVER at DomU during install and will finally create create same F12 domain without VFB specifified.

When done. Shutdown domain
Runtime profile :-

root@Opensolaris130:/home/boris/fedora# cat f12.virsh-run
<domain type='xen'>
<name>VM12S</name>
<currentMemory>1048576</currentMemory>
<memory>1048576</memory>
<bootloader>/usr/lib/xen/bin/pygrub</bootloader>
<clock offset="utc"/>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>restart</on_crash>
<distro>
<type>linux</type>
<variant>fedora11</variant>
</distro>
<vcpu>1</vcpu>
<devices>
<disk type='block' device='disk'>
<source dev='/dev/zvol/dsk/tank2/guest-pv'/>
<target dev='xvda' bus='xen'/>
</disk>
<interface type='bridge'>
<source bridge=''/>
<mac address='00:16:36:63:f0:87'/>
</interface>
<input type='mouse' bus='xen'/>
</devices>
</domain>

Load Domain :-
# virsh define f12.virsh-run
# virsh start VM12S
# virsh console VM12S

root@Opensolaris130:/home/boris/fedora# virsh list
Id Name State
----------------------------------
0 Domain-0 running
- F12PV shut off
- VM12S shut off
root@Opensolaris130:/home/boris/fedora# virsh start VM12S
Domain VM12S started
root@Opensolaris130:/home/boris/fedora# virsh console VM12S
PCI: Fatal: No config space access function found
Welcome to Fedora
Press 'I' to enter interactive startup.
Starting udev: [ OK ]
Setting hostname fedora: [ OK ]
Setting up Logical Volume Management: 2 logical volume(s) in volume group "vg_fedora" now active
[ OK ]
Checking filesystems
Checking all file systems.
[/sbin/fsck.ext4 (1) -- /] fsck.ext4 -a /dev/mapper/vg_fedora-lv_root
/dev/mapper/vg_fedora-lv_root: clean, 169695/1291808 files, 1096535/5165056 blocks
[/sbin/fsck.ext4 (1) -- /boot] fsck.ext4 -a /dev/xvda1
Remounting root filesystem in read-write mode: [ OK ]
Mounting local filesystems: [ OK ]
Enabling local filesystem quotas: [ OK ]
Enabling /etc/fstab swaps: [ OK ]
Entering non-interactive startup
Applying Intel CPU microcode update: [ OK ]
Starting monitoring for VG vg_fedora: 2 logical volume(s) in volume group "vg_fedora" monitored
[ OK ]
Starting portreserve: [ OK ]
Starting system logger: [ OK ]
Starting irqbalance: [ OK ]
Starting rpcbind: [ OK ]
Starting system message bus: ^G^G[ OK ]
Starting Avahi daemon... [ OK ]
Starting cups: [ OK ]
Starting HAL daemon: [ OK ]
Starting PC/SC smart card daemon (pcscd): [ OK ]
Retrigger failed udev events[ OK ]
Setting network parameters... [ OK ]
Starting NetworkManager daemon: [ OK ]
Starting NFS statd: [ OK ]
Starting RPC idmapd: [ OK ]
Enabling Bluetooth devices:
Starting sshd: [ OK ]
Starting abrt daemon: [ OK ]
Starting console mouse services: [ OK ]
Starting httpd: [ OK ]
[ OK ] crond: [ OK ]
[ OK ] atd: [ OK ]
New 'fedora:1 ()' desktop is fedora:1
Starting applications specified in /root/.vnc/xstartup
Log file is /root/.vnc/fedora:1.log
Fedora release 12 (Constantine)
Kernel 2.6.31.5-127.fc12.x86_64 on an x86_64 (/dev/hvc0)
fedora login: root
Password:
Last login: Sun Dec 27 17:14:50 on hvc0
[root@fedora ~]#








Now domU automatically starts vncserver with ~.vnc/xstartup , including line:-

/usr/bin/gnome-session &

instead of “xterm &”.
At this point switching to Dom0 VFB appears to be possible. Create new virsh profile f12vfb.virsh-run :-

<domain type='xen'>
<name>VM12FB</name>
<currentMemory>1048576</currentMemory>
<memory>1048576</memory>
<bootloader>/usr/lib/xen/bin/pygrub</bootloader>
<clock offset="utc"/>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>restart</on_crash>
<distro>
<type>linux</type>
<variant>fedora11</variant>
</distro>
<vcpu>1</vcpu>
<devices>
<disk type='block' device='disk'>
<source dev='/dev/zvol/dsk/tank2/guest-pv'/>
<target dev='xvda' bus='xen'/>
</disk>
<interface type='bridge'>
<source bridge=''/>
<mac address='00:16:36:63:f0:87'/>
</interface>
<input type='mouse' bus='xen'/>
<graphics type='vnc' port='-1' keymap='en-us'/>
</devices>
</domain>

# virsh define f12vfb.virsh-run
# virsh start VM12FB
# vncviewer localhost:0
Logged in and add non root user , then switch initidefault to 5 in /etc/inittab .
Shutdown domain and start it again:-



Virt-install CentOS 5.4 as follows:-

virt-install --paravirt --name RH54S \
--ram 1024 --nographics \
--network bridge -f /dev/zvol/dsk/tank3/guest-pv \
-l http://192.168.1.33/rhel --debug






Made same changes to DomU and remove “console=xvc0″ from kernel line of corresponding entry of
/boot/grub/menu.lst
Add line :

<graphics type='vnc' port='-1' keymap='en-us'/>

to rhel54-vfb.xml and change domain name say to RH54FB.

<domain type='xen'>
<name>RH54FB</name>
<currentMemory>1048576</currentMemory>
<memory>1048576</memory>
<bootloader>/usr/lib/xen/bin/pygrub</bootloader>
<clock offset="utc"/>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>restart</on_crash>
<distro>
<type>linux</type>
<variant>rhel5</variant>
</distro>
<vcpu>1</vcpu>
<devices>
<disk type='block' device='disk'>
<source dev='/dev/zvol/dsk/tank3/guest-pv'/>
<target dev='xvda' bus='xen'/>
</disk>
<interface type='bridge'>
<source bridge=''/>
<mac address='00:16:36:63:f0:87'/>
</interface>
<input type='mouse' bus='xen'/>
<graphics type='vnc' port='-1' keymap='en-us'/>
</devices>
</domain>


# virsh define rhel54-vfb.xml
# virsh start RH54FB
# vncviewer localhost:0




References
1.http://opensolaris.org/jive/thread.jspa?threadID=120192&tstart=0
2.http://opensolaris.org/jive/thread.jspa?threadID=120092&tstart=0


Virt-install at xVM 3.4.2 on top of OSOL 1002-130

December 26, 2009

Patch virt-install replacing launching vncviewer by virsh serial console put virt-install in work in vnc console mode. Another option is to run virt-install with –noautoconsole option , followed by “virsh console VF12 & > /dev/null &”, “vncviewer localhost:0″

--- virt-install	2009-12-26 19:12:09.624200907 +0300
+++ virt-install.work	2009-12-26 19:12:09.624388189 +0300
@@ -557,8 +557,8 @@ def vncviewer_console(dom):
  
     child = os.fork()
     if not child:
-        os.execvp("/usr/bin/vncviewer", ["/usr/bin/vncviewer",
-                                         "%s:%d" %(vnchost, vncport) ])
+        os.execvp("/usr/bin/virsh",
+                      ["/usr/bin/virsh", "console", "%s" %(dom.ID(),)])
         os._exit(1)
     return child

*************************
Now start virt-install :-
*************************
root@Opensolaris130:~# virt-install –paravirt –name VF12S –ram 1024 –vnc –os-type=linux –os-variant=fedora11 –network bridge -f /dev/zvol/dsk/tank2/guest-pv -l http://192.168.1.33/f12 –debug
Sat, 26 Dec 2009 19:46:48 DEBUG Using libvirt URI ‘xen’
Sat, 26 Dec 2009 19:46:48 DEBUG Requesting virt method ‘xen’
Sat, 26 Dec 2009 19:46:48 DEBUG Received virt method ‘xen’
Sat, 26 Dec 2009 19:46:48 DEBUG Hypervisor name is ‘xen’
Sat, 26 Dec 2009 19:46:48 DEBUG Setting size for existing storage to ’22.0′
Sat, 26 Dec 2009 19:46:48 DEBUG Detected storage as type ‘block’
Sat, 26 Dec 2009 19:46:48 DEBUG Could not open “/etc/sysconfig/keyboard” [Errno 2] No such file or directory: ‘/etc/sysconfig/keyboard’
Sat, 26 Dec 2009 19:46:48 DEBUG Could not open “/etc/X11/xorg.conf”: [Errno 2] No such file or directory: ‘/etc/X11/xorg.conf’
Sat, 26 Dec 2009 19:46:48 DEBUG Did not parse any usable keymapping.
Sat, 26 Dec 2009 19:46:48 DEBUG DistroInstaller location is a network source.

Starting install…
Sat, 26 Dec 2009 19:46:48 DEBUG Attempting to detect distro:
Sat, 26 Dec 2009 19:46:48 DEBUG Fetching URI: http://192.168.1.33/f12/.treeinfo
Sat, 26 Dec 2009 19:46:48 DEBUG Saved file to /var/tmp/virtinst-.treeinfo.6draB9
Retrieving file .treeinfo 100% |=========================| 1.2 kB 00:00
Sat, 26 Dec 2009 19:46:48 DEBUG Fetching URI: http://192.168.1.33/f12/images/pxeboot/vmlinuz
Sat, 26 Dec 2009 19:46:48 DEBUG Saved file to /var/tmp/virtinst-vmlinuz.eeo8aM
Retrieving file vmlinuz.. 100% |=========================| 3.3 MB 00:00
Sat, 26 Dec 2009 19:46:48 DEBUG Fetching URI: http://192.168.1.33/f12/images/pxeboot/initrd.img
Sat, 26 Dec 2009 19:46:48 DEBUG Saved file to /var/tmp/virtinst-initrd.img.opQ7ah
Retrieving file initrd.im 100% |=========================| 22 MB 00:00
Sat, 26 Dec 2009 19:46:48 DEBUG Creating guest from:

VF12S
1048576
1048576
e42dd5e0-f34e-8ea1-cfa6-2786d08dc38c

linux
/var/tmp/virtinst-vmlinuz.eeo8aM
/var/tmp/virtinst-initrd.img.opQ7ah
method=http://192.168.1.33/f12

destroy
destroy
destroy

linux
fedora11

1

Creating domain… 0 B 00:01
Sat, 26 Dec 2009 19:46:50 DEBUG Created guest, looking to see if it is running
Sat, 26 Dec 2009 19:46:50 DEBUG Launching console callback
Sat, 26 Dec 2009 19:46:50 DEBUG VNC Port: 5900; VNC host: localhost
Sat, 26 Dec 2009 19:46:50 DEBUG Saving XML boot config:

VF12S
1048576
1048576
e42dd5e0-f34e-8ea1-cfa6-2786d08dc38c
/usr/lib/xen/bin/pygrub

destroy
restart
restart

linux
fedora11

1

Initializing cgroup subsys cpuset
Initializing cgroup subsys cpu
Linux version 2.6.31.5-127.fc12.x86_64 (mockbuild@x86-4.fedora.phx.redhat.com) (gcc version 4.4.2 20091027 (Red Hat 4.4.2-7) (GCC) ) #1 SMP Sat Nov 7 21:11:14 EST 2009
Command line: method=http://192.168.1.33/f12
KERNEL supported cpus:
Intel GenuineIntel
AMD AuthenticAMD
Centaur CentaurHauls
ACPI in unprivileged domain disabled
BIOS-provided physical RAM map:
Xen: 0000000000000000 – 00000000000a0000 (usable)
Xen: 00000000000a0000 – 0000000000100000 (reserved)
Xen: 0000000000100000 – 0000000040000000 (usable)
DMI not present or invalid.
last_pfn = 0×40000 max_arch_pfn = 0×400000000
init_memory_mapping: 0000000000000000-0000000040000000
RAMDISK: 019ce000 – 04498000
No NUMA configuration found
Faking a node at 0000000000000000-0000000040000000
Bootmem setup node 0 0000000000000000-0000000040000000
NODE_DATA [0000000000008000 - 000000000001dfff]
bootmap [000000000001e000 - 0000000000025fff] pages 8
(7 early reserv69b000]

#6 [0000100000 - 00002d8000] PGTABLE ==> [0000100000 - 00002d8000]

Zone PFN ranges:

DMA 0×00000000 -> 0×00001000

DMA32 0×00001000 -> 0×00100000

Normal 0×00100000 -> 0×00100000

Movable zone start PFN for each node

early_node_map[2] active PFN ranges

0: 0×00000000 -> 0x000000a0

0: 0×00000100 -> 0×00040000

SMP: Allowing 1 CPUs, 0 hotplug CPUs

No local APIC present

APIC: disable apic facility

PM: Registered nosave memory: 00000000000a0000 – 000000000010000bio: create slab at 0

ACPI: Interpreter disabled.

xen_balloon: Initialising balloon driver.

vgaarb: loaded

SCSI subsystem initialized

usbcore: registered new interface driver usbfs

usbcore: registered new interface driver hub

usbcore: registered new device driver usb

PCI: System does not support PCI

PCI: System does not support PCI

NetLabel: Initializing

NetLabel: domain hash size = 128

NetLabel: protocols = UNLABELED CIPSOv4

NetLabel: unlabeled traffic allowed by default

pnp: PnP ACPI: disabled

NET: Registered protocol family 2

IP route cache hash table entries: 32768 (order: 6, 262144 bytes)
TCP established hash table entries: 131072 (order: 9, 2097152 bytes)
TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
TCP: Hash tables configured (established 131072 bind 65536)
TCP reno registered
NET: Registered protocol family 1
Trying to unpack rootfs image as initramfs…
Freeing initrd memory: 43816k freed
platform rtc_cmos: registered platform RTC device (no PNP device found)
audit: initializing netlink socket (disabled)
type=2000 audit(1261846010.279:1): initialized
HugeTLB registered 2 MB page size, pre-allocated 0 pages
VFS: Disk quotas dquot_6.5.2
Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
msgmni has been set to 2133
alg: No test for stdrng (krng)
Block layer SCSI generic (bsg) driver version 0.4 loaded (major 252)
io scheduler noop registered
io scheduler anticipatory registered
io scheduler deadline registered
io scheduler cfq registered (default)
pci_hotplug: PCI Hot Plug PCI Core version: 0.5
pciehp: PCI Express Hot Plug Controller Driver version: 0.4
acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
Console: switching to colour frame buffer device 100×37
console [tty0] enabled
Non-volatile memory driver v1.3
Linux agpgart interface v0.103
Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
brd: module loaded
loop: module loaded
input: Macintosh mouse button emulation as /devices/virtual/input/input0
Fixed MDIO Bus: probed
ehci_hcd: USB 2.0 ‘Enhanced’ Host Controller (EHCI) Driver
ohci_hcd: USB 1.1 ‘Open’ Host Controller (OHCI) Driver
uhci_hcd: USB Universal Host Controller Interface driver
PNP: No PS/2 controller found. Probing ports directly.
mice: PS/2 mouse device common for all mice
input: Xen Virtual Keyboard as /devices/virtual/input/input1
input: Xen Virtual Pointer as /devices/virtual/input/input2
rtc_cmos: probe of rtc_cmos failed with error -16
device-mapper: uevent: version 1.0.3
device-mapper: ioctl: 4.15.0-ioctl (2009-04-01) initialised: dm-devel@redhat.com
cpuidle: using governor ladder
cpuidle: using governor menu
usbcore: registered new interface driver hiddev
usbcore: registered new interface driver usbhid
usbhid: v2.6:USB HID core driver
nf_conntrack version 0.5.0 (8192 buckets, 32768 max)
CONFIG_NF_CT_ACCT is deprecated and will be removed soon. Please use
nf_conntrack.acct=1 kernel parameter, acct=1 nf_conntrack modul

Issuing ( via different terminal session)
# vncviewer localhost:0
starts F12 PV DomU install





Moreover VNC console stays alive at DomU reboot and final creating :-





******************************************************
Sequence of commands for unpatched virt-install:-
*******************************************************

root@Opensolaris130:~# virt-install –paravirt –name VF12A –ram 1024 –vnc –os-type=linux –os-variant=fedora11 –network bridge -f /dev/zvol/dsk/tank3/guest-pv -l http://192.168.1.33/f12 –noautoconsole –debug
Sun, 27 Dec 2009 09:19:36 DEBUG Using libvirt URI ‘xen’
Sun, 27 Dec 2009 09:19:36 DEBUG Requesting virt method ‘xen’
Sun, 27 Dec 2009 09:19:36 DEBUG Received virt method ‘xen’
Sun, 27 Dec 2009 09:19:36 DEBUG Hypervisor name is ‘xen’
Sun, 27 Dec 2009 09:19:36 DEBUG Setting size for existing storage to ’22.0′
Sun, 27 Dec 2009 09:19:36 DEBUG Detected storage as type ‘block’
Sun, 27 Dec 2009 09:19:37 DEBUG Could not open “/etc/sysconfig/keyboard” [Errno 2] No such file or directory: ‘/etc/sysconfig/keyboard’
Sun, 27 Dec 2009 09:19:37 DEBUG Could not open “/etc/X11/xorg.conf”: [Errno 2] No such file or directory: ‘/etc/X11/xorg.conf’
Sun, 27 Dec 2009 09:19:37 DEBUG Did not parse any usable keymapping.
Sun, 27 Dec 2009 09:19:37 DEBUG DistroInstaller location is a network source.
Starting install…
Sun, 27 Dec 2009 09:19:37 DEBUG Attempting to detect distro:
Sun, 27 Dec 2009 09:19:37 DEBUG Fetching URI: http://192.168.1.33/f12/.treeinfo
Sun, 27 Dec 2009 09:19:37 DEBUG Saved file to /var/tmp/virtinst-.treeinfo.A2_hjY
Retrieving file .treeinfo 100% |=========================| 1.2 kB 00:00
Sun, 27 Dec 2009 09:19:37 DEBUG Fetching URI: http://192.168.1.33/f12/images/pxeboot/vmlinuz
Sun, 27 Dec 2009 09:19:37 DEBUG Saved file to /var/tmp/virtinst-vmlinuz.ECKRIB
Retrieving file vmlinuz.. 100% |=========================| 3.3 MB 00:00
Sun, 27 Dec 2009 09:19:37 DEBUG Fetching URI: http://192.168.1.33/f12/images/pxeboot/initrd.img
Sun, 27 Dec 2009 09:19:37 DEBUG Saved file to /var/tmp/virtinst-initrd.img.Yelv8t
Retrieving file initrd.im 100% |=========================| 22 MB 00:00
Sun, 27 Dec 2009 09:19:37 DEBUG Creating guest from:

VF12A
1048576
1048576
94ecb997-16f2-468e-04db-69936ad25469

linux
/var/tmp/virtinst-vmlinuz.ECKRIB
/var/tmp/virtinst-initrd.img.Yelv8t
method=http://192.168.1.33/f12

destroy
destroy
destroy

linux
fedora11

1

Creating domain… 0 B 00:01
Sun, 27 Dec 2009 09:19:38 DEBUG Created guest, looking to see if it is running
Sun, 27 Dec 2009 09:19:38 DEBUG Saving XML boot config:

VF12A
1048576
1048576
94ecb997-16f2-468e-04db-69936ad25469
/usr/lib/xen/bin/pygrub

destroy
restart
restart

linux
fedora11

1

Sun, 27 Dec 2009 09:19:38 DEBUG Removing /var/tmp/virtinst-vmlinuz.ECKRIB
Sun, 27 Dec 2009 09:19:38 DEBUG Removing /var/tmp/virtinst-initrd.img.Yelv8t
Domain installation still in progress. You can reconnect to
the console to complete the installation process.

root@Opensolaris130:~# virsh console VF12A & > /dev/null &
[1] 1223
[2] 1224
[2] Done > /dev/null
root@Opensolaris130:~# vncviewer localhost:0

TigerVNC Viewer for X version 1.0.0
Copyright (C) 2002-2005 RealVNC Ltd.
Copyright (C) 2000-2006 TightVNC Group
Copyright (C) 2004-2009 Peter Astrand for Cendio AB
See http://www.tigervnc.org for information on TigerVNC.

Sun Dec 27 09:20:35 2009
CConn: connected to host localhost port 5900
CConnection: Server supports RFB protocol version 3.8
CConnection: Using RFB protocol version 3.8
TXImage: Using default colormap and visual, TrueColor, depth 24.
CConn: Using pixel format depth 24 (32bpp) little-endian rgb888
CConn: Using Tight encoding


How to build Sun xVM 3.4.2 from source on top OSOL 1002-128a

December 11, 2009

Following bellow is step by step instruction how to clone Sun’s xvm 3.4.2 repository and build packages , required to setup xVM 3.4.2 Dom0 on top of OpenSolaris 1002-128a. View also manuals at opensolaris.org How to Build the Development gates
1. Install SunStudio 12U1 via package manager and create
symlink
# ln -s /opt/sunstudio12.1 /opt/SUNWspro

root@opensolaris:/opt# ls -l
total 8
drwxr-xr-x 31 root bin 50 2009-12-05 17:13 DTT
drwxr-xr-x 8 root bin 8 2009-12-11 09:40 onbld
drwxr-xr-x 13 root sys 13 2009-12-11 08:25 sunstudio12.1
drwxr-xr-x 4 root bin 5 2009-12-05 17:13 SUNWmlib
lrwxrwxrwx 1 root root 13 2009-12-11 08:41 SUNWspro -> sunstudio12.1

2. Install packages

# pkg install SUNWgcc
# pkg install SUNWmercurial
# pkg install SUNWgmake
# pkg install SUNWbcc
# pkg install SUNWlibtool
# pkg install SUNWgnome-common-devel
# pkg install SUNWgnu-automake-110
# pkg install SUNWaconf
# pkg install SUNWxwinc
# pkg install SUNWgit
# pkg install SUNWgnu-readline

3.Install onbld just to avoid failure during xVM build ( no bfu intends) :-

# wget http://dlc.sun.com/osol/on/downloads/b128/SUNWonbld.i386.tar.bz2
# bzcat SUNWonbld.i386.tar.bz2 | tar xf -
# pkgadd -d onbld

Next create /root/.hgrc to activate support mercurial queues :-

[extensions]
hgext.mq=

Set build environment :-

export EMAIL=username@yourdomain.com
export EDITOR=vi
export SOLARIS_BUILD_TOOLS=/opt

4. Clone Sun’s the most recent mercurial gate for xVM (800-900 MB)

# mkdir xvm-3.4
# cd xvm-3.4/
# hg clone ssh://anon@hg.opensolaris.org//hg/xen-gate/xvm-3.4+sunos.hg sunos.hg
# export XVM_WS=`pwd`
# ./sunos.hg/bin/update-gate

Process mercurial queue for each *.hg sub-folder, where it’s not empty

# cd xen.hg
# hg qpush -a

Biuld xVM gate:-

# ./sunos.hg/bin/build-all full &> build.txt &

When done verify logs under log folder:-

root@opensolaris:~/xvm-3.4# ls -l
total 2081
-rw-r--r-- 1 root root 893 2009-12-11 10:00 build.txt
drwxr-xr-x 17 root root 68 2009-12-11 09:56 libvirt.hg
drwxr-xr-x 2 root root 23 2009-12-11 19:57 log
drwxr-xr-x 13 root root 13 2009-12-11 10:00 packages-debug
drwxr-xr-x 13 root root 13 2009-12-11 09:51 packages-nondebug
drwxr-xr-x 5 root root 9 2009-12-11 19:52 proto
drwxr-xr-x 43 root root 377 2009-12-11 09:56 qemu.hg
-rw-r--r-- 1 root root 2037705 2009-12-10 20:51 readline-5.2.tar.gz
drwxr-xr-x 6 root root 9 2009-12-10 09:54 sunos.hg
drwxr-xr-x 4 root root 7 2009-12-11 10:00 urlgrabber.hg
drwxr-xr-x 4 root root 6 2009-12-10 22:10 vbox.hg
drwxr-xr-x 4 root root 6 2009-12-10 20:35 vdisk.hg
drwxr-xr-x 10 root root 27 2009-12-11 10:00 virtinst.hg
drwxr-xr-x 11 root root 22 2009-12-11 09:51 xen.hg

Make sure 11 packages are ready to go :-

root@opensolaris:~/xvm-3.4/packages-nondebug# ls -l
total 17
drwxr-xr-x 4 root root 6 2009-12-11 09:51 SUNWlibvirt
drwxr-xr-x 4 root root 6 2009-12-11 09:51 SUNWlibvirtr
drwxr-xr-x 4 root root 6 2009-12-11 09:51 SUNWurlgrabber
drwxr-xr-x 4 root root 6 2009-12-11 09:51 SUNWvdisk
drwxr-xr-x 4 root root 6 2009-12-11 09:51 SUNWvirtinst
drwxr-xr-x 3 root root 5 2009-12-11 09:51 SUNWxvmdomr
drwxr-xr-x 4 root root 6 2009-12-11 09:51 SUNWxvmdomu
drwxr-xr-x 4 root root 6 2009-12-11 09:51 SUNWxvmh
drwxr-xr-x 4 root root 6 2009-12-11 09:51 SUNWxvmhvm
drwxr-xr-x 4 root root 6 2009-12-11 09:51 SUNWxvmr
drwxr-xr-x 4 root root 6 2009-12-11 09:51 SUNWxvmu

Install packages

# pkgadd -d . SUNWxvmr
# pkgadd -d . SUNWxvmu
# pkgadd -d . SUNWlibvirtr
# pkgadd -d . SUNWlibvirt
# pkgadd -d . SUNWurlgrabber
# pkgadd -d . SUNWvdisk
# pkgadd -d . SUNWvirtinst
# pkgadd -d . SUNWxvmdomr
# pkgadd -d . SUNWxvmdomu
# pkgadd -d . SUNWxvmh
# pkgadd -d . SUNWxvmhvm

Finally :

# svcadm enable milestone/xvm

**************************
Update on 01/02/2010
**************************
By some reasons it might be more safe from the very beginning create separate BE for xvm

# pfexec beadm create xvm_342
# pfexec beadm activate xvm_342

and perform all actions above in new BE, avoiding “svcadm enable milestone/xvm” on osol-128a.
Then

# reboot -p

Check environment

boris@opensolaris:~$ xhost +
access control disabled, clients can connect from any host
boris@opensolaris:~$ pfexec su -
Sun Microsystems Inc. SunOS 5.11 snv_128 November 2008
root@opensolaris:~# xm info
host : opensolaris
release : 5.11
version : snv_128a
machine : i86pc
nr_cpus : 4
nr_nodes : 1
cores_per_socket : 4
threads_per_core : 1
cpu_mhz : 2833
hw_caps : bfebfbff:20100800:00000000:00000140:0408e3fd:00000000:00000001:00000000
virt_caps : hvm
total_memory : 8191
free_memory : 6041
node_to_cpu : node0:0-3
node_to_memory : node0:6041
xen_major : 3
xen_minor : 4
xen_extra : .2-xvm
xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64
xen_scheduler : credit
xen_pagesize : 4096
platform_params : virt_start=0xffff800000000000
xen_changeset : Fri Dec 11 08:54:46 2009 +0300 19667:efeb205e29b8
cc_compiler : gcc version 3.4.3 (csl-sol210-3_4-20050802)
cc_compile_by :
cc_compile_domain :
cc_compile_date : Fri Dec 11 09:42:24 MSK 2009
xend_config_format : 4

root@opensolaris:~# svcs -a |grep xvm
disabled 17:07:11 svc:/system/xvm/ipagent:default
online 17:07:31 svc:/system/xvm/vnc-config:default
online 17:07:37 svc:/system/xvm/store:default
online 17:07:49 svc:/system/xvm/xend:default
online 17:07:49 svc:/system/xvm/console:default
online 17:07:50 svc:/system/xvm/virtd:default
online 17:07:50 svc:/system/xvm/domains:default
online 17:07:52 svc:/milestone/xvm:default

root@opensolaris:~# dladm show-link
LINK CLASS MTU STATE BRIDGE OVER
rge0 phys 1500 up -- --
yge0 phys 1500 down -- --

CS 19740 from xen-3.4-testing.hg appears to be applied to current xVM’s tree :-
xen_changeset : Fri Dec 11 08:54:46 2009 +0300 19667:efeb205e29b8
Build on 11/30/09 was still affected.







**************************
Update on 01/02/2010
**************************
In meantime virt-install hangs pretty often attempting to launch vncviewer to localhost:0
Workarounds :
1. virsh console DomName ( via different terminal session)
2. xVM 3.4.2 host reboot, followed by several virt-install attempts works for me , only when xVM packages are installed on OSOL128a , been built probably on higher OSOL version.
Looks pretty strange, however works always on b129,130



References
1. Sun xVM 3.4.2 available, dom0_min_mem
2.Xen 3.3.2/xVM: OpenSolaris snv_121


Ubuntu Karmic Server PV & HVM DomUs sharing same image at Sun xVM 3.4.2 on top of OSOL 1002-128a

December 7, 2009

Set up Sun xVM 3.4.2 on top of opensolaris 1002-128a via binary packages been built by Pascal Gienger at Index of /software/xvm/3.4.2. Been performing testing bellow my major concern was ability Sun xVM 3.4.2 to handle PV DomUs with boot file systems of ext4fs type. Actually, two guests have been tested : F12 and Ubuntu Karmic Server, in both cases boot partition has been converted to ext3fs to create DomUs manageable via virsh ( virt-manager).

wget http://southbrain.com/software/xvm/3.4.2/SUNWlibvirt-xvm-3.4.2.pkg
wget http://southbrain.com/software/xvm/3.4.2/SUNWlibvirtr-xvm-3.4.2.pkg
wget http://southbrain.com/software/xvm/3.4.2/SUNWurlgrabber-xvm-3.4.2.pkg
........
pkgadd -d SUNWurlgrabber-xvm-3.4.2.pkg
pkgadd -d SUNWlibvirtr-xvm-3.4.2.pkg
pkgadd -d SUNWurlgrabber-xvm-3.4.2.pkg
.......

svcadm enable milestone/xvm

After packages install and enabling xVM, grub entry for xVM should look like :-

title Xen Solaris Development snv_128a X86
findroot (pool_rpool,0,a)
bootfs rpool/ROOT/opensolaris
kernel$ /boot/$ISADIR/xen.gz console=vga dom0_mem=2048M dom0_vcpus_pin=false watchdog=false
module$ /platform/i86xpv/kernel/$ISADIR/unix /platform/i86xpv/kernel/$ISADIR/unix -B $ZFS-BOOTFS
module$ /platform/i86pc/$ISADIR/boot_archive

Install HVM DomU via virt-install :-

virt-install --debug -n KarmicSRV --hvm -r 2048 \
--vcpus=1 -f /dev/zvol/dsk/tvsk2/disk \
--vnc -c /export/home/boris/isos/karmic.iso




Login into HVM :-

# cd /etc/init
# cp tty1.conf hvc0.conf
# vi hvc0.conf
and replace tty1 with hvc0

Shutdown HVM and load PV DomU first time via profile

# cat koala1.py
name="KarmicPV"
memory=2048
bootloader="/usr/lib/xen/bin/pygrub"
kernel="/vmlinuz-2.6.31-14-server"
ramdisk="/boot/initrd.img-2.6.31-14-server"
vif=[' ']
disk=['phy:/dev/zvol/dsk/tvsk2/disk,xvda,w']
extra="root=/dev/xvda2 ro console=hvc0"

xm create -c koala1.py

When DomU gets loaded run tasksel and install Ubuntu Desktop



Shutdown PV DomU and load HVM DomU
Replace grub2 with grub legacy at Karmic HVM DomU:-

# mv /boot/grub /boot/grub_backup
# mkdir /boot/grub
# apt-get --purge remove grub-pc grub-common os-prober
# apt-get install grub
# update-grub
# grub-install /dev/sda

Modify /boot/grub/menu.lst at DomU as follows:-

title Ubuntu 9.10, kernel 2.6.31-14-server (PV)
uuid 8f2cf635-75a0-40d3-9ae5-8d13ce97fe51
kernel /vmlinuz-2.6.31-14-server root=/dev/xvda2 ro quiet splash
initrd /initrd.img-2.6.31-14-server

title Ubuntu 9.10, kernel 2.6.31-14-server (HVM)
uuid 8f2cf635-75a0-40d3-9ae5-8d13ce97fe51
kernel /vmlinuz-2.6.31-14-server root=UUID=8a70da2f-c711-4b86-a785-efdd
0f5fdf8c ro quiet splash
initrd /initrd.img-2.6.31-14-server

Shutdown HVM and load PV DomU via profile koala2.py :-

name="KarmicPV1"
memory=2048
bootloader="/usr/lib/xen/bin/pygrub"
vif=[' ']
disk=['phy:/dev/zvol/dsk/tvsk2/disk,xvda,w']
vfb=['type=vnc,vncunused=1']

# xm create koala2.py
# vncviewer localhost:0

Run virsh dumpxml KarmicPVM > karmic.xml at runtime and shutdown DomU

# virsh define karmic.xml
# virsh start KarmicPV1
# vncviewer localhost :0








References
1. Sun xVM 3.4.2 available, dom0_min_mem


Setup OpenSolaris 1002-127 PV DomU at Xen 3.4.2 & Libvirt 0.7.1-15 (2.6.31.6 pvops) on top of F12 per John Levon

December 2, 2009

Blog entry [1] was written by John Levon and supposed to help community users to install OSOL 2008-11 DomU at Xen 3.1 Dom0 on top of Fedora 8. It’s core technique utilizes virsh , so is supposed to be applicable to current release of Libvirt as well. However , /usr/bin/xenstore-read fails to obtain IP address for OSOL DomU via XenStore at Xen 3.4.2 Dom0. Posting bellow fixes this issue and provide an option to build via virsh the most recent OSOL PV DomU at Xen 3.4.2 Dom0 & Libvirt 0.7.1-15 on Fedora 12.
Create installation profile osol127-install.xml :-

<domain type='xen' id='1'>
<name>OS127</name>
<uuid>cced9a1b-e36d-b93a-45fb-fe28f484503a</uuid>
<memory>2097152</memory>
<currentMemory>2097152</currentMemory>
<vcpu>1</vcpu>
<bootloader>/usr/bin/pygrub</bootloader>
<bootloader_args>--kernel=/platform/i86xpv/kernel/amd64/unix --ramdisk=/boot/amd64/x86.microroot</bootloader_args>
<clock offset='utc'/>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>restart</on_crash>
<devices>
<disk type='file' device='cdrom'>
<driver name='file'/>
<source file='/root/osol127.iso'/>
<target dev='xvdc' bus='xen'/>
<readonly/>
</disk>
<disk type='block' device='disk'>
<driver name='phy'/>
<source dev='/dev/sdb10'/>
<target dev='xvda' bus='xen'/>
</disk>
<interface type='bridge'>
<mac address='00:16:3e:2a:0b:28'/>
<source bridge='eth0'/>
<script path='/etc/xen/scripts/vif-bridge'/>
<target dev='vif1.0'/>
</interface>
<console type='pty' tty='/dev/pts/1'>
<source path='/dev/pts/1'/>
<target port='0'/>
</console>
</devices>
</domain>

For OSOL 1002-128a :-

<bootloader_args>--kernel=/platform/i86xpv/kernel/amd64/unix --ramdisk=/platform/i86pc/amd64/boot_archive </bootloader_args>

Start install

# virsh create osol127-install.xml
# virsh console OS127

 1. Arabic                        12. Hungarian                     
 2. Catalan                       13. Indonesian                    
 3. Chinese - Simplified          14. Italian                       
 4. Chinese - Traditional         15. Japanese                      
 5. Czech                         16. Korean                        
 6. Dutch                         17. Polish                        
 7. English                       18. Portuguese - Brazil           
 8. French                        19. Russian                       
 9. German                        20. Slovak                        
10. Greek                         21. Spanish                       
11. Hebrew                        22. Swedish                       
To select desktop language, enter a number [default is 7]: 
User selected: English
Configuring devices.
Mounting cdroms
Reading ZFS config: done.

opensolaris console login: jack
Password: 
Sun Microsystems Inc.   SunOS 5.11      snv_127 November 2008
jack@opensolaris:~$ Dec  2 07:02:07 opensolaris in.routed[1502]: route 0.0.0.0/8 --> 0.0.0.0 nexthop is not directly connected

jack@opensolaris:~$ 

At this point /usr/bin/xenstore-read fails to obtain IP address for DomU
as suggested in [1] for Fedora 8 ( Xen 3.1 Dom0) and we have to look how to read IP address from XenStore. Domain ID for OSOL in this particular case is equal 1 ( domid=`virsh domid OS127`) :-

# /usr/bin/xenstore-ls /local/domain/1/guest
vnc = ""
passwd = "jDQFrE2t"
port = "5900"
ipv4 = ""
default-link = "xnf0"
0 = ""
address = "192.168.1.41"
[root@fedora12sda ~]# /usr/bin/xenstore-ls /local/domain/1/guest/ipv4
default-link = "xnf0"
0 = ""
address = "192.168.1.41"
[root@fedora12sda ~]# /usr/bin/xenstore-ls /local/domain/1/guest/ipv4/0
address = "192.168.1.41"
# /usr/bin/xenstore-read /local/domain/1/guest/ipv4/0/address
192.168.1.41

Hence, script for VNC connection to OSOL 1002-127 DomU should look like:-

domid=`virsh domid OS127`
ip=`/usr/bin/xenstore-read /local/domain/$domid/guest/ipv4/0/address`
echo $ip
port=`/usr/bin/xenstore-read /local/domain/$domid/guest/vnc/port`
echo $port
/usr/bin/xenstore-read /local/domain/$domid/guest/vnc/passwd
vncviewer $ip:$port






Create runtime profile os127.xml for OSOL 1002-127 as follows :-

<domain type='xen' id='7'>
<name>OS127</name>
<uuid>df7eab17-4d15-2aff-cd62-d44c6093c00b</uuid>
<memory>2097152</memory>
<currentMemory>2097152</currentMemory>
<vcpu>1</vcpu>
<bootloader>/usr/bin/pygrub</bootloader>
<os>
<type>linux</type>
</os>
<clock offset='utc'/>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>restart</on_crash>
<devices>
<disk type='block' device='disk'>
<driver name='phy'/>
<source dev='/dev/sdb10'/>
<target dev='xvda' bus='xen'/>
</disk>
<interface type='bridge'>
<mac address='00:16:3e:10:93:61'/>
<source bridge='eth0'/>
<script path='/etc/xen/scripts/vif-bridge'/>
<target dev='vif7.0'/>
</interface>
<console type='pty' tty='/dev/pts/2'>
<source path='/dev/pts/2'/>
<target port='0'/>
</console>
</devices>
</domain>

# virsh define os127.xml
Domain OS127 defined from os127.xml

# virsh start OS127
Domain OS127 started

# virsh console OS127
Connected to domain OS127
Escape character is ^]
v3.4.2 chgset ‘unavailable’
SunOS Release 5.11 Version snv_127 64-bit
Copyright 1983-2009 Sun Microsystems, Inc. All rights reserved.
Use is subject to license terms.
Hostname: opensolaris
Configuring devices.
Loading smf(5) service descriptions: 162/162
svccfg import warnings. See /var/svc/log/system-manifest-import:default.log .
Reading ZFS config: done.
Mounting ZFS filesystems: (6/6)
Creating new rsa public/private host key pair
Creating new dsa public/private host key pair

opensolaris console login: boris
Password:
Sun Microsystems Inc. SunOS 5.11 snv_127 November 2008
boris@opensolaris:~$ pfexec ifconfig -a
lo0: flags=2001000849 mtu 8232 index 1
inet 127.0.0.1 netmask ff000000
xnf0: flags=1004843 mtu 1500 index 2
inet 192.168.1.43 netmask ffffff00 broadcast 192.168.1.255
ether 0:16:3e:10:93:61
lo0: flags=2002000849 mtu 8252 index 1
inet6 ::1/128
xnf0: flags=2004841 mtu 1500 index 2
inet6 fe80::216:3eff:fe10:9361/10
ether 0:16:3e:10:93:61

Setup VNC via GDM configuration file at OSOL DomU

# vi /etc/X11/gdm/custom.conf
# GDM Custom Configuration file.
# overrides: /usr/share/gdm/defaults.conf
[daemon]
RemoteGreeter=/usr/lib/gdmgreeter
[security]
DisallowTCP=false
# AllowRoot=true
# AllowRemoteRoot=true
[xdmcp]
Enable=true
[gui]
[greeter]
[chooser]
[debug]
[servers]

Restart services :-

# svcadm disable xvnc-inetd gdm
# svcadm enable xvnc-inetd gdm

Been started via virt-manager OSOL 127 DomU may be connected via vncviewer :-

# cat vnc_runtime.sh
domid=`virsh domid OS127`
ip=`/usr/bin/xenstore-read /local/domain/$domid/guest/ipv4/0/address`
echo $ip
vncviewer $ip:0






References
1. OpenSolaris 2008.11 guest domain on a Linux dom0


Follow

Get every new post delivered to your Inbox.