Converting Linux Xen guests to VMWare

At work, I have been tasked with migrating our existing Xen Linux guests (RH on RH) to a new VMWare environment.

Since I have yet to find a very comprehensive procedure, I thought I'd write one.

This is a work in progress and as I fine tune the procedure I will update, but for those needing to do it, this should work for you:

How to Convert a Xen Linux Guest to VMWare

These are the high level to-do’s that are ahead of you. Please do not use this part of the guide to do the conversion, it is only a synapses of what you are in for.

Step 1: prep the source Xen guest for remote root/password access
Step 2: run the VMWare conversion process (confirm “new” disks are up to new standards)
Step 3: change nic to vmxnet3 in new VMWare guest settings
Step 4: boot the newly created VMWare guest off a Linux live CD; configure network
Step 5: mount converted file systems; chroot into this environment
Step 6: edit inittab; modprobe.conf
Step 7: install non-xen kernel version insuring the SAME kernel version
Step 8: edit grub.conf
Step 9: install vmware tools
Step 10: fix remote root/password access; reboot
Step 11: test test test. re-test.

Tools you need or need access to:
VMWare virtual center with permission to create new guests.
VMWare converter.
Access to Xen source as root.
Access to Linux live CD media (typically already on a data store).
Fairly good knowledge of Linux command line tools and VMWare.

Step by step:

Step 1:
For the conversion process to work, the VMWare converter needs remote root/password access to the Xen guest you are converting.

Change the root password to something new and unique. DO NOT use the default root password during this process.

Note the current xen kernel version for use later.

Step 2:
Access VMWare Converter via client or direct log on to converter server.

Select live OS

Enter hostname, root and password. Feel free to review source machine to insure proper access.

Enter VC credentials.

Select proper cluster.

Select proper storage.

Make sure to convert only those files systems that are required (no NAS mounts for instance).

Make sure NEW file systems adhere to any new or updated policy regarding size. Adjust accordingly.

Select the option to configure the network using a unique IP (unless your server network uses DHCP) and NOT the source machines IP.

IF THIS IS A REAL CONVERSION AND NOT JUST A TEST: Make sure to select the option to power off the source Xen machine (leave the new machine off as well).

Run the conversion process.

Connect to VC, open the console of the new VMWare machine that the conversion process is creating.

NOTE: for some reason (a bug?), the NEW “helper” VM sometimes cannot get on the network on its own. You must log into it as root using the temp password you created on the source Xen machine, and ping something off the network. This will kick start the networking and you should now see the converter doing its thing.

NOTE: In order to access the new "helper" VM that the converter creates, you must first configure the converter itself to allow such a thing, BEFORE you begin this process. I'll link to those instructions when I find them again. This change will allow you to log into the helper VM using the source Xen credentials.

Enjoy an espresso or two while you wait.

Step 3:

Once done, both the source Xen machine and the new VMWare machine should be powered off automatically.

IF THIS IS A REAL CONVERSION: Disable the source Xen network configuration to insure it doesn’t come back on the network. This process documented elsewhere.

Re-configure the new VMWare machine to use the vmxnet3 nic driver (remove existing e1000 and add a new vmxnet3 nic), and also point the CDROM to a Linux live CD of your choice (I’ll use Fedora in this document).

Also change the boot option delay to at least 10ms (or higher) to make your life easier.

NOTE: The new VMWare guest WILL NOT boot up in its existing condition. This is a known problem we are about to address.

Step 4:

Boot the new VMWare machine via the Linux live CD.

Configure the live CD network with any available IP (the source Xen IP is fine to use here if you are on the same network obviously).

Step 5:

From the Linux live cd, mount the guest file systems (the specifics will depend on what your source Xen machine looked like):
open a terminal:
$ su - (no password is needed on the Fedora live cd)
# mkdir /mnt/server
# mount /dev/mapper/vgroot-lvroot /mnt/server
# mount /dev/sda1 /mnt/server/boot
# mount /dev/mapper/vgroot-lvusr /mnt/server/usr
# mount /dev/mapper/vgroot-lvopt /mnt/server/opt
# mount /dev/mapper/vgroot-lvvar /mnt/server/var
# mount --bind /dev /mnt/server/dev/

chroot into this environment:
#chroot /mnt/server

mount proc:
#mount /proc /proc -t proc

Step 6:

Edit /etc/inittab and comment out the Xen console line:
(At my office, the first getty line is co:blahblahblah, which is the Xen console. You don't need this so comment it out)

Edit /etc/modprobe.conf to include new scsi driver and vmxnet3 driver:
This is key. Xen uses a different scsi controller than vmware.
Comment out the xenblk line and make sure you have:
alias eth0 vmxnet3 ---- the recommended vmware nic driver
alias scsi_hostadapter mptbase ----- the recommended vmware scsi adapter
alias scsi_hostadapter1 mptspi ----- the recommended vmware scsi adapter

Step 7:

Install non-Xen version of the SAME kernel as the source Xen machine (you can look at /boot and /boot/grub/grub.conf for which kernel if you didn’t note this in Step 1 as instructed):

# yum install kernel-XXXXXXXXXXXX


Step 8:

Edit grub to make this new kernel the default:
default=(what ever line your new non-Xen kernel is

Step 9:

From the VC client, choice to “Install VMWare tools” for this machine.

Mount the CDRom (again, this is from within the chroot'd environment):
# mkdir /media/cdrom
# mount /dev/cdrom /media/cdrom

Explode the vmware-tools installation files:
# tar -zxvf /media/cdrom/YYYYYYYY -C /var/tmp/

# cd /var/tmp/YYYYYYYYYY

run the installer:
# ./vmware-install

Pick all default answers.

You may have to pick a screen size IF x-server is installed on this guest already.

Pick 1024x768 unless you know a reason not to.

Step 10:

fix the sshd config file (if you normally dis-allow remote root access) and change the root password back to the current one.

reboot.

Step 11:

Test access; test for proper kernel version; test; test; test; test.