VMX: Remove VMware tools from Windows guests

For Windows guests, you should remove VMware tools before conversion. Although this is not strictly necessary, and the guest will still be able to run, if you don't do this then the converted guest will complain on every boot. The tools cannot be removed after conversion because the uninstaller checks if it is running on VMware and refuses to start (which is also the reason that virt-v2v cannot remove them).

This is not necessary for Linux guests, as virt-v2v is able to remove VMware tools.

VMX: Guest must be shut down

The guest must be shut down before conversion starts. If you don't shut it down, you will end up with a corrupted VM disk on the target. With other methods, virt-v2v tries to prevent concurrent access, but because the -i vmx method works directly against the storage, checking for concurrent access is not possible.

VMX: Access to the storage containing the VMX and VMDK files

If the vmx and vmdk files aren't available locally then you must either mount the NFS storage on the conversion server or enable passwordless SSH on the ESXi hypervisor.

VMX: Passwordless SSH using ssh-agent

You must also use ssh-agent, and add your ssh public key to /etc/ssh/keys-root/authorized_keys (on the ESXi hypervisor).

After doing this, you should check that passwordless access works from the virt-v2v server to the ESXi hypervisor. For example:

 $ ssh root@esxi.example.com
 [ logs straight into the shell, no password is requested ]

Note that password-interactive and Kerberos access are not supported. You have to set up ssh access using ssh-agent and authorized_keys.

VMX: Construct the SSH URI

When using the SSH input transport you must specify a remote "ssh://..." URI pointing to the VMX file. A typical URI looks like:

 ssh://root@esxi.example.com/vmfs/volumes/datastore1/my%20guest/my%20guest.vmx

Any space must be escaped with %20 and other non-ASCII characters may also need to be URI-escaped.

The username is not required if it is the same as your local username.

You may optionally supply a port number after the hostname if the SSH server is not listening on the default port (22).

VMX: Importing a guest

To import a vmx file from a local file or NFS, do:

 $ virt-v2v -i vmx guest.vmx -o local -os /var/tmp

To import a vmx file over SSH, add -it ssh to select the SSH transport and supply a remote SSH URI:

 $ virt-v2v \
     -i vmx -it ssh \
     "ssh://root@esxi.example.com/vmfs/volumes/datastore1/guest/guest.vmx" \
     -o local -os /var/tmp

Virt-v2v processes the vmx file and uses it to find the location of any vmdk disks.

VDDK: Prerequisites

1.

As the VDDK library is not open source, and the license of this library does not permit redistribution or commercial use, you must obtain VDDK yourself and satisfy yourself that your usage of the library is permitted by the license.

2.

You must also compile nbdkit, enabling the VDDK plugin. nbdkit ≥ 1.1.25 is recommended, but it is usually best to compile from the git tree.

 export PATH=/path/to/nbdkit-1.1.x:$PATH

3.

You must find the SSL "thumbprint" of your VMware server. How to do this is explained in nbdkit-vddk-plugin(1), also available at the link above.

4.

VDDK imports require a feature added in libvirt ≥ 3.7.

VDDK: ESXi NFC service memory limits

In the verbose log you may see errors like:

 nbdkit: vddk[3]: error: [NFC ERROR] NfcFssrvrProcessErrorMsg:
 received NFC error 5 from server: Failed to allocate the
 requested 2097176 bytes

This seems especially common when there are multiple parallel connections open to the VMware server.

These can be caused by resource limits set on the VMware server. You can increase the limit for the NFC service by editing /etc/vmware/hostd/config.xml and adjusting the "<maxMemory>" setting:

 <nfcsvc>
   <path>libnfcsvc.so</path>
   <enabled>true</enabled>
   <maxMemory>50331648</maxMemory>
   <maxStreamMemory>10485760</maxStreamMemory>
 </nfcsvc>

and restarting the "hostd" service:

 # /etc/init.d/hostd restart

For more information see https://bugzilla.redhat.com/1614276.

VDDK: URI

Construct the correct "vpx://" (for vCenter) or "esx://" (for ESXi) URL. It will look something like these:

 vpx://root@vcenter.example.com/Datacenter/esxi
 esx://root@esxi.example.com

To verify that you have the correct URL, use the virsh(1) command to list the guests on the server:

 $ virsh -c 'vpx://root@vcenter.example.com/Datacenter/esxi' list --all
 Enter root's password for vcenter.example.com: ***
 
  Id    Name                           State
 ----------------------------------------------------
  -     Fedora 20                      shut off
  -     Windows 2003                   shut off

If you get an error "Peer certificate cannot be authenticated with given CA certificates" or similar, then you can either import the vCenter host’s certificate, or bypass signature verification by adding the "?no_verify=1" flag:

 $ virsh -c 'vpx://root@vcenter.example.com/Datacenter/esxi?no_verify=1' list --all

You should also try dumping the metadata from any guest on your server, like this:

 $ virsh -c 'vpx://root@vcenter.example.com/Datacenter/esxi' dumpxml "Windows 2003"
 <domain type='vmware'>
   <name>Windows 2003</name>
   [...]
   <vmware:moref>vm-123</vmware:moref>
 </domain>

If "<vmware:moref>" does not appear in the metadata, then you need to upgrade libvirt.

If the above commands do not work, then virt-v2v is not going to work either. Fix your URI and/or your VMware server before continuing.

VDDK: Importing a guest

The -it vddk parameter selects VDDK as the input transport for disks.

To import a particular guest from vCenter server or ESXi hypervisor, use a command like the following, substituting the URI, guest name and SSL thumbprint:

 $ export PATH=/path/to/nbdkit-1.1.x:$PATH
 $ virt-v2v \
     -ic 'vpx://root@vcenter.example.com/Datacenter/esxi?no_verify=1' \
     -it vddk \
     -io vddk-libdir=/path/to/vmware-vix-disklib-distrib \
     -io vddk-thumbprint=xx:xx:xx:... \
     "Windows 2003" \
     -o local -os /var/tmp

Other options that you might need to add in rare circumstances include -io vddk-config, -io vddk-cookie, -io vddk-nfchostport, -io vddk-port, -io vddk-snapshot, and -io vddk-transports, which are all explained in the nbdkit-vddk-plugin(1) documentation. Do not use these options unless you know what you are doing.

VDDK: Debugging VDDK failures

The VDDK library can be operated in a verbose mode where it gives (very) verbose messages. Use ‘virt-v2v -v -x’ as usual to enable verbose messages.

OVA: Remove VMware tools from Windows guests

For Windows guests, you should remove VMware tools before conversion. Although this is not strictly necessary, and the guest will still be able to run, if you don't do this then the converted guest will complain on every boot. The tools cannot be removed after conversion because the uninstaller checks if it is running on VMware and refuses to start (which is also the reason that virt-v2v cannot remove them).

This is not necessary for Linux guests, as virt-v2v is able to remove VMware tools.

OVA: Create OVA

To create an OVA in vSphere, use the "Export OVF Template" option (from the VM context menu, or from the File menu). Either "Folder of files" (OVF) or "Single file" (OVA) will work, but OVA is probably easier to deal with. OVA files are really just uncompressed tar files, so you can use commands like "tar tf VM.ova" to view their contents.

Create OVA with ovftool

You can also use VMware’s proprietary "ovftool":

 ovftool --noSSLVerify \
   vi://USER:PASSWORD@esxi.example.com/VM \
   VM.ova

To connect to vCenter:

 ovftool  --noSSLVerify \
   vi://USER:PASSWORD@vcenter.example.com/DATACENTER-NAME/vm/VM \
   VM.ova

For Active Directory-aware authentication, you have to express the "@" character in the form of its ascii hex-code (%5c):

 vi://DOMAIN%5cUSER:PASSWORD@...

OVA: Importing a guest

To import an OVA file called VM.ova, do:

 $ virt-v2v -i ova VM.ova -o local -os /var/tmp

If you exported the guest as a "Folder of files", or if you unpacked the OVA tarball yourself, then you can point virt-v2v at the directory containing the files:

 $ virt-v2v -i ova /path/to/files -o local -os /var/tmp

vCenter: Remove VMware tools from Windows guests

For Windows guests, you should remove VMware tools before conversion. Although this is not strictly necessary, and the guest will still be able to run, if you don't do this then the converted guest will complain on every boot. The tools cannot be removed after conversion because the uninstaller checks if it is running on VMware and refuses to start (which is also the reason that virt-v2v cannot remove them).

This is not necessary for Linux guests, as virt-v2v is able to remove VMware tools.

vCenter: URI

The libvirt URI of a vCenter server looks something like this:

 vpx://user@server/Datacenter/esxi

where:

"user@"

is the (optional, but recommended) user to connect as.

If the username contains a backslash (eg. "DOMAIN\USER") then you will need to URI-escape that character using %5c: "DOMAIN%5cUSER" (5c is the hexadecimal ASCII code for backslash.) Other punctuation may also have to be escaped.

"server"

is the vCenter Server (not hypervisor).

"Datacenter"

is the name of the datacenter.

If the name contains a space, replace it with the URI-escape code %20.

"esxi"

is the name of the ESXi hypervisor running the guest.

If the VMware deployment is using folders, then these may need to be added to the URI, eg:

 vpx://user@server/Folder/Datacenter/esxi

For full details of libvirt URIs, see: http://libvirt.org/drvesx.html

Typical errors from libvirt / virsh when the URI is wrong include:

• Could not find datacenter specified in [...]
• Could not find compute resource specified in [...]
• Path [...] does not specify a compute resource
• Path [...] does not specify a host system
• Could not find host system specified in [...]

vCenter: Test libvirt connection to vCenter

Use the virsh(1) command to list the guests on the vCenter Server like this:

 $ virsh -c 'vpx://root@vcenter.example.com/Datacenter/esxi' list --all
 Enter root's password for vcenter.example.com: ***
 
  Id    Name                           State
 ----------------------------------------------------
  -     Fedora 20                      shut off
  -     Windows 2003                   shut off

If you get an error "Peer certificate cannot be authenticated with given CA certificates" or similar, then you can either import the vCenter host’s certificate, or bypass signature verification by adding the "?no_verify=1" flag:

 $ virsh -c 'vpx://root@vcenter.example.com/Datacenter/esxi?no_verify=1' list --all

You should also try dumping the metadata from any guest on your server, like this:

 $ virsh -c 'vpx://root@vcenter.example.com/Datacenter/esxi' dumpxml "Windows 2003"
 <domain type='vmware'>
   <name>Windows 2003</name>
   [...]
 </domain>

If the above commands do not work, then virt-v2v is not going to work either. Fix your libvirt configuration and/or your VMware vCenter Server before continuing.

vCenter: Importing a guest

To import a particular guest from vCenter Server, do:

 $ virt-v2v -ic 'vpx://root@vcenter.example.com/Datacenter/esxi?no_verify=1' \
   "Windows 2003" \
   -o local -os /var/tmp

where "Windows 2003" is the name of the guest (which must be shut down).

Note that you may be asked for the vCenter password twice. This happens once because libvirt needs it, and a second time because virt-v2v itself connects directly to the server. Use -ip filename to supply a password via a file.

In this case the output flags are set to write the converted guest to a temporary directory as this is just an example, but you can also write to libvirt or any other supported target.

vCenter: Non-administrator role

Instead of using the vCenter Administrator role, you can create a custom non-administrator role to perform the conversion. You will however need to give it a minimum set of permissions as follows (using VMware vCenter 6.5):

1.

Create a custom role in vCenter.

2.

Enable (check) the following objects:

 Datastore:
  - Browse datastore
  - Low level file operations
 
 Sessions:
  - Validate session
 
 Virtual Machine:
   Interaction:
     - Guest operating system management by VIX API
   Provisioning:
     - Allow disk access
     - Allow read-only disk access
    

vCenter: Firewall and proxy settings

vCenter: Ports

If there is a firewall between the virt-v2v conversion server and the vCenter server, then you will need to open port 443 (https) and port 5480.

Port 443 is used to copy the guest disk image(s). Port 5480 is used to query vCenter for guest metadata.

These port numbers are only the defaults. It is possible to reconfigure vCenter to use other port numbers. In that case you would need to specify those ports in the "vpx://" URI. See "vCenter: URI" above.

These ports only apply to virt-v2v conversions. You may have to open other ports for other vCenter functionality, for example the web user interface. VMware documents the required ports for vCenter in their online documentation.

 ┌────────────┐   port 443 ┌────────────┐        ┌────────────┐
 │ virt-v2v   │────────────▶ vCenter    │────────▶ ESXi       │
 │ conversion │────────────▶ server     │        │ hypervisor │
 │ server     │  port 5480 │            │        │   ┌─────┐  │
 └────────────┘            └────────────┘        │   │guest│  │
                                                 └───┴─────┴──┘

(In the diagram above the arrows show the direction in which the TCP connection is initiated, not necessarily the direction of data transfer.)

Virt-v2v itself does not connect directly to the ESXi hypervisor containing the guest. However vCenter connects to the hypervisor and forwards the information, so if you have a firewall between vCenter and its hypervisors you may need to open additional ports (consult VMware documentation).

The proxy environment variables ("https_proxy", "all_proxy", "no_proxy", "HTTPS_PROXY", "ALL_PROXY" and "NO_PROXY") are ignored when doing vCenter conversions.

vCenter: SSL/TLS certificate problems

You may see this error:

  CURL: Error opening file: SSL: no alternative certificate subject
  name matches target host name

(You may need to enable debugging with ‘virt-v2v -v -x’ to see this message).

This can be caused by using an IP address instead of the fully-qualified DNS domain name of the vCenter server, ie. use "vpx://vcenter.example.com/..." instead of "vpx://11.22.33.44/..."

Another certificate problem can be caused by the vCenter server having a mismatching FQDN and IP address, for example if the server acquired a new IP address from DHCP. To fix this you need to change your DHCP server or network configuration so that the vCenter server always gets a stable IP address. After that log in to the vCenter server’s admin console at "https://vcenter:5480/". Under the "Admin" tab, select "Certificate regeneration enabled" and then reboot it.