How to provision nested ESXi hosts on free ESXi

Nested ESXi hosts (that means ESXi running inside a VM) are a very cool method to test and evaluate even complex vSphere scenarios on a single physical box. There are a lot of guides already available for installing a nested ESXi host, but most of them assume that you have a paid ESXi license or even vCenter available to manage it.

The free ESXi license though has some limitations that makes the job harder:

• It is not manageable via vCenter, and thus ...
• ... only via the legacy vSphere client, not the Web Client, which in turn means ...
• ... you are limited to virtual hardware version 9 and must not use version 10.
• And - that's most annoying - you also cannot use PowerCLI to create or configure VMs (because the relevant APIs are restricted with the free license)!

You can overcome these limitations by using free and fully featured evaluation licenses, but these are time limited. So, I thought it's time to write a little Nested ESXi Guide for free license users (It also includes some nice tips for paid-license users though!) ...

Before you start going through the guide please make sure that your physical machine resp. its CPU and BIOS configuration really allows running nested ESXi hosts. William Lam has already supplied instructions on how to check that.

For proper networking connectivity you also need to ensure that you have enabled both Promiscuous Mode and Forged Transmits on the virtual port group that your nested hosts will be connected to (on the physical host).


1. Optional: Create a custom ESXi installation ISO

If you install a host using the VMware supplied ESXi installation ISO then you will usually not get the latest ESXi patch level, but have to update the host directly after the installation to get there. And if you want to use the new VMware Tools for Nested ESXi package then you need to run a second step to get this installed.

You can save both these steps by creating a customized installation ISO with my ESXi-Customizer-PS script by following these steps:

• Create an Offline Bundle from the esx-tools VIB following this guide and copy the zip file to a newly created empty directory (e.g. C:\TEMP\zip)
• Download ESXi-Customizer-PS and run it in a PowerCLI session like this

ESXi-Customizer-PS-v2.0.ps1 -obDir C:\TEMP\zip -outDir C:\TEMP -nsc

This will create an ESXi 5.5 installation ISO with the current patch level and the VMware Tools for Nested ESXi included. You will find it in the -outDir directory (C:\TEMP in this example).

Please note: This is an optional step. You can as well install nested ESXi machines with the vanilla VMware ISO, but if you plan to do this very often it will save a lot of time to use an ISO that already includes the latest ESXi patch level and VIB packages that you want to add.

2. Create the nested ESXi VM

Use the vSphere Client to connect to the physical ESXi host that you will use to run the nested host. Launch the "New VM" wizard and fill in the forms like this:

• Choose Custom Configuration at the beginning, type a name for the machine (e.g. vESXi) and select a datastore for it.
• Select Virtual Machine Version: 8
• For the Guest Operating System choose Other, in the Version dropdown select Other (64-bit), then enter VMware ESXi 5.x in the input field:

• For the CPUs select a configuration that results in at least 2 virtual cores (this can be either 1 socket and 2 cores per socket or 2 sockets and 1 core per socket).
• Memory: ESXi 5.5 requires a minimum of 4 GB. With ESXi 5.0 and 5.1 the minimum was 2 GB.
• Network: ESXi will work fine with just 1 NIC, but there are certain scenarios where you get warnings about missing redundancy. So, I usually use 2 NICs. Depending on the test scenarios that you are targeting you might also use more than 2.
• Pick the default SCSI Controller LSI Logic Parallel.
• Choose to create a new virtual disk with a size of at least 910 MB. This is the minimum size that ESXi requires. If you want to have a local persistent scratch partition on the same disk then you need to configure a size of at least 5.5 GB. Even bigger sizes will result in a VMFS datastore being automatically created on the remainder of the disk.
  I usually use the minimum size, don't care about persistent scratch for test hosts and use separate disks for local datastores, but it doesn't really matter if you choose the disk to be Thin Provisioned.

After the VM has been created edit its General Options and change the Other (64-bit) to VMware ESXi 5.x in the Guest Operating System version dropdown. This is not possible in the New VM wizard, but now after the VM has been created (because running ESXi in ESXi is officially unsupported).

I usually do two other config changes will all my VMs, but this is optional and not required to run ESXi in the VM:

• remove the Floppy drive from the virtual hardware (anyone still using floppies?)
• in Advanced Options / Boot Options raise the Power On Boot Delay to 5.000 ms (or higher). After powering on the VM and opening its console this will give you some time to press ESC for the boot menu or F2 for the BIOS setup before the installed OS starts booting.

3. Upgrade virtual hardware and enable VHV

We have created the VM with hardware version 8, because this is the maximum version that you can select in the New VM wizard. However, if your physical host runs ESXi 5.1 or 5.5 then you should definitely upgrade the VM to version 9, because there were substantial improvements for running nested ESXi in this version.

Only if the physical host that you will run your nested hosts on is running ESXi 5.1 then you can just use the vSphere Client to upgrade the virtual hardware of the VM to version 9 (just select Upgrade Virtual Hardware from its context menu).

However, if your physical host runs ESXi 5.5 then upgrading the VM this way will result in hardware version 10, and you will no longer be able to edit the VM's configuration using the vSphere Client! In this case to upgrade to version 9 only we need to open an ESXi shell (see this KB article if you need instructions for doing this) and run the following commands

  vim-cmd vmsvc/getallvms

This will list all VMs that are registered on the host. Find the nested ESXi VM that you just created and note its vmid. Then run

  vim-cmd vmsvc/upgrade vmid vmx-09

This will upgrade the VM with the id vmid to hardware version 9.

The second thing to do is enabling VHV (Virtual Hardware-Assisted Virtualization). Since ESXi 5.1 this must be done on a per-VM basis by adding a new configuration option to a VM's vmx file (see this post by William Lam for details). You cannot do this through the vSphere Client, so stay at the command line and ...

• change to the directory of the VM with something like
     cd /vmfs/volumes/datastore/vESXi
  (Replace datastore with the name of the datastore that you created the VM on and vESXi with the name that you chose for the VM)
• Append the line vhv.enable = "TRUE" to the VM's vmx file by running
    echo vhv.enable = \"TRUE\">> vESXi.vmx
  (Replace vESXi with the real name of the VM again. If you are comfortable with vi - the mother of all Unix editors - then you can of course also just edit the vmx-file with that ;-)
• Then reload the changed configuration. You need the vmid for this again (see above):
    vim-cmd vmsvc/reload vmid

4. Install (and update) ESXi

You install ESXi on the nested host just like you would do this on a physical machine. Connect the installation ISO to the VM, power it on and walk through the installation screens. A video guide of an interactive installation is available in the VMware KB article Methods of installing ESXi.

If you have DHCP available on your network then I recommend keeping the host's IP configuration at DHCP. Otherwise you need to configure valid static IP settings for the VMkernel interface.

If you install the nested host with the custom ISO created in step 1 then you are now done with this step and you can proceed to step 5. If not then perform these additional steps:

• Log in to a shell on your nested host (not on your physical one!)
• Open its firewall for outgoing http-requests:

  esxcli network firewall ruleset set -e true -r httpClient

• Find out what ESXi 5.5 image profiles (= patch levels) are available in the VMware Online Depot:

  esxcli software sources profile list -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml | grep ESXi-5.5

  (Please note: This is one command-line!)
  You can also grep for ESXi-5.0 or ESXi-5.1 if you do not want to install ESXi 5.5.
• At the time of this post the most recent one is ESXi-5.5.0-20131204001-standard (of Dec 2013). So you can now update to this release by running

  esxcli software profile install -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml -p ESXi-5.5.0-20131204001-standard

  (Please note: Again this is one command line!)
• Reboot and log in again to the nested host. Now install the VMware Tools for Nested ESXi by running

  esxcli software vib install -v http://download3.vmware.com/software/vmw-tools/esxi_tools_for_guests/esx-tools-for-esxi-9.7.0-0.0.00000.i386.vib -f

  (Please note: This is also one command line!)
• Reboot again to complete the installation.

5. Configure the nested host

Now use the vSphere Client to configure your nested host to suit your needs. Here is a non-exclusive list of things to consider:

• Hostname and DNS settings (you may skip this if you can use DHCP and DNS reverse lookups, especially if you want to clone this host later. See comment in 7.)
• Virtual switch(es): If your nested host has two NICs then you will want to add the second one to the virtual switch that was automatically created during installation.
• NTP servers: If you do not have any on your own network then use publicly available ones like these from pool.ntp.org.
• Datastores: add local disks, NFS datastores, iSCSI targets via the iSCSI software adapter
• Optional: Syslogging to a datastore path or syslog server and/or persistent scratch

6. Prepare for cloning and export to OVA

Now you have provisioned your first nested ESXi host on a physical free-license ESXi host. We could stop here, but I want to take this guide a little further ... What about re-using what you have done for future nested ESXi deployments? How about cloning this host?

If you had vCenter available then you could easily convert or clone the VM to a template and use that template to provision as many nested ESXi hosts as you need. With a standalone host you have another option to achieve basically the same: Export the VM to an OVA file. You can then create new nested ESXi hosts by importing this OVA file again and again.

No matter what method you choose you should properly prepare the nested host before you clone it! It's William Lam again who wrote an excellent article about how to do this. Here is a short summary:

• Log in to a shell on your nested host
• Run the following command:
  

  esxcli system settings advanced set -o /Net/FollowHardwareMac -i 1

  (Guess what? This is one command line again!)
  This will let ESXi change the VMkernel interface's MAC address whenever its first NIC's MAC address changes. If you do not do this then you will end up in all cloned hosts using the same MAC address (and that would be really bad!).
• Now remove the line starting with /system/uuid from the configuration file /etc/vmware/esx.conf:

  sed -i "/\/system\/uuid/d" /etc/vmware/esx.conf

  (You could of course also just use vi instead to edit the file and remove the line).
• To make the last change persistent run

  /sbin/auto-backup.sh

Then shut down the nested host (on the command line you can just use poweroff for this).

From the vSphere Client (connected to your physical host) you can now export your nested ESXi host to an OVA file.


7. Provision new nested hosts from the OVA

By importing the OVA file you can now create as many new nested ESXi hosts as you want. They will all inherit the configuration of the template host, but you can of course reconfigure each to meet your specific testing needs. The only things that must be configured per host is the hostname and - if you are not using DHCP - the IP address configuration.

But even manually setting a hostname is not necessary if you are not only using DHCP but also have access to your own DNS server. Then you can just create a reverse DNS lookup entry for a host's IP address and point it to the desired host name. The DHCP client in ESXi will look up the reverse entry and - if successful - will configure its hostname using the result of the lookup.

---

This post first appeared on the VMware Front Experience Blog and was written by Andreas Peetz. Follow him on Twitter to keep up to date with what he posts.

---

---