Building a self-configuring nested ESXi host vApp

In my last post I presented a walk-through about how to create a nested ESXi host and make an OVF template of it. After deploying this template a manual step remained: Configuring a hostname and the IP address configuration. If you deploy a virtual appliance that was produced by VMware (e.g. the vCenter Log Insight appliance) then you are often presented with the choice to configure the networking of the VM in the OVF deployment wizard. How is this done, and can we use the same method to customize the nested ESXi vApp? Yes, we can - and here is how.

Please note: vApps are only available with vCenter, not on standalone hosts. If you do not have vCenter availabe then you can register for a 60-days evaluation copy.

1. Enable and edit OVF properties

The following configurations need to be done on the nested host that you will use to create the OVF template from. First we need to turn it into a vApp by enabling the vApp Options in the Options tab of its properties:

In OVF Settings select VMware Tools as the OVF Environment Transport:

In the Advanced section of the vApp Options you can fill in some informational properties that will be displayed in the General section in the VM's Summary tab (like shown in the first screenshot of this post):

The Product Name string will be displayed as a hyperlink to the Product URL, and the Vendor string as a hyperlink to the Vendor URL.

The Application URL is special: After the VM is powered on vCenter will periodically check whether it is accessible and - if yes - the vApp State will be displayed as Available in its Summary tab. I use the OVF property hostname as a variable in this URL. We add that by clicking on the Properties button in the view above:

The Category, Label and Description of a property have just a descriptive purpose and will appear in the Properties section of the vApp Options (where you can also change the values of the properties).

The ID will be the name of the property variable. To build some kind of object hierarchy you can optionally specify a Class ID and Instance ID - the resulting name of the variable will then be of the form Class_ID.ID.Instance_ID.

For the property Type you have various options including String, Integer, Real, Boolean, Password, a selection from a string list and an IP address.

For automatically assigning an IP address to a vApp you have two options: Using the vCenter managed IP Pools (see the IP Pools in the view of the parent Datacenter object) or/and a DHCP server on your network. I choose DHCP for my example, and this is what I configure as the IP Allocation scheme in the Advanced vApp Options and in the IP Allocation Policy. For the latter I also need to choose the IP protocol to use: either IPv4 or IPv6, strangely I cannot choose both!?

2. Writing a custom boot script

The OVF properties that we have defined (only hostname in our example) will be passed to the Guest OS in XML encoded format. Once the VM is powered on you can look at the XML code by clicking on the View... button in the VM's OVF Settings dialog:

There you can find the property hostname that we defined and the assigned value (vESXi34 in this example). But how is it passed to the Guest OS?

Since we have chosen VMware Tools as the OVF Environment Transport and the "VMware Tools for Nested ESXi" are installed in the virtual ESXi host we can now output the XML data displayed above by running

   /sbin/vmtoolsd --cmd='info-get guestinfo.ovfEnv'

in the ESXi guest. We now just need a script that extracts the hostname from this XML data and applies it to the host during the first boot after the OVF deployment.

The following script does this job and a bit more. It is tailored towards my own lab environment, but will be useful - at least in parts - for a lot of cases. Save it as

   /etc/rc.local.d/local.sh

on the nested host that you will use to create the OVF template from. This script already exists in ESXi, but does nothing by default. Changes to it will be saved in the configuration state backup of ESXi and persist a reboot.

#!/bin/sh

# local configuration options

# Note: modify at your own risk!  If you do/use anything in this
# script that is not part of a stable API (relying on files to be in
# specific places, specific tools, specific output, etc) there is a
# possibility you will end up with a broken system after patching or
# upgrading.  Changes are not supported unless under direction of
# VMware support.

getprops_from_ovfxml() {
/bin/python - <<EOS
from xml.dom.minidom import parseString
ovfEnv = open("$1", "r").read()
dom = parseString(ovfEnv)
section = dom.getElementsByTagName("PropertySection")[0]
for property in section.getElementsByTagName("Property"):
   key = property.getAttribute("oe:key").replace('.','_')
   value = property.getAttribute("oe:value")
   print "{0}={1}".format(key,value)
dom.unlink()
EOS
}
#
log() {
   echo "$*"
   logger init "$*"
   esxcfg-init --set-boot-status-text "$*"
}
#
[ -x /sbin/vmtoolsd ] || {
   log "ERROR: VMware Tools for Nested ESXi are not installed. Exiting ..."
   exit 1
}
/sbin/vmtoolsd --cmd='info-get guestinfo.ovfEnv' >/tmp/ovf.xml 2>/dev/null
[ -s /tmp/ovf.xml ] || {
   log "ERROR: Cannot get OVF parameters through VMware Tools. Exiting ..."
   exit 1
}
o=`hostname`
[ "${o%%.*}" == "$hostname" ] && {
   log "Hostname is unchanged. Skipping OVF customizations ..."
   exit 0
}
grep "fqdn.fqdn" /etc/dhclient-vmk0.conf >/dev/null && {
   log "OVF customization already done. Exiting ..."
   exit 0
}
#
log "Applying OVF customizations ..."
#
eval `getprops_from_ovfxml /tmp/ovf.xml`
rm -f /tmp/ovf.xml
#
esxcli system hostname set -H $hostname
fqdn=`hostname -f`
echo "send fqdn.fqdn \"$fqdn\";" >>/etc/dhclient-vmk0.conf
echo "send fqdn.encoded on;" >>/etc/dhclient-vmk0.conf
echo "send fqdn.fqdn \"$fqdn\";" >>/etc/dhclient6-vmk0.conf
echo "send fqdn.encoded on;" >>/etc/dhclient6-vmk0.conf
#
# esxcfg-init --set-boot-progress-text "OVF Settings applied. Rebooting once now ..."
# /sbin/reboot
esxcfg-init --set-boot-progress-text "OVF Settings applied. Restarting Management Network now ..."
esxcli network ip interface set -e false -i vmk0
esxcli network ip interface set -e true -i vmk0
#
exit 0

Comments:

Lines 12-24: Here I use an inline Python script to implement the shell function getprops_from_ovfxml(). It accepts the name of an XML file as parameter and outputs the OVF properties' key/value pairs that are stored in this file in a way that can be directly evaluated by the shell (in our example it's just the line "hostname=vESXi34"). The Python script was heavily inspired by a code example that was given in a VMware Blog post about the OVF Self-configuration topic.
This is surely the most generic and reusable part of my script, because it handles any OVF properties that you might have defined. Please note that periods (.) in the variable name will be replaced by underscores (_) in the case you used Class or Instance IDs in the OVF properties.

Line 26-30: The log() function outputs a string to stdout, to syslog and also updates the status text on the boot screen via an esxcfg-init command. It will look like this then:

Nice, eh?

Line 32-35: Sanity check #1: First we check if the VMware Tools are installed and exit otherwise.

Line 36: Now run the VMware Tools command to get the XML data and save it to the file /tmp/ovf.xml.

Line 27-40: Sanity check #2: Check if the produced XML file exists and is not empty, exit otherwise (The above command will produce an empty file if the VM does not have the vApp options enabled.)

Line 41-45: Sanity check #3: If the current hostname already matches the value of the OVF property then there is nothing to do.

Line 46-49: Sanity check #4: Check if a change that the script will do later (see lines 57ff) was already done. Then the script already did its work and we can exit now.

Line 53-54: Now we call the getprops_from_xml() function and pass the XML file to it. Enclosing the call into backquotes will result in its output being evaluated by the shell. So we actually execute the line "hostname=vESXi34" here.
The XML file is removed then to nicely clean up.

Line 56: Here we actually set the new short hostname by using the appropriate esxcli command.

Line 57-64: Now that's the part that is probably the most specific to my environment: I add some parameters to the configuration files of the ISC DHCP client here (for both IPv4 and IPv6) that configure it to send the newly configured hostname with the DHCP lease request (see the man page of dhclient.conf).

I have configured my DHCP server to then register the hostname with the DNS server and create both a forward and reverse lookup record there. This was quite easy to setup on the Windows box that I use to run both DHCP and DNS, but I had the same setup on a Linux box before, so this is also possible. In this way I get the whole DNS setup nicely automated just by setting the hostname in the machines that I deploy.

Now, if there was only an easy way to restart the DHCP client to pick up these changes! Restarting the management network would include this, and you can actually do this from the DCUI, but I could not figure out a command line that does the same (Anyone?!). So, I ended up with rebooting the host at the end of the script to ensure the proper name registration ... By the way, this is why I do all these sanity checks at the beginning of the script. During testing I often found myself in an endless reboot cycle, so I got very careful ...

Yeah, and that's it!

Update (2014-01-29): William Lam found out a great way to restart the management network via the command-line. This can be done by disabling and re-enabling the Management interface. I changed my script to just do that instead of rebooting, and it's working fine. Thanks, William!

---

This post first appeared on the VMware Front Experience Blog and was written by Andreas Peetz. Follow him on Twitter to keep up to date with what he posts.

---

---