Community developed software packages are typically ESXi drivers for hardware that is not supported by VMware (or 3rd party hardware vendors) out-of-the-box. Other examples are: Custom firewall rules (like the one that allows remote NTP queries) or scripts/binaries that add custom functionalities to ESXi (like the ProFTPd daemon).
Requirements
The scripts run on any recent Windows Workstation or Server version (both 32-bit or 64-bit) starting with Windows XP resp. Server 2003.
Installation Instructions
Download the latest version of the Packaging Tools (see Download section below). They are distributed as a signed self-extracting archive (created with 7-zip). Unpack the archive to a directory of your choice.
Usage Instructions
Step 1: Create your TGZ file
As a first step you need to create a gzip compressed tar archive (tar.gz or tgz) of all the files that make up your software package. It is necessary that
- all file paths in the archive are relative to the root (/) directory of the ESXi system, and
- all directories included in file paths must be explicitly added to the archive on their own
The easiest way to achieve that is to create an empty directory (on the ESXi or Linux system that you used for development), create the needed directory structure (starting at the root) in there and copy all files you want to include into the proper sub directories.
Here is an example of the required commands. For this example we assume that we want to build a driver package that consists of the following three files:
/etc/vmware/driver.map.d/igb.map
/usr/lib/vmware/vmkmod/igb
/usr/share/hwdata/driver.pciids.d/igb.ids
# cd /tmp
# mkdir build
# cd build
# mkdir -p etc/vmware/driver.map.d
# cp /etc/vmware/driver.map.d/igb.map etc/vmware/driver.map.d
# mkdir -p usr/lib/vmware/vmkmod
# cp /usr/lib/vmware/vmkmod/igb usr/lib/vmware/vmkmod
# mkdir -p usr/share/hwdata/driver.pciids.d
# cp /usr/share/hwdata/driver.pciids.d/igb.ids usr/share/hwdata/driver.pciids.d
You can then create the tgz file with
# tar cvzf ../net-igb.tgz *
When used with the v (=verbose) option the tar command will output the files that it added to the archive:
etc/
etc/vmware/
etc/vmware/driver.map.d/
etc/vmware/driver.map.d/igb.map
usr/
usr/lib/
usr/lib/vmware/
usr/lib/vmware/vmkmod/
usr/lib/vmware/vmkmod/igb
usr/share/
usr/share/hwdata/
usr/share/hwdata/driver.pciids.d/
usr/share/hwdata/driver.pciids.d/igb.ids
And this is exactly what it should look like. Please note that if you just add the individual files to the archive but not the directories you will risk a boot failure with a "Boot image corrupt" message once you install this package on a host.
So what have we got now? We now have a tgz archive of the files that make up our new ESXi software package. But before we can actually use or distribute it we need to convert it to the VMware proprietary VIB format. Here is how ...
Step 2: Build a VIB file with tgz2vib5.cmd
What is a VIB file? VIB stands for VMware Installation Bundle. A VIB file includes a file archive like the one we created in Step 1 (a so-called payload file), an XML file (descriptor.xml) with descriptive metadata and an electronic signature file (sig.pkcs7).
To create a VIB file from our tgz file run tgz2vib5.cmd:
The GUI is pretty self-explanatory, but here are some guidelines for using the right options and entering suitable metadata.
With the first Browse button you select the tgz file that we created in step 1. Since version 2.1 of the script you can alternatively select a "build directory" to have it build the tgz file on its own from a Windows directory structure. This will save some of the work of step 1 and might completely eliminate the requirement to run a Linux system, but please note that by creating the tgz file in Windows you will lose any Linux specific file attributes like the X bit for executable binaries or symbolic links!
You can toggle between the two modes with the radio bottom at the top.
With the second Browse button you select the "working directory" of the script. This directory will be used to store temporary files, the log file and the resulting VIB file.
For the VIB Type you will choose "bootbank" in almost all cases. Currently there is only one known case of a "locker" type package, and that is the tools-light VIB including the VMware Tools that are embedded into ESXi for easy Guest OS installation. Detailed instructions for how to build a tools-light VIB using tgz2vib5.cmd are available in a separate blog post.
VIB description data: This is part of the metadata that will be stored in the file descriptor.xml.
I suggest that you follow these recommendations to select proper values:
Package properties: In this section you can define dependencies to other packages, tags for your package and various installation flags. You can load presets for all these properties for four different types of packages: A "Hardware driver" (ESXi 5.0+, 5.1+, 5.5+ or 6.0+, see explanations below), a "Firewall rule", "esxcli plugin" and "VMware Tools". If you have customized the settings by yourself you can also load the settings from your last TGZ2VIB5 session.
Since version 2.2 of the script you can specify the compatibility level of a hardware driver by loading the corresponding preset. Please pick the version of the ESXi Open Source code that the driver was compiled with. Drivers that were compiled for one version will also work with later ESXi versions. So if the Open Source Code of ESXi 5.0 was used for compiling the driver then load the preset "Driver (ESXi 5.0+)" and the resulting driver package will also work with ESXi 5.1, 5.5 and 6.0.
Since version 2.1 of the script you can also select an Acceptance Level for the VIB package. The default is community (CommunitySupported) which does not require any trusted electronic signature, but limits the system directories that are allowed to be modified by the VIB package. If you want to modify arbitrary system directories with your package then you need to select accepted (VMwareAccepted) or certified (VMwareCertified) here. Please see this post for details about the limitation of the community acceptance level and the implications of changing it.
For the various installation flags there are tooltips that explain their meanings in case you want to fine tune them. Such quick help tooltips are available for almost all dialog elements of the GUI.
The screenshot shows the presets for an ESXi 5.0+ driver package with the CommunitySupported acceptance level.
Packaging options: In this section you can choose a repacking option for the TGZ file:
Enable automatic update check: Please leave this option enabled to be informed about updates of this script. Disable it only if the machine that is running the script does not have a direct internet connection, because then it may delay the script execution.
Pause execution for manual editing of descriptor.xml: Below you see a hardcopy of the descriptor.xml file that tgz2vib5.cmd will produce for the given example. Although you can set most of the tags through the GUI shown above you might have the requirement to change some of the more advanced options that are not covered in the GUI. You can do this by manually editing the descriptor.xml file, and this is what this option is for.
So what have we got now? The VIB file that you created with tgz2vib5.cmd can be installed into a running ESXi 5.x/6.x system by using the following commands inside an ESXi shell (or via the vSphere remote CLI):
The second command installs the package that needs to be locally accessible by ESXi. Alternatively you can provide a http-URL to download the vib file from a web site.
If you have changed the Acceptance Level of the VIB package when building it to e.g. accepted then you normally do not need the first command to change the system acceptance level (this is set to partner by default). However, you then need to modify the install command to include an option to skip the electronic signature check like this:
You can also create a customized ESXi installation ISO with the VIB file already added by using my ESXi-Customizer-PS script. This is e.g. necessary to get whitebox systems installed that need a community supported driver for their NICs. Please find details about this usage scenario in my blog post How to get your unsupported NIC to work with ESXi 5.x and 6.0.
Step 3: Build an Offline Bundle with vib2zip.cmd
What is an Offline Bundle? Basically it is a package containing one or multiple VIB files and additional metadata. The idea behind that is to create a depot of software packages. Before we dive deeper into this here is how you can create your own Offline bundle.
Alex Lopez has created a great video tutorial for using the vib2zip.cmd tool:
Highly recommended - if you like learning by watching videos then give it a try!
To get the full picture though you should read through the following documentation before you start using vib2zip.
Run the vib2zip.cmd script:
All VIB files that you want to be included in the Offline bundle must be copied into the same directory (which is referred to as the source VIB directory here). If you will add only one VIB file it makes perfect sense to use the same metadata attributes for the Offline bundle that you also used for the VIB file (whereas the KB-URL corresponds to the Details URL of the VIB file). Otherwise you should just choose anything else that makes sense here.
With an Offline bundle you have two additional attributes available:
With the ESXi version compatibility setting you can define on what versions of ESXi your package can be installed. The default is all minor versions of ESXi 5 (5.*) plus 6.0. Binary compatibility depends on the version of the ESXi Open Source Code that was used to build the driver. If the OSS code of ESXi 5.0 was used then the driver will be compatible with ESXi 5.* and 6.0. If the OSS code of ESXi 5.1 was used then it will be compatible with ESXi 5.1, 5.5, 6.0 and 6.5. And if the OSS code of ESXi 5.5 was used then it will be compatible with ESXi 5.5, 6.0, 6.5 and 6.7.
Now press Run! to create the Offline bundle ZIP file ...
So what have we got now? Like VIB files you can use Offline bundles with my ESXi-Customizer-PS script to create customized ESXi installation images. Like VIB files Offline bundles can also be used to install a contained software package into a running ESXi 5.x/6.x system by using the following commands inside an ESXi shell (or via the vSphere Remote CLI):
So, what is unique to Offline bundles? You can
Support
# mkdir build
# cd build
# mkdir -p etc/vmware/driver.map.d
# cp /etc/vmware/driver.map.d/igb.map etc/vmware/driver.map.d
# mkdir -p usr/lib/vmware/vmkmod
# cp /usr/lib/vmware/vmkmod/igb usr/lib/vmware/vmkmod
# mkdir -p usr/share/hwdata/driver.pciids.d
# cp /usr/share/hwdata/driver.pciids.d/igb.ids usr/share/hwdata/driver.pciids.d
You can then create the tgz file with
# tar cvzf ../net-igb.tgz *
When used with the v (=verbose) option the tar command will output the files that it added to the archive:
etc/
etc/vmware/
etc/vmware/driver.map.d/
etc/vmware/driver.map.d/igb.map
usr/
usr/lib/
usr/lib/vmware/
usr/lib/vmware/vmkmod/
usr/lib/vmware/vmkmod/igb
usr/share/
usr/share/hwdata/
usr/share/hwdata/driver.pciids.d/
usr/share/hwdata/driver.pciids.d/igb.ids
So what have we got now? We now have a tgz archive of the files that make up our new ESXi software package. But before we can actually use or distribute it we need to convert it to the VMware proprietary VIB format. Here is how ...
Step 2: Build a VIB file with tgz2vib5.cmd
What is a VIB file? VIB stands for VMware Installation Bundle. A VIB file includes a file archive like the one we created in Step 1 (a so-called payload file), an XML file (descriptor.xml) with descriptive metadata and an electronic signature file (sig.pkcs7).
To create a VIB file from our tgz file run tgz2vib5.cmd:
 |
| TGZ2VIB5 v2.4 GUI |
The GUI is pretty self-explanatory, but here are some guidelines for using the right options and entering suitable metadata.
With the first Browse button you select the tgz file that we created in step 1. Since version 2.1 of the script you can alternatively select a "build directory" to have it build the tgz file on its own from a Windows directory structure. This will save some of the work of step 1 and might completely eliminate the requirement to run a Linux system, but please note that by creating the tgz file in Windows you will lose any Linux specific file attributes like the X bit for executable binaries or symbolic links!
You can toggle between the two modes with the radio bottom at the top.
With the second Browse button you select the "working directory" of the script. This directory will be used to store temporary files, the log file and the resulting VIB file.
For the VIB Type you will choose "bootbank" in almost all cases. Currently there is only one known case of a "locker" type package, and that is the tools-light VIB including the VMware Tools that are embedded into ESXi for easy Guest OS installation. Detailed instructions for how to build a tools-light VIB using tgz2vib5.cmd are available in a separate blog post.
VIB description data: This is part of the metadata that will be stored in the file descriptor.xml.
I suggest that you follow these recommendations to select proper values:
- For the Name attribute use the name of the associated Linux device driver or software. For drivers prepend a prefix that denotes the driver's device type: net- for a network interface driver, scsi- for a SCSI controller driver, ata- and sata- for (S)ATA controller drivers.
- For the Version attribute use the version of the associated Linux device driver or software (5.2.10 in this example) and append a unique build number (-9 in the example) that will help to distinguish multiple builds of the same source code version.
- For the Vendor attribute fill in the vendor's or author's name (meaning the producer of the device, resp. the author of the software).
- In the Summary field explain shortly what the package is about, in the Description field enter a more detailed explanation of e.g. what devices the driver can be used for or what features the software has. This is also the place to give credits to yourself or adding any other information that might be helpful for the users of this driver package.
- In the Details URL field you can enter the address of a web site with further information about the package (e.g. a forum's post, a KB article or a page on your personal web site).
Package properties: In this section you can define dependencies to other packages, tags for your package and various installation flags. You can load presets for all these properties for four different types of packages: A "Hardware driver" (ESXi 5.0+, 5.1+, 5.5+ or 6.0+, see explanations below), a "Firewall rule", "esxcli plugin" and "VMware Tools". If you have customized the settings by yourself you can also load the settings from your last TGZ2VIB5 session.
Since version 2.2 of the script you can specify the compatibility level of a hardware driver by loading the corresponding preset. Please pick the version of the ESXi Open Source code that the driver was compiled with. Drivers that were compiled for one version will also work with later ESXi versions. So if the Open Source Code of ESXi 5.0 was used for compiling the driver then load the preset "Driver (ESXi 5.0+)" and the resulting driver package will also work with ESXi 5.1, 5.5 and 6.0.
Since version 2.1 of the script you can also select an Acceptance Level for the VIB package. The default is community (CommunitySupported) which does not require any trusted electronic signature, but limits the system directories that are allowed to be modified by the VIB package. If you want to modify arbitrary system directories with your package then you need to select accepted (VMwareAccepted) or certified (VMwareCertified) here. Please see this post for details about the limitation of the community acceptance level and the implications of changing it.
For the various installation flags there are tooltips that explain their meanings in case you want to fine tune them. Such quick help tooltips are available for almost all dialog elements of the GUI.
The screenshot shows the presets for an ESXi 5.0+ driver package with the CommunitySupported acceptance level.
Packaging options: In this section you can choose a repacking option for the TGZ file:
- If you have followed the instructions from step 1 it is safe to choose "Do not touch" here.
- If you have not created the tgz file yourself you might want to choose "Force repacking", because this will repack the file and ensure that the complete directory structure is included (thus preventing the infamous "Boot image corrupt" failures).
- The last option "Force repacking and pause for adv. editing" will first unpack the tgz archive, then pause the script execution to let you manually edit the included files or even add new files to it, before it repacks the files into a new tgz package. Use this option only if you know what you are doing!
By default checksums are generated for the payload file and embedded into the descriptor.xml file. You can omit these by unchecking the respective option.
Runtime options:
Pause execution for manual editing of descriptor.xml: Below you see a hardcopy of the descriptor.xml file that tgz2vib5.cmd will produce for the given example. Although you can set most of the tags through the GUI shown above you might have the requirement to change some of the more advanced options that are not covered in the GUI. You can do this by manually editing the descriptor.xml file, and this is what this option is for.
 |
| descriptor.xml file created by TGZ2VIB5 v2.4 |
So what have we got now? The VIB file that you created with tgz2vib5.cmd can be installed into a running ESXi 5.x/6.x system by using the following commands inside an ESXi shell (or via the vSphere remote CLI):
# esxcli software acceptance set --level=CommunitySupportedThe first command is necessary for the system to accept packages that have the acceptance level "CommunitySupported" and do not have a valid electronic signature (like the ones produced by tgz2vib5.cmd).
# esxcli software vib install -v /path/to/vib-file
The second command installs the package that needs to be locally accessible by ESXi. Alternatively you can provide a http-URL to download the vib file from a web site.
If you have changed the Acceptance Level of the VIB package when building it to e.g. accepted then you normally do not need the first command to change the system acceptance level (this is set to partner by default). However, you then need to modify the install command to include an option to skip the electronic signature check like this:
# esxcli software vib install --no-sig-check -v /path/to/vib-filePlease note that this imposes a security risk to ESXi (and you will see a big fat warning about this in /var/log/vmkernel.log)! So you will want to use the --no-sig-check option only for packages that come from an origin that you fully trust.
You can also create a customized ESXi installation ISO with the VIB file already added by using my ESXi-Customizer-PS script. This is e.g. necessary to get whitebox systems installed that need a community supported driver for their NICs. Please find details about this usage scenario in my blog post How to get your unsupported NIC to work with ESXi 5.x and 6.0.
Step 3: Build an Offline Bundle with vib2zip.cmd
What is an Offline Bundle? Basically it is a package containing one or multiple VIB files and additional metadata. The idea behind that is to create a depot of software packages. Before we dive deeper into this here is how you can create your own Offline bundle.
Alex Lopez has created a great video tutorial for using the vib2zip.cmd tool:
Highly recommended - if you like learning by watching videos then give it a try!
To get the full picture though you should read through the following documentation before you start using vib2zip.
Run the vib2zip.cmd script:
 |
| vib2zip v1.4 GUI |
With an Offline bundle you have two additional attributes available:
- The Vendor code is an abbreviation of the vendor's or author's name. For commercial packages it is common practice to use the vendor's stock ticker symbol here. I suggest to not use more than five characters, all capital, without blanks.
- For Contact you might enter your own E-mail address if you are willing to answer questions via E-mail. However, like KB-URL this is optional!
With the ESXi version compatibility setting you can define on what versions of ESXi your package can be installed. The default is all minor versions of ESXi 5 (5.*) plus 6.0. Binary compatibility depends on the version of the ESXi Open Source Code that was used to build the driver. If the OSS code of ESXi 5.0 was used then the driver will be compatible with ESXi 5.* and 6.0. If the OSS code of ESXi 5.1 was used then it will be compatible with ESXi 5.1, 5.5, 6.0 and 6.5. And if the OSS code of ESXi 5.5 was used then it will be compatible with ESXi 5.5, 6.0, 6.5 and 6.7.
Now press Run! to create the Offline bundle ZIP file ...
So what have we got now? Like VIB files you can use Offline bundles with my ESXi-Customizer-PS script to create customized ESXi installation images. Like VIB files Offline bundles can also be used to install a contained software package into a running ESXi 5.x/6.x system by using the following commands inside an ESXi shell (or via the vSphere Remote CLI):
# esxcli software acceptance set --level=CommunitySupportedLike with the VIB file installation in Step 2 you can also skip the first command if you changed the acceptance level of all included VIB files to partner or accepted. And then you also must add the --no-sig-check option to the install command.
# esxcli software vib install -d /path/to/zip-file
So, what is unique to Offline bundles? You can
- import them into vSphere Update Manager (to automate the update process)
Licensing
- The ESXi Community Packaging Tools are licensed under the GNU GPL version 3 (see the included file COPYING.txt).
- They are distributed with and make use of several tools that are freely available, but are partly under different licenses (see the included file tools\README.txt for details.)
- Please check the known bugs section here first to see if your issue is covered there.
- If you have trouble using the scripts then please send an email to ESXi-CPT@v-front.de. Be sure to include the log file(s). Otherwise I might just ignore your message.
- vib2zip.cmd in versions prior to 1.4 produced Offline bundles that could not be imported into Update Manager (error message: "Invalid package"). This was fixed in version 1.4.
Changes / History
v2.4 (2019-02-25):
v2.3 (2015-11-09):
v2.2.1 (2014-01-05):
v2.2 (2013-09-08):
v2.1 (2012-11-12):
v2.0 (2012-07-10):
v1.0 (2012-02-12):
- Updated VIB2ZIP.cmd to version 1.4:
- Added ESXi 6.5 and 6.7 and 6.* to compatibility setting choices
- Fixed a bug that prevented import of Offline bundles into VUM 6.5 and newer
v2.3 (2015-11-09):
- Renamed "ESXi5 Community Packaging Tools" to "ESXi Community Packaging Tools"
- Updated TGZ2VIB5.cmd to version 2.3:
- fixed acceptance level setting "certified" (falsely set to "vmware" before)
- added support for vibType "locker"
- added presets for "ESXi 6.0+ driver" and "VMware Tools" (tools-light.vib)
- Updated VIB2ZIP.cmd to version 1.3:
- Added ESXi 6.0 to compatibility setting choices
v2.2.1 (2014-01-05):
- Updated TGZ2VIB5.cmd to version 2.2.1:
- Moved version check to vibsdepot.v-front.de
- added checks for "(" and ")" in file names (might make the script fail)
- Updated VIB2ZIP.cmd to version 1.2.1:
- Fixed kbUrl tag spelling
- Moved version check to vibsdepot.v-front.de
- added checks for "(" and ")" in file names (might make the script fail).
v2.2 (2013-09-08):
- Updated TGZ2VIB5.cmd to version 2.2:
- Ask for "Summary" and optional "Details URL" tag in description data
- Added presets "Driver (ESXi 5.0+)", "Driver (ESXi 5.1+)" and "Driver (ESXi 5.5+)" (replacing former "Hardware driver" preset)
- Added preset "esxcli plugin"
- Added informational tool tips for installation flags
- Updated VIB2ZIP.cmd to version 1.2:
- Ask for vendor code tag in metadata
- Ask for ESXi version compatibility (5.x or (5.0 and/or 5.1 and/or 5.5))
- Added option to pre-load metadata from a VIB file in the build directory
v2.1 (2012-11-12):
- Updated TGZ2VIB5.cmd to version 2.1:
- Added option to build payload file from build directory
- Added "Acceptance Level" option to package properties
- Updated VIB2ZIP.cmd to version 1.1:
- GUI improvements (tooltips) and code cleanup
v2.0 (2012-07-10):
- Updated TGZ2VIB5.cmd to version 2.0:
- Added GUI options to allow setting of advanced VIB properties and installation parameters
- Added menu to load presets of GUI options for various package types ("Hardware driver" and "Firewall rule")
- Added option to omit payload checksums in descriptor XML file
- VIB2ZIP was not changed in this release.
v1.0 (2012-02-12):
- first release