vSphere 4.1 Update 2 released - What's in it for me (and you)

VMware released Update 2 for vSphere 4.1 on Oct 27th. It includes numerous bug fixes for the vCenter server and client (see VC resolved issues) and ESXi (see ESXi resolved issues).

 I will list some of the fixes here, because I personally welcome them very much, and I'm sure that others will feel the same:

• The vSphere client performed badly with Windows 7, because of frequent screen-redraws when the Windows desktop composition feature was enabled. The only workaround was to disable desktop composition while running the vSphere client. This should be fixed now.
• There are multiple fixes and enhancements to ESXi syslogging:
• If ESXi fails to reach the syslog server while booting, it now keeps retrying it every 10 minutes.
• Very long syslog messages (like these produced by the vpxagent ...) are no longer truncated or split into multiple lines. If you are using a third-party solution like Splunk for collecting syslog messages then you will certainly welcome this, because it is nearly impossible to handle split messages correctly with them.
• But the most important issue that is resolved in Update 2 is this: "Virtual machine with large amounts of RAM (32GB and higher) loses pings during vMotion". Uh, what? VMs losing pings when being vMotioned? Yes, this can really happen (without Update 2), and I personally experienced this problem: When one of two clustered Microsoft Exchange 2010 VM with 48GB RAM was vMotioned it lost network connectivity for more than 15 seconds (between 20 and 30% of the vMotion progress) which triggered a cluster failover. We have not yet checked if this particular issue is really resolved now with Update 2, but VMware Support had put us off to it when we complained about that, so there is a good chance ...

Update: ESXi 5.0 on HP G7 blades, now a Go!

About three weeks back I reported on Emulex firmware problems that prevented the use of ESXi 5.0 on HP G7 blade hardware. This was fixed now, somehow...

HP has now updated the advisory that describes the issue and published an updated firmware that fixes the VLAN handling problems with ESXi 5.0 if it is used together with the be2net driver 4.0.355.1.

Be sure that you read the release notes of the firmware! It looks like it is an emergency/workaround release that leaves many issues unresolved. A firmware version that you can really trust for production will probably be available mid-November.

Update (2012-12-09): HP and Emulex published the final version of the OneConnect firmware (4.0.360.15a) on Nov 19th. VMware's KB2007397 also lists the recommended drivers to use with this firmware for both ESXi 4.1 and 5.0.

Update (2012-03-09): HP has published yet another firmware update on March 5th. Download version 4.0.360.15b. The previous link has become invalid.

Update (2012-04-16): Please refer to my HP & VMware links page to find the download for the latest version of the firmware.

VMware finally released the Open Source Code of vSphere 5.0!

Great news! Today VMware finally made the vSphere v5.0 Open Source code archives available for download.

Why is that important?

Since the release of VMware's ESXi 5.0 (Aug 24, 2011) many people are asking for the development of drivers for hardware devices that are not supported by ESXi 5.0 out-of-the-box.

ESXi device drivers are based on Linux device drivers (which lead to the persistent misunderstanding that ESXi itself is based on Linux), but the stock Linux driver code must be modified in a specific way to be compatible with ESXi.

With past versions of ESXi (up to 4.1) it was possible to study and reproduce these required modifications, because VMware published the source code of the ESXi device drivers (the original Linux code plus their modifications). The reason for this is that most Linux drivers are licensed under the GPL (General Public License), and the GPL requires that derived works are also published under the GPL and their source code is made freely available (aka the "Copyleft" principle).

Now, that VMware also published the Open Source Code of ESXi 5.0 (including the device drivers that it contains) it will be possible (or at least much easier) to develop custom ESXi 5.0 drivers for devices that are not officially supported by VMware.

HP Virtual Connect profile not applied ...

When I recently rebooted one of our BL620c G7 blades with ESXi 4.1 installed on it I found that the server had suddenly lost network connectivity after the reboot.
A quick check on the console revealed that the Virtual Connect profile that was defined for that blade had not been applied to it. I realized that because the MAC addresses of the NICs had not been overwritten with the virtual addresses of the Virtual Connect profile.

I tried powering down and up the blade, re-assigning the Virtual Connect profile multiple times, all to no avail ... Then I had the idea that it might be related to the iLO-board of the blade, and - yes, indeed - after resetting the iLO3-board of the blade the Virtual Connect profile was properly applied and all was fine again.

While later looking on hp.com for some related information I stumbled over the Customer advisory c02820591 that described an issue with the Virtual Connect profile being lost upon an iLO3 reset. Not exactly the issue that I had, and the advisory also stated that this is fixed with iLO3 firmware version 1.20, and that is already installed on our iLOs. However, the advisory confirmed my assumption that the Virtual Connect profile is applied by the iLO-board.

So, if you have similar problems try resetting the iLO-board before you start pulling your hair out, or the blade out of the chassis ...

Currently a No-Go: ESXi 5.0 on HP G7 blades

Back in May I reported on problems with ESXi 4.1 and the Emulex OneConnect CNA that is built into HP's G7 blade servers.
If you now try to install ESXi 5.0 on such a hardware you will have a strong déjà vu: The be2net driver that is available right now for ESXi 5.0 is not really functioning due to "VLAN tagging issues". HP has published an advisory on this stating that an updated driver (that should fix these issues) is "currently in the certification process" and will be made available in "Q4 2011".

Okay, I won't update our production hosts to ESXi 5.0 that soon anyway, but I just wanted to install it on some spare blades for testing and evaluation. Too bad ... waiting for a fix again ...

Update (2011-10-27):
HP has now updated the advisory and published an updated firmware that fixes the VLAN handling problems with ESXi 5.0 if it is used together with the be2net driver 4.0.355.1.
Be sure that you read the release notes of the firmware! It looks like it is an emergency/workaround release that leaves many issues unresolved. A firmware version that you can really trust for production will probably be available mid-November.

Update (2012-04-16):
In the meantime it looks like all problems have been fixed with newer firmware and driver versions. Please refer to this newer post of mine!

Unable to assign license after installing a server with the HP ESXi 5.0 ISO

With the availability of vSphere 5.0 HP published a customized ESXi installation ISO for HP servers.
There have been reports that this build includes an annoying bug: HP has included a license file that has wrong permissions set. That potentially causes errors once you want to assign an own license to the host.

You can fix this by removing the offending license file with the following commands (to be executed on the ESXi host after directly after it has been installed):

  esxcli software vib remove -n hp-esx-license --no-live-install

and reboot the host. The command will remove the HP license and restore the original state of the host being in evaluation mode.

HP has also published an advisory describing the problem and providing a way to update their license package to fix the problem.