2014 - the fourth year of V-Front - is coming to an end. Like in the last years I have a list of highlights that I want to share - in case you missed any of them ...
2014: Year-end review
2014 - the fourth year of V-Front - is coming to an end. Like in the last years I have a list of highlights that I want to share - in case you missed any of them ...
How to make your unsupported NIC work with ESXi 5.x or 6.0
When you try to install VMware ESXi on some whitebox hardware that is not officially supported by VMware then your attempt might come to an unpleasant end after the installer presented the error message shown above. ESXi has in-box support for a limited number of network interface cards (NICs), and sadly a lot of consumer grade devices are not on the list.
Is this the end of the world? No. If you are a regular reader of my blog then you probably already know that help (and in some cases even rescue) is available. However, I keep getting e-mails from people asking how to get their NIC xyz to work with ESXi ... So I finally took the time to write down all the steps that you need to take and the options you have - just to point them here instead of giving the same answers per e-mail again and again.
A very similar post of mine is How to make your unsupported SATA AHCI controller work with ESXi 5.5 and 6.0 - it is about a year old now and with 80k pageviews my most successful blog post ever. So far. Let's see how this one goes ...
How to avoid browser warnings when using the vCenter Web Client with a self-signed certificate
If you have a recent VMware vSphere installation with a vCenter server in production or in a lab then you will be aware of that the Web Client is the recommended choice for managing the environment and that the well known C# based vSphere Client is considered deprecated or legacy (since version 5.1 already).
However, when you connect to the Web Client of your freshly installed vCenter server for the first time using your favorite Internet browser you will be greeted by a more or less alarming warning. Chrome even warns you that VMware might steal your credit card information ;-) (well, they probably already have that) ... You should really be worried whenever you see this warning on a random Internet site, but you don't need to if it's your company internal vCenter server that you try to access.
So, why do you get this message, and how can you get rid of it?
VMware silently adds native USB 3.0 support to ESXi 5.5
The October 2014 patch of ESXi 5.5 already got a lot of attention, because it introduced additional Transparent Page Sharing (TPS) management capabilities to prepare for TPS being disabled by default in upcoming ESXi releases. And William Lam pointed out that this patch enables support for the Apple Mac Pro 6,1 ... but there is even more goodness in this patch!
When adding the associated Image Profiles to the VibMatrix I noticed that this bundle includes a new package named xhci-xhci. The related KB2087362 article only includes the standard disclaimer, but no information about what this really means: xHCI stands for Extensible Host Controller Interface, a USB standard that supports USB 3.0 (or SuperSpeed) controllers and devices.
That means with the latest ESXi 5.5 patch you are - for the first time - able to utilize USB Passthrough with USB 3.0 devices!
[Release] ESXi-Customizer-PS 2.3 - The ESXi image customization script
I have just released version 2.3 of my ESXi-Customizer-PS script - a PowerCLI script to create customized ESXi installation images and the destined successor of my ESXi-Customizer tool.
I was somewhat in a hurry to get this out, because with the release of ESXi 5.5 Update 2 a bug in PowerCLI ImageBuilder manifested itself that needed a workaround. The new version implements this workaround, but also has one other improvement that you will like.
Great flings and a contest!
VMware releases vSphere 5.5 Update 2 - What's new and how to update free ESXi
It looks like yesterday the whole IT Community was pretty much focused on the much anticipated Apple Live event, so you might have missed an important VMware announcement: vSphere 5.5 Update 2 was released. Besides the usual bug fixes there was one important new functionality added that will make a lot of VMware Admins happy, especially those who do not like the new Web Client.
Before going into some details here are the relevant links to downloads and release notes:
Between VMworlds 2014 ... See you in Barcelona?
Unless you have been living under a rock during the last two weeks you will know that VMworld 2014 US took place in San Francisco. I have not been there, but enjoyed the summer vacation with my family in a rural area of Germany with very poor mobile network coverage. Anyway the constant never-ending stream of VMworld news and buzz could not completely elude my attention ...
Again, this was a great conference with exciting announcements - No worries, I will not repeat here what the crowd of well-known virtualization bloggers has already shouted out. But I am somewhat concerned that there is no more good news left for the yet upcoming VMworld 2014 Europe conference in Barcelona (Oct 14th to 16th) that I will be happy to attend again.
The ESXi MAC learning network filter fling - and how to install it from the V-Front Online depot
With all the ongoing VMworld buzz there is at least one news over at William Lam's VirtuallyGhetto blog that did not get the attention that it deserves: VMware has released a new fling for ESXi that implements a MAC learning network filter. That is an important one and highly recommended for all the folks that use nested virtualization on top of ESXi.
Why?
Do you need disk encryption for hosted VMs?
Then this happened:
Useful PowerCLI scripts for storage migrations with RDMs
I'm currently supporting a large VMware Storage migration project affecting MS Exchange 2010 server VMs that use lots of Raw Device Mappings (RDM) for mailbox storage. This was a good opportunity to further improve my Powershell / PowerCLI skills - I wrote some scripts to gather the data needed in a friendly format, and I thought it would be a good idea to share them here.
Sponsor news 02/2014
This is the second edition of my 2014 random blog sponsor news. This time we get greener - I'm glad to welcome a new sponsor, and that is ...
CPU microcode update needed to fix VM crashes with certain Intel Xeon E5/E7 v2 processors
The root cause is a known bug in the CPU microcode of the mentioned processors. It leads to incorrect memory page translations when the EPT (Extended Page Tables) feature of the CPU is enabled and is mentioned as Erratum CA135 in an Intel specification update of the E5 v2 CPU family. EPT is an important CPU virtualization feature and helps to offload the VM memory address translation to the CPU which can significantly increase a VM's performance. So this bug is rather critical! How can it be fixed?
New ESXi 5.5 patch fixes NFS bug and another serious OpenSSL bug!
Yesterday VMware finally released a fix for the nasty NFS bug that was introduced with ESXi 5.5 Update 1. Customers who were waiting to update to U1 because of this bug can now safely update their hosts and will also get protection from the OpenSSL Heartbleed bug.
But there is another reason why you should update your ESXi 5.5 hosts with this patch very soon - even if you are not affected by the NFS bug and have already applied the Heartbleed fix!
How to download individual VIBs from the VMware Online Depot
Lately I was troubleshooting a PSOD issue on one of our ESXi 5.0 hosts with VMware Support, and they advised me to downgrade the bnx2 driver to the ESXi builtin version (I had a newer HP supplied version installed because it was HP hardware). Later it turned out that this was a wrong advice, but at that time I wondered: How do you do this?
The support guy pointed me to the well known KB article Installing async drivers on VMware ESXi 5.0, 5.1 and 5.5, but the instructions there assume that you have downloaded an Offline bundle of the new driver from somewhere. However, the version of the net-bnx2 package that I was advised to install (2.0.15g.v50.11-5) was not available for download anywhere ...
When Onboard Administrators go bad ... - an important heads-up for HP Blade Enclosure users
We recently had a very unpleasant event with one of our HP Blade Enclosures. Four of the eight ProLiant BL620c G7 servers suddenly lost access to the SAN storage for no apparent reason. By looking at the logs of the Onboard Administrators (OA) and Virtual Connect (VC) FlexFabric modules we found out that something really bad happened inside the enclosure, and it took us some time and the help of HP Support to permanently fix that.
Free backup for free ESXi: Thinware vBackup vs. Trilead VM Explorer
VMware ESXi with the free license (also known as vSphere Hypervisor) is a great way to get started with server virtualization and run your own hypervisor at home or in small environments. As soon as you have some sort of "production" workload running in VMs you will start thinking about how to protect them from data loss. You need backup ... but unfortunately with the free ESXi license VMware has disabled some functionality that is important for efficient backups of VMs: VADP (vStorage APIs for Data Protection) and CBT (Changed Block Tracking) are the features that all modern software products for VM backups make use of.
Nevertheless there are several solutions available to back up VMs running on free ESXi. And best of all these are available for free themselves. Here are your options.
Sponsor news 01/2014
From now on I will post news and updates from my blog's sponsors in irregular intervals. This is not only a tribute to my sponsors, but also a service to my readers, because this will get you fresh news and information and lots of opportunities for e.g. free webinars, to win cool swag etc.
So here is this year's first edition of my sponsor news ...
Struck by the NFS bug? Here is how to uninstall ESXi 5.5 Update 1.
In the meantime the only workaround is to downgrade your hosts to ESXi 5.5 GA (without Update 1). But how do you do this without re-installing ESXi?
OpenSSL Heartbleed patches for ESXi 5.5 are available now!
Protect your ESXi hosts against Heartbleed attacks
VMware has reacted to this by publishing a list of vulnerable products and announced appropriate patches to be released very soon. ESXi is also affected. A general recommendation of VMware is to not expose the management interface of your hosts directly to the Internet, but there are situations where you cannot really avoid that or just do not want to sacrifice the convenience to have your hosts manageable from (theoretically) everywhere.
If you have ESXi hosts directly connected to the Internet and can not or do not want to wait for a patch to be released (or just do not want to spoil your easter weekend with patching hosts) then you should think about other means to mitigate the issue.
How to automate changing the VMware Tools installation in Windows
A while ago I blogged about the VMware Tools for Windows MSI package, what hidden components it includes, and how you can take complete control over the installation or upgrade process by using appropriate msiexec command lines.
The following question was raised in this context: How do you modify an existing installation of VMware Tools (i.e. add or remove components) without re-installing or updating to a new version?
Almost a Top 50 vBlog and vExpert again!
Last week Eric Siebert announced the results of this year's Top Virtualization Blog voting. I am very happy that I could make a huge jump up in the list, climbing 90 places to rank #53. Many thanks to everyone who voted for me!
And today VMware announced the first list of vExpert 2014 awardees. I say the first, because there were substantial changes to the process this year resulting in quarterly nominations. So if you have missed to apply in February or were not accepted then you will get a new chance soon.
Anyway, after 2012 and 2013 I was awarded vExpert the third time in a row! Thanks to John Troyer, Corey Romero and all the other great guys caring about the VMware community!
I am always very glad to see that the hard work that I put in my blog and the tools and resources that I provide get recognized by the community. This - and the fact that I just have fun doing it - are the best reasons to carry on.
Welcome to Opvizor
I am excited to welcome Opvizor as a new sponsor to my blog! This fairly young startup offers a unique approach to keep your vSphere environment healthy. It is not about real-time monitoring (there are already plenty of products that do this), but Opvizor analyzes your environment for compliance with best practices, and it knows about and detects lots of common issues, misconfigurations and mistakes. Its goal is to prevent issues before they happen.
Updated: [ALERT] Issue with the ESXi 5.5 U1 Driver Rollup ISO: Software iSCSI adapter crashes hostd
While browsing through the VMware Community Forums I stumbled over a thread that made me curious: A customer experienced serious issues after upgrading his vSphere environment to 5.5 Update 1. The ESXi hosts became unresponsive right after the Software iSCSI adapter was added.
Issue after upgrading your vCenter 5.5 U1 server from Windows 2012 to 2012 R2
I had successfully done this kind of in-place upgrade before with my two Domain Controllers and the Veeam Backup server, and I was not really expecting any big issues, but this time there was one: After the upgrade to 2012 R2 finished the VMware vSphere Web Client service failed to start! It took me some time to find the root cause of this, but in the end it was easy to fix ...
How to update your standalone host to ESXi 5.5 U1
How to speed up the installation of Windows Updates
We recently upgraded a vSphere 5.0 environment to 5.0 U3 and used the maintenance window of the vCenter server to also install missing Windows updates on this and the vCenter database SQL server (both running Windows 2008 R2).
A generously dimensioned downtime of four hours was communicated before the upgrade -, and we managed to finish all the work just in time. Why? Because it took two hours alone to install the Windows updates! 18 patches were missing, and they took ages to install, particularly the .NET Framework 4 patches. I tried all sorts of things to speed this up during the installation (stopping unnecessary services and the Virus scanner), but to no avail. Eventually the updates finished successfully, and later I took myself some time to find out why it took so long and what you can do about this. Here is what I found out ...
Various ways to address the "Safely Remove Hardware" Tray Icon issue
Since a long time VMware supports device hot plug in its Virtual Machines. This leads to the "Safely Remove Hardware and Eject Media" tray icon to be displayed in Windows VMs, and it allows a user to eject the network card and even hard disks from the machine. While the latter will fail in most cases, because the hard disk is in use by Windows, ejecting the NIC will succeed ... and it will do exactly this. The NIC will disappear from the VM, which will disconnect from the network until a VMware administrator re-adds a new NIC to it!
How many times were curious Windows Admins or VDI desktop users not only tempted by this possibility, but even used it - only to find themselves disconnected from their machines with no way to re-connect ;-) ?! The first few times this may be funny, but then you will want to look for ways to prevent this.
Google is you friend, and you will easily find many different ways to address this issue, but scattered across even more blogs and sites. This is my attempt to collect all of them in one blog post.
Top VMware and Virtualization Blog voting 2014 now open
You will find this blog not only in the overall favorites list, but it is also participating in the categories "Best Scripting Blog" and "Best Independent Blogger". If you have found useful information, scripts or tools here at the VMware Front Experience Blog or/and are a regular reader then please show your support by voting for me.
Head over to vSphere-Land.com to cast your vote - Thanks!!
[Release] ESXi-Customizer-PS 2.2 - The ESXi image customization script
I have released version 2.2 of my PowerCLI ImageBuilder based ESXi customization script ESXi-Customizer-PS! This version integrates Online Depots as a source for customization packages.
Here are the changes in detail and some examples of how to make use of them.
An analysis of the vSphere 5.5 VMware Tools for Windows installation
It is a good practice to update VMware Tools on your VMs after you have updated your vSphere environment to a new major or minor release. VMware tries to make this very easy by providing means to automate the VMware Tools installation/update (through the vSphere legacy and Web Client, and PowerCLI), but in a lot of environments there is a requirement to take complete control over software provisioning on Windows servers and/or the need to customize the VMware Tools installation and remove unwanted features that are installed by default.
For this purpose you will want to take a careful look at the VMware Tools MSI package. I did this with the latest Tools version of vSphere 5.5, and here are my findings.
How to upgrade your VMs' virtual hardware to version 9 with ESXi 5.5
After upgrading your ESXi hosts to 5.5 the "Upgrade Virtual Hardware" function of the legacy vSphere Client will update the virtual hardware of a VM to version 10, although the legacy client is not able to edit the properties of version 10 VMs (see my earlier post about How to update to ESXi 5.5 ...). Only the Web Client is able to do this with version 10 VMs, and that requires vCenter. If you do not have vCenter available or do not feel comfortable with the Web Client for other reasons then you want to avoid upgrading virtual hardware to version 10. But how can you upgrade to only version 9?
Building a self-configuring nested ESXi host vApp
In my last post I presented a walk-through about how to create a nested ESXi host and make an OVF template of it. After deploying this template a manual step remained: Configuring a hostname and the IP address configuration. If you deploy a virtual appliance that was produced by VMware (e.g. the vCenter Log Insight appliance) then you are often presented with the choice to configure the networking of the VM in the OVF deployment wizard. How is this done, and can we use the same method to customize the nested ESXi vApp? Yes, we can - and here is how.
How to provision nested ESXi hosts on free ESXi
The free ESXi license though has some limitations that makes the job harder:
- It is not manageable via vCenter, and thus ...
- ... only via the legacy vSphere client, not the Web Client, which in turn means ...
- ... you are limited to virtual hardware version 9 and must not use version 10.
- And - that's most annoying - you also cannot use PowerCLI to create or configure VMs (because the relevant APIs are restricted with the free license)!
You can overcome these limitations by using free and fully featured evaluation licenses, but these are time limited. So, I thought it's time to write a little Nested ESXi Guide for free license users (It also includes some nice tips for paid-license users though!) ...
Annoying IP dual stack issue with VMware ESXi
While testing direct host access to the new V-Front Online Depot (via esxcli) I stumbled over an annoying issue that I was finally able to resolve, but it was hard to find the root cause ... So I want to share my findings here in the hope to make life easier for all the others that will - very likely - stumble over the same issue.
After opening the firewall for outgoing http(s) requests using
esxcli network firewall ruleset set -e true -r httpClient
esxcli software sources vib list -d http://vibsdepot.v-front.de
esxcli software vib install -d http://vibsdepot.v-front.de -n package-name
If you experience the issue then these commands will work as expected, but they will take a very very long time (>10 minutes!) to execute and return. Of course there are other and more obvious root causes for slow network access: a slow or saturated up-link to the Internet, improper NIC speed negotiation settings etc.
But in my case it had something to do with IPv6 ...
A new version of ESXi-Customizer - and why it is the final one
- The included mkisofs tool (that is used to build the customized ISO file) was replaced by a newer version that does no longer produce ISO files with corrupt headers when used with the UEFI boot option. In earlier versions I made the UEFI boot optional (and disabled by default), because it would create invalid ISO files that can not be opened by ESXi-Customizer itself or other applications. This is now fixed, and UEFI boot is no longer an option, but always enabled.
- Added detection of Windows 8/8.1 and Server 2012 (R2) to the Windows version check. These are all supported for customizing ESXi 5.x, but not for ESXi 4.1.
- Added a check for the parentheses characters "(" and ")" in package file names (These would make the script fail in earlier versions).
- Moved the version update check script to my new site http://vibsdepot.v-front.de. This is also the new place to download ESXi-Customizer, because Google does no longer want us to provide downloads at Google Code.