Protect your ESXi hosts against Heartbleed attacks

Unless you live behind a rock you have most probably already heard about a vulnerability in the OpenSSL software that was dubbed Heartbleed. A lot of VMware products make use of OpenSSL to enable SSL secured web access, and a lot of them use versions that are vulnerable.

VMware has reacted to this by publishing a list of vulnerable products and announced appropriate patches to be released very soon. ESXi is also affected. A general recommendation of VMware is to not expose the management interface of your hosts directly to the Internet, but there are situations where you cannot really avoid that or just do not want to sacrifice the convenience to have your hosts manageable from (theoretically) everywhere.

If you have ESXi hosts directly connected to the Internet and can not or do not want to wait for a patch to be released (or just do not want to spoil your easter weekend with patching hosts) then you should think about other means to mitigate the issue.

You can use the builtin ESXi firewall not only to close down access to the SSL port completely (which I would not recommend), but you can also limit the range of IP addresses that are allowed to connect to the port:


In the vSphere Client go to Host Configuration / Security Profile and click on the Properties... link next to the Firewall section on the right side. This will open the dialog shown above. Select the rule for the vSphere Client here (which includes opening the https port 443 for incoming connections) and click on the Firewall... button below.

Now you can select Only allow connections from the following networks and enter the IP subnets that should be allowed to do the SSL connects. All other IP ranges will then be blocked. Please note: This limits the ability to manage your host (not only via the vSphere Client, but also through e.g. PowerCLI) to the computers that are part of the allowed IP ranges!

If you prefer to do this through the command line then you can use the following esxcli commands:
# Disable the "Allowed-All" flag
esxcli network firewall ruleset set -r vSphereClient -a false
# Define allowed IP ranges(s)
esxcli network firewall ruleset allowedip add -r vSphereClient -i 192.168.33.0/24
Please be careful when restricting the management access, because you might lock yourself out when doing it wrong! It is a good idea to temporarily enable ssh access to the host before you do this configuration. If you then happen to lock yourself out then you can still access the host through ssh and revert the change by running

   esxcli network firewall ruleset set -r vSphereClient -a true

Update (2014-04-17):

alpacapowered pointed out in the comments of this post that it's not only the management https port 443 that exposes the OpenSSL vulnerability on ESXi, but also the CIM Secure Server port 5989, the NFC port 902 and the HA agent port 8182 (only if HA is enabled).

When limiting access to these ports keep in mind that the vCenter server (if present) needs to access each host on the CIM Secure Server port, and that all hosts (if in an HA enabled cluster) need to be able to access each other on port 8182. Access to port 902 is also needed between all hosts in a cluster and from the vCenter server to hosts (see VMware KB1012382 for more information about required network ports).


This post first appeared on the VMware Front Experience Blog and was written by Andreas Peetz. Follow him on Twitter to keep up to date with what he posts.



3 comments:

  1. Good point to use the local ESXi Firewall to filter certain sources. However, it should be noted that besides port 443 and 902 (using a STARTTLS-alike functionality ), ESXi is listening on other ports for TLS traffic as well:
    TCP/5989 sfcb-HTTPS-Daemon (CIM hardware management)
    TCP/8182 FDM/HA traffic

    There are quite a few other processes loading the openssl libs, most of them not listening on their own socket though:
    lsof | grep -E "libssl|libcrypto|libeay|ssleay"

    ReplyDelete
    Replies
    1. Thanks for your comment, and you are right: At least the CIM Secure Server port 5989 is also vulnerable. Port 902 does not seem to be vulnerable, and I haven't checked any of the other ports ... I will update the post later.

      Delete
  2. 902 is vulnerable as well, though you can't test it with the usual scripts because there is an additional frame from the server before the actual TLS handshake. Just telnet to port 902 and you'll see it.

    You can modify the perl script below to cope with this just by adding "_readframe($cl);" on line 147 (after establishing the socket).
    https://github.com/noxxi/p5-scripts/blob/master/check-ssl-heartbleed.pl
    With this addition the script correctly reports ESXi 5.5 hosts on port 902 as vulnerable too.

    Port 8182 runs a normal TLS handshake just like 443 (as long as HA is enabled).

    ReplyDelete

***** All comments will be moderated! *****
- Please post only comments or questions that are related to this post's contents!
- Advertising and link spamming will not be tolerated!