Mysterious port 903

I recently investigated what network ports are used by ESXi 4.1, because I had to compile the firewall requirements for a new deployment of ESXi hosts in a DMZ. There is a detailed source available for that in the VMware KB:

• KB1012382: TCP and UDP Ports required to access vCenter Server, ESX hosts, and other network components

And there are numerous other sources available (even nice diagrams like this one). In most cases it is obvious that their authors referred to and relied on the above mentioned official VMware KB source.

I'm usually not paranoid, but maybe I talked too much with the IT security guys (who tend to be extremely paranoid ;-)). Anyway, following the rule "Trust no one" I started looking at the network ports that are really used in our current production environment and compared them to the list in the KB article.

So I stumbled over port 903... According to the list both the vCenter server and any vSphere Client connect to an ESXi 4.1 host on that port for accessing the VM remote console. However, when I checked the network connections on the vCenter server and my Windows Desktop running the vSphere Client (with "netstat -an") I was not able to see any connection to an ESXi host's port 903, even when I opened multiple VM consoles. Instead it was obvious that port 902 is used for console connections.

This made me really curious, so I logged on to an ESXi host (in Tech support mode) and checked the open network connections there. In ESXi you use the command "esxcli network connection list" for that which produces an output that is quite similar to the netstat output (With classic ESX the netstat command is still available in the service console).

This command will also list all ports that are opened in LISTEN mode, that means there is some process waiting for connections on that port. But there was no listening process for port 903, and that means that no one and nothing would be able to connect to that port!

I opened a support request with VMware asking for clarification on the mysterious port 903 and was very curious about their answer. Of course, they quoted their own KB article first, insisted on that the port was actually used for this and that, but finally - after raising the issue to engineering - they admitted that "ESXi does not use port 903".

Also a request was made to update the KB article accordingly. So, when you read this it might already have been corrected to not include port 903 anymore, but the numerous third party documents based on KB1012382 will take some more time to be updated ...

Bottom line: Information is good. Correct information is better. Try to verify it if it is really important to you.