Remote access for ESXi local user account 'root' has been locked for 120 seconds after xxx failed login attempts.At the same time he could no longer log in to the vSphere client although he was using the correct password. I immediately thought: Wow, this is new in 6.0, never heard of this before! What the heck was going on here?
Well, I just RTFM (read the fine manual) and quickly found out what was going on (yes, VMware's public product documentation is pretty good, highly recommended!).
In the section ESXi Passwords, ESXi Pass Phrases, and Account Lockout of the ESXi and vCenter Server 6.0 Documentation you can find the following information:
ESXi Account Lockout Behavior
Starting with vSphere 6.0, account locking is supported for access through SSH and through the vSphere Web Services SDK. The Direct Console Interface (DCUI) and the ESXi Shell do not support account lockout. By default, a maximum of ten failed attempts is allowed before the account is locked. The account is unlocked after two minutes by default.
You can configure the login behavior with the following advanced options:
- Security.AccountLockFailures. Maximum number of failed login attempts before a user's account is locked. Zero disables account locking.
- Security.AccountUnlockTime. Number of seconds that a user is locked out.
And indeed this is new in version 6.0!
In this specific case it turned out that the host was directly connected to the Internet and had shell access via ssh permanently enabled. Of course this is an invitation to hackers all over the world, and they soon and repeatedly tried to break into the ESXi host using brute force. The new account lockout feature helps to mitigate this kind of attack, but it will also prevents legitimate access to the system like in this case. It is important to understand that not only ssh access, but also access to the host via the vSphere client or API calls is prevented when the root account is locked out!
This case reminds us of some basic security rules and best practices for ESXi that are mandatory for production use, but should also be honored for home and lab use:
1. Connect the host management interface to a secured internal network, but not to the public Internet!
2. Enable ssh access only temporarily for troubleshooting, but do not keep it permanently enabled!
And I like to add the following recommendations, especially for cases where you have no other choice than connecting the host directly to the Internet:
3. Create a timeout for ESXi shell availability and for idle ESXi shell sessions. This way the ssh access will automatically be disabled again after a configurable timeout, and forgotten idle sessions will automatically be terminated.
4. If you frequently enable ssh access to your hosts for whatever reason then consider using key based ssh authentication and disable password log in. For further information see KB1002866 and the instructions on Uploading an SSH Key Using a vifs Command.
5. Use the ESXi builtin firewall to limit access for at least ssh (port 22) and the vSphere client (port 443) to known trusted IP addresses. You can read my earlier blog post about Protecting your ESXi hosts against Heartbleed attacks for a practical scenario and examples.
I hope you will find this information useful - stay safe!
This post first appeared on the VMware Front Experience Blog and was written by Andreas Peetz. Follow him on Twitter to keep up to date with what he posts.