August 22

How to Add a Hard Disk to an ESXi Virtual Machine

VMware ESXI Hard Drive

In an effort to consolidate my homelab, I recently deployed an ESXi host to my main server. There’s been a bit of a learning curve but overall it’s been a pleasant experience. In this tutorial, I will show you how to add a new hard disk to an ESXi virtual machine. (If you’re looking for hard drives, I usually use WD Reds that I shuck from a WD Easystore or Elements drive. They’re high capacity, well-priced, and reliable.)

Now, theoretically, this should be an easy task where you can just edit the VM and click the Add hard disk > New standard hard disk, specify the hard disk location and be done. Unfortunately, though, there’s a bug in the ESXi host web client and so instead this will be a tutorial on adding an additional hard disk to an ESXi guest via the command line (esxcli).

Starting with the bug in the ESXi host web client, if you add a new hard disk to a VM, and the primary datastore where the VM is located is full, you’ll be greeted with the following message and the save button will gray out:

The disk size specified is greater than the amount available in the datastore. You cannot overcommit disk space when using a thick provisioned disk. You must increase the datastore capacity, or reduce the disk size before proceeding.
ESXi error message

The disk size specified is greater than the amount available in the datastore. You cannot overcommit disk space when using a thick provisioned disk. You must increase the datastore capacity, or reduce the disk size before proceeding.

ESXi host web client bug: Save button is always disabled even despite changing location of hard disk. The save button being disabled will persist even after removing the offending "New Hard disk".
ESXi host web client bug: Save button is always disabled even despite changing location of hard disk. The save button being disabled will persist even after removing the offending “New Hard disk”.

Now, that’s all fine and good, but the problem is, the “Save” button stays that way even when you choose a datastore that does have space. Heck, the save button even remains disabled when you remove the new hard disk! It appears that the web client only evaluates once, cripples the save button, and never re-evaluates the settings ever again.

Another thing I’ve learned along the way is that vSphere != ESXi host web client and unfortunately almost all of VMware’s documentation refers explicitly to vSphere, not the ESXi web client, so in this tutorial I’ll be showing you how to use the command line if you don’t have vSphere set up.

Creating the Virtual Disk via the Command Line (esxcli and vmkfstools):

1. Enable SSH on the ESXi host by right-clicking on the host in the web client > Services > Enable Secure Shell:

Enabling SSH service in ESXi.
Enabling SSH service in ESXi

2. SSH into ESXi host. If you’re using Ubuntu, open a terminal and type the following:

ssh root@<insert IP address of ESXi host here>

3. Find the location of your datastore with the following command:

esxcli storage filesystem list
esxcli storage filesystem list
Note the Mount Point (i.e. the location of the volume)

4. Change directory into that location with the following command:

cd <copy Mount Point from above here>

5. Create your hard disk with the following command:

vmkfstools --createvirtualdisk 256G --diskformat thin <nameYourDriveHere>.vmdk

Note that the above creates a 256 GB virtual drive. If you want a different size, just change it.

vmkfstools --createvirtualdisk 256G --diskformat thin <nameYourDriveHere>.vmdk
vmkfstools –createvirtualdisk 256G –diskformat thin .vmdk

Adding the New Virtual Disk to Your VM:

At this point, we’ve created our virtual disk, but it currently only exists in isolation in the datastore. It isn’t connected to any VM so we need to add it to a VM.

6. Add the hard disk to your VM:

Here is the “choose your own adventure” part of the guide. I originally assumed that the majority of readers will want to add the virtual disk to their VM using the ESXi host’s GUI. That’s fine since this part of the ESXi Embedded Host Client isn’t broken. If that’s you, use the steps outlined in 6a.

However, based on reader feedback, I have learned that some readers are interested in also adding the new hard disk to the VM via the command line, so that both disk creation and hot-adding the disk to the VM are 100% command line. If that’s you, refer to 6b.

6a. Add the hard disk to your VM via the ESXi Embedded Host Client (i.e. the ESXi web client GUI):

We’re done with the command line now. Go back to your ESXi web client and edit your VM. Now click “Add a hard disk” but this time instead of a new hard disk, we’re going to select “Existing hard disk” and navigate to the .vmdk you created in step 5:

Add existing (new) hard disk in ESXi web client
Add existing (new) hard disk in ESXi web client

Defaults are fine. Don’t panic if your new hard disk shows 0 GB. The virtual disk just hasn’t been formatted yet. Save and boot up your VM. As you can see I’m using a Windows VM here.

6b. Hot Adding the Disk to the VM via the Command Line:

To hot-add our new virtual disk to the VM via the command line, we first need to find the ID of our VM. To do so, still SSH’d into the ESXi host, we issue the following command:

vim-cmd vmsvc/getallvms
Result of  vim-cmd vmsvc/getallvms showing Vmid of VMs
Result of vim-cmd vmsvc/getallvms showing Vmid of VMs

I want to add this to my Windows VM, so I want Vmid 1. Now to hot-add our new virtual disk, we simply issue the following command:

vim-cmd vmsvc/device.diskaddexisting <vmid> </vmfs/volumes/pathToDisk.vmdk> 0 1

Replace <vmid> and </vmfs/volumes/pathToDisk.vmdk> with the Vmid and path to your new virtual disk, respectively. The path has to be the full path to the disk you created in steps 3/4/5 above. The 0 at the end refers to your SCSI controller (typically 0) and the 1 refers to your SCSI target number (the 0 slot is occupied by the primary virtual disk your VM is using, so 1 is just the next available target. If you already have a second hard disk on your VM, you would increment this number and use 2 instead. You get the idea). In my case, the command would look something like this:

vim-cmd vmsvc/device.diskaddexisting 1 /vmfs/volumes/5d5d560e-962037c1-16f8-fcaa142fc77d/Windows10Storage/win10aux.vmdk 0 1

7. Add the Hard Drive to the OS:

When you boot up your VM and navigate to “This PC” in File Explorer, you still won’t see the hard drive. Again, don’t panic, this is just because we haven’t added the drive in Windows yet.

Open up Disk Management (you can search for it in Cortana) and you should see your new drive, albeit unformatted. Right-click on the hard drive and select “New Simple Volume”:

Disk Management - Adding unallocated disk to Windows
Disk Management – Adding unallocated disk to Windows

Follow the wizard and when you go back to “This PC”, voila, your new hard disk appears!

New disk installed on VM
New disk installed in VM
June 19

Preparing Your Server for Vacation – The Unattended Server Access Checklist

Unattended Server Checklist showing steps of 1) Check VPN Connection 2) Reverse SSH Tunnel Backup 3) VNC available on backup devices 4) Test automatic server shutdown after loss of utility power 5) Test Server WOL form one of backup devices.

I am about to embark on a much-needed vacation this Friday. I travel relatively frequently for work and something that has always been a stressor in the back of my mind was leaving my homelab unattended. With my luck, as soon as I lock my door and get in the car, the server will wait for that exact minute to go down. The way I have managed this problem is by creating a checklist to ensure that, no matter where I am, I can access my network. Stuff goes wrong, that’s the nature of any real work, but I can fix anything so long as I have access. Best of all, this checklist can save you from having to call your wife and tell her to push a button on your computer!

In this post, I give you the checklist I use for unattended server access. In future posts, I will create how-to’s for implementing these features.

1) Check VPN Connection:

VPNing into my DMZ is my primary method of access for resources on my network when I am not at home. The advantages of being able to connect to my DMZ via VPN is enormous since it allows me to act just like another client on the network.

2) Check Reverse SSH Tunnel Backup (x2):

Let’s say my VPN server has gone down (in either a controlled or uncontrolled fashion); in that situation, my primary means of unattended access has gone with it- I’d be locked out of my own network. The Army has a phrase for this: “One is none, two is one, three is better.” Since we have currently only discussed one way of accessing the network remotely, applying the military’s logic, I have no way into my network to fix it; therefore, I need a backup. This is where a reverse SSH tunnel comes to the rescue. You can either create one manually or use a 3rd party service.

I use Remote.it’s connectd. With it, I can SSH into another device on my network (keeping with the idea of redundancy for critical systems, this device should not be the same one running your VPN server) and do what I need to do from there. (See WOL below).

3) VNC Server Available on Backup Devices:

My router uses a GUI, so being able to spin up a VNC server on demand from the SSH connection above is necessary.

4) Test Automatic Server Shutdown When Running on Backup Power (UPS):

I have an APC UPS directly connected to my server over USB. In the event of a power failure, the UPS tells my server that we are no longer on utility power and allows it to gracefully shutdown. Testing is essential here since it not only checks to make sure that this feature is still functional, but it also serves as a check on the UPS’s battery life.

5) Test Server Wake-on-LAN (WOL) from Backup Device:

Unlike my other network devices (router, hardware firewall, the above backup devices, etc.), my server is set up to NOT automatically restart upon the restoration of utility power. Since power outages often can have brief periods of power restoration, I don’t want it to continuously startup to then only lose power again. Therefore, after a graceful shutdown, I want the server to stay down until I bring it back up. I accomplish this via a WOL message (“magic packet”) from one of the backup devices to the server.

Category: Servers | LEAVE A COMMENT