Creating a Datastore on your ESXi USB Boot Drive
A quick note on this, I suspect that this would have been less of a pain if we’d have opted for using the internal micro-SD slot in the server rather than stuffing a 16GB USB3.0 stick into one of the internal USB ports, so you might want to explore that option before going down this path, I chose the USB stick because theoretically at least a decent USB3.0 stick should be faster than an equivalent micro-SD.
Intro
When you install ESXi onto a USB stick it doesn’t create a “datastore” partition like it does when you install to a “real” hard disk, this is undoubtedly a deliberate design choice on VMWare’s part, because in the main running VMs from a USB stick is a “bad idea(tm)”, however in our particular application it makes sense YMMV.
There are other articles elsewhere that go into similar territory but they are concerned with using a different USB device as a datastore, and their process needs modification to be applicable in this particular context, for the details of those articles check the References section at the bottom of the post.
Another quick note before we get on with it, this tutorial is specific to ESXi 6.7, in older versions (not sure which) it is also necessary to disable the usbarbitrator service which means you can’t attach host-connected USB devices to VMs so bear that in mind.
The Plan
I’ve been burnt by hardware RAID controllers in the past, a particularly memorable instance was losing a 30TB array which was configured as RAID6 + Hot Spare. One disk died in the array, the “Hot” Spare did NOT come online, and then a second disk died taking the rest of the array with it, meanwhile the “Hot” Spare was happily sitting there doing nothing whatsoever.
So basically I want to take the RAID controller out of the equation and manage my storage directly, (a sidenote here, the newer HP SmartArray controllers have a really neat feature whereby if you remove the Logical Drive config from them they revert to “dumb” HBA mode and present disks directly to the OS, which is nice because it means I don’t have to configure 30 single-disk RAID0 volumes and pass THOSE through), probably by way of a RAID-Z2 pool.
In order to achieve that I need a small VM built on my boot storage, and for that I need a VMFS volume on my boot storage.
Let’s Do It
The first thing you’ll need to do is enable SSH into your ESXi host, there’s info elsewhere as to how to do this so I won’t cover it here, once that’s set up SSH into your host as root.
Once you’re in you’ll need to ID the device node that your controller is on, chances are it will be the only device with partitions 1, 5, 6, 7 and 8 defined, e.g.
[root@esxi:~] ls /dev/disks/
...
/dev/disks/vml.010...520
/dev/disks/vml.010...520:1
/dev/disks/vml.010...520:5
/dev/disks/vml.010...520:6
/dev/disks/vml.010...520:7
/dev/disks/vml.010...520:8
...ESXi ships with some basic tools for managing partitions, you need to dump your partition table;
[root@esxi:~] partedUtil getptbl /dev/disks/vml.010...520
gpt
1882 255 63 30244864
1 64 8191 C12A7328F81F11D2BA4B00A0C93EC93B systemPartition 128
5 8224 520191 EBD0A0A2B9E5443387C068B6B72699C7 linuxNative 0
6 520224 1032191 EBD0A0A2B9E5443387C068B6B72699C7 linuxNative 0
7 1032224 1257471 9D27538040AD11DBBF97000C2911D1B8 vmkDiagnostic 0
8 1257504 1843199 EBD0A0A2B9E5443387C068B6B72699C7 linuxNative 0If you’ve got the right volume it should look pretty much like the above (at least the actual partition table entries).
The second line is some particulars about your disk, the important number is the last one (30244864 in this case) which is the last sector of the disk. The other figure you’re particularly interested in is the third column in the last line (1843199 in this case) this value +1 will be your start sector (though you may want to adjust for alignment).
I recommend taking a copy of this data and sticking it somewhere safe, if you screw up you’re going to need it.
Now we’re going to create a new partition in the free space on the disk.
[root@esxi:~] partedUtil add /dev/disks/vml.010...520 gpt "2 <start sector> <last sector> AA31E02A400F11DB9590000C2911D1B8 0"
gpt
1882 255 63 30244864
1 64 8191 C12A7328F81F11D2BA4B00A0C93EC93B 128
5 8224 520191 EBD0A0A2B9E5443387C068B6B72699C7 0
6 520224 1032191 EBD0A0A2B9E5443387C068B6B72699C7 0
7 1032224 1257471 9D27538040AD11DBBF97000C2911D1B8 0
8 1257504 1843199 EBD0A0A2B9E5443387C068B6B72699C7 0
2 1843200 30244830 AA31E02A400F11DB9590000C2911D1B8 0Now we need to turn that into a VMFS volume, we do that with vmkfstools (note that we’re using
[root@esxi:~] vmkfstools -C vmfs6 -S BootDS /dev/disks/vml.010...520:2
create fs deviceName:'/dev/disks/vml.010...520:2', fsShortName:'vmfs6', fsName:'BootDS'
deviceFullPath:/dev/disks/mpx.vmhba32:C0:T0:L0:2 deviceFile:mpx.vmhba32:C0:T0:L0:2
ATS on device /dev/disks/mpx.vmhba32:C0:T0:L0:2: not supported
.
Checking if remote hosts are using this device as a valid file system. This may take a few seconds...
Creating vmfs6 file system on "mpx.vmhba32:C0:T0:L0:2" with blockSize 1048576, unmapGranularity 1048576, unmapPriority default and volume label "BootDS".
Successfully created new volume: 5d1ec3fb-47e91acc-0861-08f1eaebba32Now if all’s well you should have a new Datastore appearing in the ESX Web UI.
ESXi Storage
Troubleshooting
I’m not going to go into a great deal of detail here, there were a few issues I ran into but I don’t recall the specifics.
One issue was not having set the partition type GUID correctly, I copied one of the others not realising it was necessary to use the AA3…1B8 GUID, vmkfstools will throw its toys if you do that, change the partition type and it should work.
The other issue was an initial lack of understanding of what the partedUtil tool was doing which lead to trashing the partition table, if you do this then the recovery process is pretty simple (if a bit fiddly);
- Take the backup you made of the details from ‘partedUtil getptbl’ (you did take a backup… RIGHT?).
- Strip out the fifth column (the one with human-readable text in it) which will yield something like this;
1 64 8191 C12A7328F81F11D2BA4B00A0C93EC93B 128
5 8224 520191 EBD0A0A2B9E5443387C068B6B72699C7 0
6 520224 1032191 EBD0A0A2B9E5443387C068B6B72699C7 0
7 1032224 1257471 9D27538040AD11DBBF97000C2911D1B8 0
8 1257504 1843199 EBD0A0A2B9E5443387C068B6B72699C7 0- Wrap each line in double quotes;
"1 64 8191 C12A7328F81F11D2BA4B00A0C93EC93B 128"
"5 8224 520191 EBD0A0A2B9E5443387C068B6B72699C7 0"
"6 520224 1032191 EBD0A0A2B9E5443387C068B6B72699C7 0"
"7 1032224 1257471 9D27538040AD11DBBF97000C2911D1B8 0"
"8 1257504 1843199 EBD0A0A2B9E5443387C068B6B72699C7 0" - Now turn them into a space-delimited list and slap ‘partedUtil setptbl
gpt ' in front of it and run that;
[root@esxi:~] partedUtil setptbl <device> gpt "1 64 8191 C12A7328F81F11D2BA4B00A0C93EC93B 128" "5 8224 520191 EBD0A0A2B9E5443387C068B6B72699C7 0" "6 520224 1032191 EBD0A0A2B9E5443387C068B6B72699C7 0" "7 1032224 1257471 9D27538040AD11DBBF97000C2911D1B8 0" "8 1257504 1843199 EBD0A0A2B9E5443387C068B6B72699C7 0"
gpt
1882 255 63 30244864
1 64 8191 C12A7328F81F11D2BA4B00A0C93EC93B systemPartition 128
5 8224 520191 EBD0A0A2B9E5443387C068B6B72699C7 linuxNative 0
6 520224 1032191 EBD0A0A2B9E5443387C068B6B72699C7 linuxNative 0
7 1032224 1257471 9D27538040AD11DBBF97000C2911D1B8 vmkDiagnostic 0
8 1257504 1843199 EBD0A0A2B9E5443387C068B6B72699C7 linuxNative 0