RDM mapping of local SATA storage for ESXi
This post has been sat in my WordPress Drafts folder for
sometime since I no longer use local storage this way. I decided to post
it however as (a) it’s a good learning curve for ESXi work and (b)
others may have more luck that me.
I recently acquired three 1TB drives and decided to do something
about my lack of storage at home. Always trying to make best use of
existing kit (and save money) I decided to stick the drives in to my HP
ML110 and try something in a VM instead of doing the sensible thing of
lobbing them in to a dedicated NAS box.
After wasting a few hours I realised that the onboard SATA RAID
controller of the ML110 just can’t do RAID5 and to make matters worse,
when I gave up and created a RAID1 array with a hot spare, vSphere 4.1
didn’t recognise the array and instead saw the drives as 3 individual
drives. I saw this as a chance to try out the WAFL-alike ZFS file
system. FreeNAS had been my NAS of choice recently so I chose to try ZFS
I point blank refused to create 3 1TB VMDK’s (one of each of the
three drives) so I set about figuring out how to create Raw Device
Mappings (RDMs) of the local SATA drives. There were a couple of posts
on the net that got me a little closer, but no guide/article had the
whole thing down, so that’s my aim with this blog post.
Once you had your drives installed, SSH to your ESXi box (now even
easier in vSphere 4.1) and go to the /dev/disks directory. There, if you
perform a ls -l, you’ll see your drives listed:
Ignore the instances of your drives which show them as VM stores (vm1.*****). We want to look at the raw devices.
Now move to the /vmfs/volumes folder. Here you can see your existing local datastore(s). If, like me, you had a solitary hard-drive, you’ll just see localdisk01 or whatever you chose to name the local datastore:
Now we are going to use the vmkfstools utility to create our RDM’s. Remember that a RDM is just another VMDK, but instead of the VMDK pointing to a xxx-flat.vmdk file
(which is the actual virtual hard disk), the VMDK points to our
physical device. Being as we still need to create this VMDK file we need
to save it somewhere. Since we just have the one local datastore, we
are going to create the RDM VMDK files in it’s root.
The following command creates the RDM VMDK for us:
vmkfstools -z /vmfs/devices/disks/<name of RAW device from Step 1> <location to store VMDK>/<RDM name>.vmdk
In my personal example below, I am creating an RDM called rdm_WD2DWCAVU0477582.vmdk and it is being stored in the location /vmfs/volumes/localdisk01/
I chose the name of the VMDK to match the name of the serial number of
the physical drive (and what is shown in Step 1) to help with
troubleshooting in the future when I get an inevitable drive failure).
You can call your RDM’s whatever you wish.
The name of the RAW device
in my example) you will have noted from Step 1 when you listed all
local devices attached to your ESXi host. This is why the tech Gods
created Copy n Paste! You will want to copy the full device name as
shown in Step 1 in to the vmkfstools command.
Once you have repeated the steps for all of your local SATA drives,
you can navigate to where you created the RDM’s (in my case
/vmfs/volumes/localdisk01) and perform an ls -l *.vmdk to see the new
VMDK’s you have created:
Don’t panic – the xxx-rdmp.vmdk files will reflect
the size of the RAW devices they are mapping to, but rest assured it
will be taking no more space than a few bytes on your local disk!
You can now add your RDM’s to an existing VM. vSphere doesn’t
recognise this as a true RDM (to a SAN) so you just browse the local
disk datastore for the VMDK files that we created.
Edit the properties of an existing VM and click Add…
Select Use an existing virtual disk and click Next >
Click Browse. You now need to navigate your local datastore and select the VMDK’s that we created in Step 3).
Once complete you will be shown a confirmation window. Repeat Steps 5 through 7 to add additional RDM’s to your VM.
You should now see your new Hard Disk’s in your VM and vSphere will correctly identify them as Mapped Raw LUN.
NOTE: One thing I forgot to show in the screen
shots, is that you should create your RDM’s on a new SCSI controller!
You do this by simply selecting a new SCSI ID starting with 1:x instead
of 0:x. Existing VMDK’s should be on SCSI Controller 0. Your RDM’s
should be on SCSI controller 1. Although my screenshot shows 0:3 this
should read 1:3.
You can now save your VM configuration. Your VM will now access the
RAW SATA drives and be able to use things like SMART to monitor its
See below; I am adding my three 1TB drive to FreeNAS to create a new ZFS pool.
Stay tuned for an upcoming blog post on FreeNAS and NexentaStor which may or may not put you off ZFS [in a VM] altogether