Map Shared RDMs using PowerCLI

Adding shared RDMs using GUI is easy, but not something you want to do if there are hundreds of shared RDMs that need to be mapped to VMs in a given stipulated time. One such case is a DR exercise, where you need to add RDMs from replicated SAN to various SQL cluster VMs. Here is one of the ways to accomplish this task using PowerShell scripts and a well-organized process.

Normally, we do following to connect RDMs to a pair of cluster nodes (SQL VMs).

  1. Present LUNs to ESXi hosts.
  2. Mark the RDM LUNs as Perennially Reserved.
  3. Add the LUN as RDM to the first node (VM).
  4. Map RDM file to second node.

In our case, we will follow the similar process step by step, but we will offload the execution to a script for each step. The scripts will take input from a CSV file and will output the results to a CSV file. The output CSV file will then become the input for the next script. Before moving to next step we can review the output. The reason to do it this way is to keep it simple and to make sure we have control over the process. We definitely don’t want to mess up LUNs between different clusters. The process below is for one guest cluster at a time.

Note:

  • All scripts and CSV files are prefixed with “<step no>_” like 1_ , 2_ to clearly identify the sequence of use.
  • All CSV files are prefixed with “in_” or “out_”, depending on whether it’s an input file or output file. For example, 1_in_lunids_sqlnode1_sqlnode2.csv
  • Parameters like ESXi host, vCenter host, username, password and input/output file paths are part of the script itself and needs to be changed to suit your environment.
  • The actual VMs and SCSI controllers in them need to be pre-staged. That is not part of the script. You can add one small VMDK to pre-stage SCSI controller. These VMDKs can be removed once the actual RDMs are mapped.

Step 1: Identify LUN mapping and LUNs on ESXi host

Input File: 1_in_lunids_sqlnode1_sqlnode2

In a CSV file, create a list of LUNs participating in this guest cluster as following. “controllerid” is the name of the controller on the VM. “scsiid” is the SCSI ID that we want to assign to that RDM.

1_in_lunids_sqlnode1_sqlnode2

1_in_lunids_sqlnode1_sqlnode2

Script: 1_Get-LUNDeviceInfo.ps1

This script will connect to the vCenter and ESXi host specified to get details of all the LUNs mentioned in CSV file. The details will be saved to Output File: 1_out_deviceids_sqlnode1_sqlnode2.csv. Here you should cross check if all LUNs are properly exposed to ESXi host or not.

Output File: 1_out_deviceids_sqlnode1_sqlnode2

Output File: 1_out_deviceids_sqlnode1_sqlnode2

# Script: 1_Get-LUNDeviceInfo.ps1

#<#$
$vcip = '192.168.1.36'
$vcuser = '[email protected]'
$vcpassword = 'somepassword'
$esxi = 'esxi38'
$in_csvpath = 'C:\RDMs\1_in_lunids_sqlnode1_sqlnode2.csv'
$out_csvpath = 'C:\RDMs\1_out_deviceids_sqlnode1_sqlnode2.csv'
#>

$vc = Connect-VIServer $vcip -User $vcuser -Password $vcpassword

if($vc.IsConnected){
 "Connected to vCenter: " + $vc.Name
 }else{
 "Unable to connect to vCenter: " + $vcip
 exit
}


$esxiview = Get-VMHost $esxi | Get-View
$LUNsbyKey = $esxiview.Config.StorageDevice.ScsiTopology | 
 ForEach {$_.Adapter} | ForEach {$_.Target} | ForEach {$_.Lun}

$LUNsbyCN = Get-VMHost $esxi | Get-SCSILun

$LunObjects = @()

$lunids = Import-Csv -Path $in_csvpath

# Loop each LUN by Canonical name and get the LUN ID

foreach ($lun in $lunids)
{
 $MatchingLuns = $LUNsbyKey | Where {$_.Lun -eq $lun.lunid}
 
 foreach ($foundlun in $MatchingLuns)
 {
 $object = New-Object -TypeName PsCustomObject
 $object | Add-Member -Name 'LUNID' -MemberType Noteproperty -Value $foundlun.Lun
 $object | Add-Member -Name 'controllerid' -MemberType Noteproperty -Value $lun.controllerid
 $object | Add-Member -Name 'scsiid' -MemberType Noteproperty -Value $lun.scsiid

 $MatchingNames = $LUNsbyCN | Where {$_.Key -eq $foundlun.ScsiLun}

 $object | Add-Member -Name 'CanonicalName' -MemberType Noteproperty -Value ($MatchingNames | Select -First 1).CanonicalName
 $object | Add-Member -Name 'ConsoleName' -MemberType Noteproperty -Value ($MatchingNames | Select -First 1).ConsoleDeviceName
 $object | Add-Member -Name 'LUNType' -MemberType Noteproperty -Value ($MatchingNames | Select -First 1).luntype
 $object | Add-Member -Name 'CapacityGB' -MemberType Noteproperty -Value ($MatchingNames | Select -First 1).capacitygb

 $LunObjects += $object
 }
}

$lunobjects | export-csv -Path $out_csvpath -Delimiter "," -NoTypeInformation

Step 2: Configure Perennial Reservation and Add RDM to First Node

As you can see in step 1 output file, it has picked up some local devices as well for LUN ID 0. We will clean up these lines and save it as the input file for step 2.

Input File: 2_in_rdmdeviceids_sqlnode1_sqlnode2

Input File: 2_in_rdmdeviceids_sqlnode1_sqlnode2

Input File: 2_in_rdmdeviceids_sqlnode1_sqlnode2

Script: 2_PR-RDMFromCSV.ps1

This script will configure perennial reservations for each LUN in the input file, and on each ESXi host listed in $esxis variable.

# Script: 2_PR-RDMFromCSV.ps1
#<#$
$vcip = '192.168.1.36'
$vcuser = '[email protected]'
$vcpassword = 'somepassword'
$esxis = @('esxi38','esxi39')
$in_csvpath = "C:\RDMs\2_in_rdmdeviceids_sqlnode1_sqlnode2.csv"
#>

$vc = Connect-VIServer $vcip -User $vcuser -Password $vcpassword

if($vc.IsConnected){
 "Connected to vCenter: " + $vc.Name
 }else{
 "Unable to connect to vCenter: " + $vcip
 exit
}

$rdmlist = Import-Csv -Path $in_csvpath

foreach ($vmhost in $esxis){
 $myesxcli = Get-EsxCli -VMHost $vmhost
 foreach ($rdm in $rdmlist)
 {
 $diskinfo = $myesxcli.storage.core.device.list($rdm.CanonicalName) | Select -ExpandProperty IsPerenniallyReserved
 $vmhost + " " + $rdm.CanonicalName + " " + "IsPerenniallyReserved= " + $diskinfo
 #$myesxcli.storage.core.device.setconfig($false,$rdm.CanonicalName,$false)
 if($diskinfo -eq "false")
 {
 write-host "Configuring Perennial Reservation for LUN " $rdm.CanonicalName " ......."
 $myesxcli.storage.core.device.setconfig($false,$rdm.CanonicalName,$true)
 $diskinfo = $myesxcli.storage.core.device.list($rdm.CanonicalName) | Select -ExpandProperty IsPerenniallyReserved
 $vmhost + " " + $rdm.CanonicalName + " " + "IsPerenniallyReserved= " + $diskinfo
 "----------------"
 }
 }
}

Script: 2_Add-RDMFromCSV.ps1

This script will add all the RDMs on the VM mentioned in $vm1name

After running this script ensure that all RDMs are added properly.

# Script: 2_Add-RDMFromCSV.ps1
function add-rawHD {
 param($VM1Name, $DeviceName, $SCSIcntrl, [int]$UnitNumber, $compatibilityMode
 )
 
 $vm1 = Get-View (Get-VM $VM1name).ID
 foreach($dev in $vm1.config.hardware.device){
 if ($dev.deviceInfo.label -eq $SCSIcntrl){
 $CntrlKey = $dev.key
 }
 }
 $DevKey = 0
 foreach($dev in $vm1.config.hardware.device){
 if ($dev.controllerKey -eq $CntrlKey){
 #if ($dev.Unitnumber -gt $Unitnumber){$Unitnumber = $dev.Unitnumber}
 if ($dev.key -gt $DevKey) {$DevKey = $dev.key}
 }
 }
 $spec = New-Object VMware.Vim.VirtualMachineConfigSpec
 $spec.deviceChange = @()
 $spec.deviceChange += New-Object VMware.Vim.VirtualDeviceConfigSpec
 $spec.deviceChange[0].device = New-Object VMware.Vim.VirtualDisk
 $spec.deviceChange[0].device.backing = New-Object VMware.Vim.VirtualDiskRawDiskMappingVer1BackingInfo
 $spec.deviceChange[0].device.backing.compatibilityMode = $compatibilityMode
 $spec.deviceChange[0].device.backing.datastore = $vm1.Datastore[0]
 $spec.deviceChange[0].device.backing.deviceName = $devicename
 $spec.deviceChange[0].device.backing.fileName = ""
 $spec.deviceChange[0].device.backing.diskMode = ""
 $spec.deviceChange[0].device.connectable = New-Object VMware.Vim.VirtualDeviceConnectInfo
 $spec.deviceChange[0].device.connectable.allowGuestControl = $false
 $spec.deviceChange[0].device.connectable.connected = $true
 $spec.deviceChange[0].device.connectable.startConnected = $true
 $spec.deviceChange[0].device.key = $DevKey + 1
 $spec.deviceChange[0].device.unitnumber = $Unitnumber
 $spec.deviceChange[0].device.controllerKey = $CntrlKey
 $spec.deviceChange[0].operation = "add"
 $spec.deviceChange[0].fileoperation = "create"
 $taskMoRef = $vm1.ReconfigVM_Task($spec)
 $task = Get-View $taskMoRef
 while($task.info.state -eq "running" -or $task.info.state -eq "queued"){
 $task = Get-View $taskMoRef
 }
}

#<#
$vcip = '192.168.1.36'
$vcuser = '[email protected]'
$vcpassword = 'somepassword'
$vm1name = 'sqlnode1'
$in_csvpath = "C:\RDMs\2_in_rdmdeviceids_sqlnode1_sqlnode2.csv"
#>

$vc = Connect-VIServer $vcip -User $vcuser -Password $vcpassword

if($vc.IsConnected){
 "Connected to vCenter: " + $vc.Name
 }else{
 "Unable to connect to vCenter: " + $vcip
 exit
}


$vm1 = get-vm -name $vm1name

$rdmlist = Import-Csv -Path $in_csvpath

foreach ($rdm in $rdmlist)
{
 Add-rawhd -vm1name $vm1name -devicename $rdm.consolename -SCSIcntrl $rdm.controllerid -UnitNumber $rdm.scsiid -compatibilityMode 'physicalMode'
}

Step 3: Export RDM Mappings from First Node

Now that we have added RDMs to the first node, we need to export RDM mappings to add to subsequent nodes. Running following script will generate Output File: 3_out_diskmappings_sqlnode1.csv

Output File: 3_out_diskmappings_sqlnode1

Output File: 3_out_diskmappings_sqlnode1

Script: 3_Export-RDMToCSV.ps1

# Script: 3_Export-RDMToCSV.ps1
#<#
$vcip = '192.168.1.36'
$vcuser = '[email protected]'
$vcpassword = 'somepassword'
$vm1name = 'sqlnode1'
$out_csvpath = 'C:\RDMs\3_out_diskmappings_sqlnode1.csv'
#>

$vc = Connect-VIServer $vcip -User $vcuser -Password $vcpassword

if($vc.IsConnected){
 "Connected to vCenter: " + $vc.Name
 }else{
 "Unable to connect to vCenter: " + $vcip
 exit
}

$vmview = Get-View (Get-VM -name $vm1name).ID
$DiskInfo= @()

foreach ($VirtualSCSIController in ($VMView.Config.Hardware.Device | where {$_.DeviceInfo.Label -match "SCSI Controller"}))
{
 foreach ($VirtualDiskDevice in ($VMView.Config.Hardware.Device | where {$_.ControllerKey -eq $VirtualSCSIController.Key}))
 {
 $VirtualDisk = "" | Select VM, SCSIController, DiskName, DiskType, SCSIId, DiskFile, DiskSize
 $VirtualDisk.VM = $vmview.name
 $VirtualDisk.SCSIController = $VirtualSCSIController.DeviceInfo.Label
 $VirtualDisk.DiskName = $VirtualDiskDevice.DeviceInfo.Label
 $VirtualDisk.DiskType = $VirtualDiskDevice.backing
 $VirtualDisk.SCSIId = $VirtualDiskDevice.UnitNumber
 $VirtualDisk.DiskFile = $VirtualDiskDevice.Backing.FileName
 $VirtualDisk.DiskSize = $VirtualDiskDevice.CapacityInKB * 1KB / 1GB

 $DiskInfo += $VirtualDisk
 }
}

$DiskInfo | export-csv -Path $out_csvpath -Delimiter "," -NoTypeInformation

Step 4: Map RDMs to Subsequent Nodes

As you can see in step 3 output file, all virtual disks associated with the first node are listed. However, we only need virtual disks with disk type as “VMware.Vim.VirtualDiskRawDiskMappingVer1BackingInfo”. So remove other lines and save it as Input File “4_in_MAP-rdmmappings-of-sqlnode1.csv”

Note: There is no need to change VM name in CSV file as it is not used by the script. It is only for information and clarity.

Input File: 4_in_MAP-rdmmappings-of-sqlnode1

Input File: 4_in_MAP-rdmmappings-of-sqlnode1

Script: 4_Map-RDMFromCSV.ps1

This script will map RDMs to node mentioned in $vm2name variable. So run this script for each subsequent node.

# Script: 4_Map-RDMFromCSV.ps1

function map-rawHD {
 param($VM2Name, $DiskFile, $SCSIcntrl, [int]$UnitNumber
 )
 
 $vm2 = Get-View (Get-VM $VM2name).ID
 foreach($dev in $vm2.config.hardware.device){
 if ($dev.deviceInfo.label -eq $SCSIcntrl){
 $CntrlKey = $dev.key
 }
 }
 #$Unitnumber = 0
 $DevKey = 0
 foreach($dev in $vm2.config.hardware.device){
 if ($dev.controllerKey -eq $CntrlKey){
 #if ($dev.Unitnumber -gt $Unitnumber){$Unitnumber = $dev.Unitnumber}
 if ($dev.key -gt $DevKey) {$DevKey = $dev.key}
 }
 }
 "Key: " + $cntrlkey
 "Controller: " + $SCSIcntrl
 $spec = New-Object VMware.Vim.VirtualMachineConfigSpec
 $spec.deviceChange = @()
 $spec.deviceChange += New-Object VMware.Vim.VirtualDeviceConfigSpec
 $spec.deviceChange[0].device = New-Object VMware.Vim.VirtualDisk
 $spec.deviceChange[0].device.backing = New-Object VMware.Vim.VirtualDiskRawDiskMappingVer1BackingInfo
 $spec.deviceChange[0].device.backing.fileName = $DiskFile
 $spec.deviceChange[0].device.backing.diskMode = "independent_persistent"
 $spec.deviceChange[0].device.connectable = New-Object VMware.Vim.VirtualDeviceConnectInfo
 $spec.deviceChange[0].device.connectable.allowGuestControl = $false
 $spec.deviceChange[0].device.connectable.connected = $true
 $spec.deviceChange[0].device.connectable.startConnected = $true
 $spec.deviceChange[0].device.key = $DevKey + 1
 $spec.deviceChange[0].device.unitnumber = $Unitnumber
 $spec.deviceChange[0].device.controllerKey = $CntrlKey 
 $spec.deviceChange[0].operation = "add"

 $taskMoRef = $vm2.ReconfigVM_Task($spec)
 $task = Get-View $taskMoRef
 while($task.info.state -eq "running" -or $task.info.state -eq "queued"){
 $task = Get-View $taskMoRef
 }
}

#<#
$vcip = '192.168.1.36'
$vcuser = '[email protected]'
$vcpassword = 'somepassword'
$vm2name = 'sqlnode2'
$in_csvpath = "C:\RDMs\4_in_rdmmappings_sqlnode1.csv"
#>

$vc = Connect-VIServer $vcip -User $vcuser -Password $vcpassword

if($vc.IsConnected){
 "Connected to vCenter: " + $vc.Name
 }else{
 "Unable to connect to vCenter: " + $vcip
 exit
}

$vm2 = get-vm -name $vm2name
$rdmmappinglist = Import-Csv -Path $in_csvpath

foreach ($rdmmapping in $rdmmappinglist)
{
 map-rawhd -VM2Name $vm2name -DiskFile $rdmmapping.diskfile -SCSIcntrl $rdmmapping.SCSIController -UnitNumber $rdmmapping.SCSIId
}

This is one way to organize and automate the process of RDM mapping. But one can reuse the code to do it differently or integrate it with orchestrators for more sophisticated automation.

All input/output files and scripts can be downloaded from here.

Thanks for reading.

About Dinesh Sharma

Experienced system architect, programmer, and trainer. This blog is a way of giving back and helping the community. So feel free to ask a question or to leave a comment.