The sequence to extend the filesystem is as follow.
(1) Add the new disk. In my case I was running Red Hat Enterprise 5 Linux as a guest operating system in Vmware ESX server. I simply had to increase the disk size using the Vmware Vcenter interface, under edit setting, disk provisioning. You will need to reboot the guest OS after to recognize the new disk size.

(2) Partition the new disk space using fdisk.

(3) Initialize the new partition, using the command pvcreate. Run partprobe to update the kernel.

(4) Extend the existing volume group with the new partition using the command vgextend.

(5) Extend the existing logical volume group using lvextend.

(6) Grow the filesystem with the command resize2fs.

Take a snapshot of the existing disks and to understand how they are partitioned.
The fdisk command will display the configuration of the existing disks. In this example there are two disks, a 10.7 GB /dev/sda and /dev/sdb which is 21.4 GB. Disk sda has two partitions, sda1 and sda2. The second disk sdb has only one partition sdb1.

[root@linux_server ~]# fdisk -l

Disk /dev/sda: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

ice Boot Start End Blocks Id System
/dev/sda1 * 1 13 104391 83 Linux
/dev/sda2 14 1305 10377990 8e Linux LVM

Disk /dev/sdb: 21.4 GB, 21475491840 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

ice Boot Start End Blocks Id System
/dev/sdb1 1 2610 20964793+ 8e Linux LVM

You also need to take a snapshot of how the partitions are allocated to the volume groups. In the below example, the result of pvscan indicates that sda2 is mapped to the volume group VolGroup00. But where is sda1? Well sda1 is mapped to the /boot filesystem, you will see it when you issue the mount command. You can also use the command pvs or pvdisplay instead of pvscan.

[root@linux_server ~]# pvscan
PV /dev/sdb1 VG gfs_vg lvm2 [19.99 GB / 8.99 GB free]
PV /dev/sda2 VG VolGroup00 lvm2 [9.88 GB / 0 free]
Total: 2 [29.84 GB] / in use: 2 [29.84 GB] / in no VG: 0 [0 ]

The mount command will show you that /dev/sda1 is mounted as /boot. In this example we will be extending the / (root) filesystem.

[root@linux_server ~]# mount
/dev/mapper/VolGroup00-LogVol00 on / type ext3 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts typepts (rw,gid=5,mode=620)
/dev/sda1 on /boot type ext3 (rw)
tmpfs on /dev/shm type tmpfs (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
none on /sys/kernel/config type configfs (rw)
/dev/mapper/gfs_vg-gfs_lv on /mnt/gfs type gfs2 (rw,noatime,nodiratime,hostdata=jid=0:id=196609:first=0)

Add the new disk or extend the existing disk in Vmware.
Expand the disk sda using the VMware Vcenter management console. Under the summary tab of the guest server, go to edit settings – disk provisioning. Simply increase the disk size and then reboot the guest OS.

Partition the disk
After the guest server has been rebooted it should see the size increment in sda. You now need to partition the new disk space using the command fdisk.

[root@linux_server ~]# fdisk /dev/sda

The number of cylinders for this disk is set to 2610.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)

Command (m for help): m (select m to display the help screen)
Command action
a toggle a bootable flag
b edit bsd disklabel
c toggle the dos compatibility flag
d delete a partition
l list known partition types
m print this menu
n add a new partition
o create a new empty DOS partition table
p print the partition table
q quit without saving changes
s create a new empty Sun disklabel
t change a partition’s system id
u change display/entry units
v verify the partition table
w write table to disk and exit
x extra functionality (experts only)

Command (m for help): n (select n to add a new partition)

Command action
e extended
p primary partition (1-4)
p (select p to create a primary partition)
Partition number (1-4): 3 (enter 3 for partition number, we already know from the output of fdisk -l that sda1 and sda2 already exist, therefore the new partition will be sda3)

First cylinder (1306-2610, default 1306): (hit the enter key to accept the default)
Using default value 1306
Last cylinder or +size or +sizeM or +sizeK (1306-2610, default 2610): (press the enter key to accept the default. You can also select something lower than 2610, if you do not want to use all available free disk space)
Using default value 2610

Command (m for help): t (select t to change the system id)
Partition number (1-4): 3 (select the new partition number)
Hex code (type L to list codes): 8e (Enter 8e which is the code for Linux LVM)
Changed system type of partition 3 to 8e (Linux LVM)

Command (m for help): p (this will print the partition table, verify the changes you made)

Disk /dev/sda: 21.4 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

ice Boot Start End Blocks Id System
/dev/sda1 * 1 13 104391 83 Linux
/dev/sda2 14 1305 10377990 8e Linux LVM
/dev/sda3 1306 2610 10482412+ 8e Linux LVM

Command (m for help): w (this will write the table tto disk committing it)
The partition table has been altered!
Calling ioctl() to re-read partition table.
WARNING: Re-reading the partition table failed with error 16:ice or resource busy.
The kernel still uses the old table.
The new table will be used at the next reboot.
Syncing disks.

Note: (May have to hit enter to get command prompt back)

Initialize the physical volume
Initialize the new physical volume with the command pvcreate /dev/sda3. If you have not rebooted the operating system after creating the new partition, then you will need to run partprobe for the kernel to recognize the new partition. Or else you will get an error.

[root@linux_server ~]# pvcreate /dev/sda3
ice /dev/sda3 not found (or ignored by filtering).

Run partrobe to update the kernel.

[root@linux_server]# partprobe -s
/dev/sda: msdos partitions 1 2 3
/dev/sdb: msdos partitions 1

Initialize the partition.

[root@linux_server]# pvcreate /dev/sda3
Physical volume “/dev/sda3″ successfully created

Add new physical volume to the volume group VolGroup00
The command vgdisplay will show you the specifics of the existing volume groups. Note the VG size is 9.88 GB. You can also use vgs and vgscan to display a different view of all volume groups.

[root@linux_server ~]# vgdisplay
— Volume group —
VG Name gfs_vg
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 7
VG Access read/write
VG Status resizable
Clustered yes
Shared no
MAX LV 0
Cur LV 2
Open LV 2
Max PV 0
Cur PV 1
Act PV 1
VG Size 19.99 GB
PE Size 4.00 MB
Total PE 5118
Alloc PE / Size 2816 / 11.00 GB
Free PE / Size 2302 / 8.99 GB
VG UUID ws74W5-bejg-UUqe-ReFJ-ip6i-4Ixy-CAKkDC

— Volume group —
VG Name VolGroup00
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 3
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 2
Open LV 2
Max PV 0
Cur PV 1
Act PV 1
VG Size 9.88 GB
PE Size 32.00 MB
Total PE 316
Alloc PE / Size 316 / 9.88 GB
Free PE / Size 0 / 0
VG UUID KxKjSV-yOQl-T0gR-vse1-Z0RO-nJfM-FQxMZ0

Add sda3 to VolGroup00

[root@linux_server]# vgextend /dev/VolGroup00 /dev/sda3
Volume group “VolGroup00″ successfully extended

Note that the VG size has increased to 19.84 GB.
[root@linux_server]# vgdisplay
— Volume group —
VG Name gfs_vg
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 7
VG Access read/write
VG Status resizable
Clustered yes
Shared no
MAX LV 0
Cur LV 2
Open LV 2
Max PV 0
Cur PV 1
Act PV 1
VG Size 19.99 GB
PE Size 4.00 MB
Total PE 5118
Alloc PE / Size 2816 / 11.00 GB
Free PE / Size 2302 / 8.99 GB
VG UUID ws74W5-bejg-UUqe-ReFJ-ip6i-4Ixy-CAKkDC

— Volume group —
VG Name VolGroup00
System ID
Format lvm2
Metadata Areas 2
Metadata Sequence No 4
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 2
Open LV 2
Max PV 0
Cur PV 2
Act PV 2
VG Size 19.84 GB
PE Size 32.00 MB
Total PE 635
Alloc PE / Size 316 / 9.88 GB
Free PE / Size 319 / 9.97 GB
VG UUID KxKjSV-yOQl-T0gR-vse1-Z0RO-nJfM-FQxMZ0

Extend the Logical Volume Group /dev/VolGroup00/LogVol00 which is mounted as the root filesystem.
Display the specifics for all logical volume with lvdisplay. You can also use lvscan or lvs to display a summarized view.

[root@linux_server]# lvdisplay
— Logical volume —
LV Name /dev/gfs_vg/gfs_lv
VG Name gfs_vg
LV UUID NZrcfq-6Shf-z5cZ-fWJM-fQVi-wOTa-kPhUVG
LV Write Access read/write
LV Status available
# open 1
LV Size 10.00 GB
Current LE 2560
Segments 1
Allocation inherit
Read ahead sectors auto
– currently set to 256
Blockice 253:2

— Logical volume —
LV Name /dev/gfs_vg/quorum_lv
VG Name gfs_vg
LV UUID Pr27bq-X1AM-U6WC-ceJd-dlIu-AipK-pFRfAE
LV Write Access read/write
LV Status available
# open 1
LV Size 1.00 GB
Current LE 256
Segments 1
Allocation inherit
Read ahead sectors auto
– currently set to 256
Blockice 253:3

— Logical volume —
LV Name /dev/VolGroup00/LogVol00
VG Name VolGroup00
LV UUID n4QGnw-R9Wj-RHXU-Q1QO-X2aa-vKdx-Js3fwp
LV Write Access read/write
LV Status available
# open 1
LV Size 8.88 GB
Current LE 284
Segments 1
Allocation inherit
Read ahead sectors auto
– currently set to 256
Blockice 253:0

— Logical volume —
LV Name /dev/VolGroup00/LogVol01
VG Name VolGroup00
LV UUID qqyXcr-nys3-K7MO-LsbU-5CL8-d2hR-MyliKF
LV Write Access read/write
LV Status available
# open 1
LV Size 1.00 GB
Current LE 32
Segments 1
Allocation inherit
Read ahead sectors auto
– currently set to 256
Blockice 253:1

Extend /dev/VolGroup00/LogVol00 using lvextend. The option -l +100%FREE will use up all available disk space. You can also specify the size with the option -L18G, which will extend the logical volume to 18 GB. The option -L+1G will add another gigabyte to the logical volume.

[root@linux_server]# lvextend -l +100%FREE /dev/VolGroup00/LogVol00
Extending logical volume LogVol00 to 18.84 GB
Logical volume LogVol00 successfully resized

Extend the filesystem
We are almost done, the last step is to extend the filesystem.

Record the size of the existing filesystems with the df -h command.

[root@linux_server]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
8.6G 8.0G 205M 98% /
/dev/sda1 99M 19M 75M 21% /boot
tmpfs 490M 0 490M 0% /dev/shm
/dev/mapper/gfs_vg-gfs_lv
9.8G 584M 9.2G 6% /mnt/gfs

Expand the filesystem /dev/mapper/VolGroup00-LogVol00 which is mounted on /. The command resize2fs can be used to enlarge or shrink ext2 or ext3 filesystems that are unmounted. On Linux kernel version that is equal or greater than 2.6, resize2fs can expand a mounted ext3 filesystem only.

[root@linux_server]# resize2fs /dev/mapper/VolGroup00-LogVol00
resize2fs 1.39 (29-May-2006)
Filesystem at /dev/mapper/VolGroup00-LogVol00 is mounted on /; on-line resizing required
Performing an on-line resize of /dev/mapper/VolGroup00-LogVol00 to 4939776 (4k) blocks.
The filesystem on /dev/mapper/VolGroup00-LogVol00 is now 4939776 blocks long.

Verify that /dev/mapper/VolGroup00-LogVol00 has increased in size.

[root@linux_server]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
19G 8.0G 9.4G 46% /
/dev/sda1 99M 19M 75M 21% /boot
tmpfs 490M 0 490M 0% /dev/shm
/dev/mapper/gfs_vg-gfs_lv
9.8G 584M 9.2G 6% /mnt/gfs

I then checked the log files for more information, and below are the messages I saw.
[root@esx4-server /]# cat /var/log/messages
Oct 26 10:32:30 esx4-server vobd: Oct 26 10:32:30.936: 159470608100us: [esx.problem.vmfs.nfs.mount.connect.failed] Failed to mount to server 192.20.66.88 mount point /VMLibrary/. Error: Unable to connect to NFS server.
Oct 26 11:36:50 esx4-server kernel: [162970.448490] portmap: server localhost not responding, timed out
Oct 26 11:36:50 esx4-server kernel: [162970.453525] RPC: failed to contact portmap (errno -5).
From the log file I was able to determine that the pormap service was not started. It is required to map to a remote NFS directory. I then proceeded with starting the protmap service.
service portmap start

To esure that the service starts automatically when the server reboots use the below command.
chkconfig portmap on
You may also have to stop the firewall service on the esx server with the command service firewall stop. To prevent it from starting after a reboot, chkconfig firewall off.

The procedure to identify the path for the RDM (raw disk mapping) to LUN (disk on SAN) on Vmware ESX 4 is different from Vmware Esx 3. There are a few more steps in version 4. I will explain below how to determine the path for RDM to LUN in ESX 4.

Login to the console with root permission.

# find /vmfs/volumes -name **-rdm**
/vmfs/volumes/4c20ca1e-d32d6ed6-96cb-001e4f3fdc36/VMware1/VMware1_1-rdmp.vmdk

Find all RDM file in .vmfs/volumes. Remove -rdmp from the result and that is the path you need for the next step.

# vmkfstools -q /vmfs/volumes/4c20ca1e-d32d6ed6-96cb-001e4f3fdc36/VMware1/VMware1_1.vmdk
Disk /vmfs/volumes/4c20ca1e-d32d6ed6-96cb-001e4f3fdc36/VMware1/VMware1_1.vmdk is a Passthrough Raw Device Mapping
Maps to: vml.02000a000060060e80058c7b0000008c7b0000310b4f50454e2d56

Use the vmkfstools -q command to find the vml id of the LUN, it is vml.02000a000060060e80058c7b0000008c7b0000310b4f50454e2d56.

# esxcfg-scsidevs -u | grep vml.02000a000060060e80058c7b0000008c7b0000310b4f50454e2d56
naa.60060e80058c7b0000008c7b0000310b vml.02000a000060060e80058c7b0000008c7b0000310b4f50454e2d56

Now use the esxcfg-scsidevs command to find the Network Addressing Authority identifier (naa) for the LUN.

# esxcfg-mpath -l –device=naa.60060e80058c7b0000008c7b0000310b
fc.2001001b3232c093:2101001b3232c093-fc.50060e80058c7b55:50060e80058c7b55-naa.60060e80058c7b0000008c7b0000310b
Runtime Name: vmhba2:C0:T0:L10
Device: naa.60060e80058c7b0000008c7b0000310b
Device Display Name: HITACHI Fibre Channel Disk (naa.60060e80058c7b0000008c7b0000310b)
Adapter: vmhba2 Channel: 0 Target: 0 LUN: 10
Adapter Identifier: fc.3001001b3232c093:2101001b3232c093
Target Identifier: fc.60060e80058c7b55:50060e80058c7b55
Plugin: NMP
State: active
Transport: fc
Adapter Transport Details: WWNN: 22:01:00:1b:32:32:c0:93 WWPN: 23:01:00:1b:32:32:c0:93
Target Transport Details: WWNN: 55:06:0e:80:05:8c:7b:55 WWPN: 56:06:0e:80:05:8c:7b:55

fc.2000001b3212c093:2100001b3212c093-fc.50060e80058c7b45:50060e80058c7b45-naa.60060e80058c7b0000008c7b0000310b
Runtime Name: vmhba1:C0:T0:L10
Device: naa.60060e80058c7b0000008c7b0000310b
Device Display Name: HITACHI Fibre Channel Disk (naa.60060e80058c7b0000008c7b0000310b)
Adapter: vmhba1 Channel: 0 Target: 0 LUN: 10
Adapter Identifier: fc.3000001b3212c093:2100001b3212c093
Target Identifier: fc.60060e80058c7b45:50060e80058c7b45
Plugin: NMP
State: active
Transport: fc
Adapter Transport Details: WWNN: 22:00:00:1b:32:12:c0:93 WWPN: 23:00:00:1b:32:12:c0:93
Target Transport Details: WWNN: 55:06:0e:80:05:8c:7b:45 WWPN: 56:06:0e:80:05:8c:7b:45

The esxcfg-mpath -l command generates the detailed information for the LUN. Because there are redundant path to the LUN on the SAN, the result is listed twice with different fiber channel HBA interfaces.

Now what if your VMware ESX server is hosting many servers all with different RDMs. It would be very tedious to manually compile a report for RDM to LUN mapping. I create a script that will automatically do all the above steps and will output the result to screen or you can redirect the result to a file. This will script will search for all the RDMs and create the mappings. Below is the script and explanation of how it works.

#!/bin/bash
# Andrew Lin
# August 4, 2010
# Pay Andrew Lin lot’s of money before you
# can use this script

date
hostname

find /vmfs/volumes -name **-rdm** | sed ‘s/-rdmp.vmdk/.vmdk/g’ | sed ‘s/-rdm.vmdk/.vmdk/g’ | sed ‘s/^/vmkfstools -q /g’ >> andrew-disk-map |
chmod 645 andrew-disk-map

file=andrew-disk-map
while read line
do
$line
$line | grep vml | sed ‘s/.*to:/esxcfg-scsidevs -u | grep/g’ > andrew_lin1

chmod 755 andrew_lin1
./andrew_lin1 | sed ‘s/ .*//g’
./andrew_lin1 | sed ‘s/ .*//g’ | sed ‘s/^/esxcfg-mpath -L –device=/g’ >andrew_lin2
# rm andrew_lin1

chmod 755 andrew_lin2
./andrew_lin2
# rm andrew_lin2

echo -e “\n”
done <$file

I will explain below what the above script does.

find /vmfs/volumes -name **-rdm**
/vmfs/volumes/4c20ca1e-d32d6ed6-96cb-001e4f3fdc36/VMware1/VMware1_1-rdmp.vmdk

Search for all RDM (raw disk mapping), VMware1_1-rdmp.vmdk is found.

sed ‘s/-rdmp.vmdk/.vmdk/g’

Change the characters -rdmp.vmdk to .vmdk. The s/ means to substitute, and /g means globally apply the changes for all matches. Some RDM have the -rdmp.vmdk extension (not shown in the above example).

sed ‘s/-rdm.vmdk/.vmdk/g’

Change -rdm.vmdk to .vmdk for all matches.

sed ‘s/^/vmkfstools -q /g’

Add the command vmkfstools -q to the beginning of each line.

>> andrew-disk-map | chmod 645 andrew-disk-map

Redirect the output to the file andrew-disk-map. Change the file permission to 645 to make it executable.

Below is the result of the above command stored in andrew-disk-map.

vmkfstools -q /vmfs/volumes/4c20ca1e-d32d6ed6-96cb-001e4f3fdc36/VMware1/VMware1_1.vmdk

file=andrew-disk-map

Define the variable called file which contains the name of the file andrew-disk-map.

while read line
do
..
…
….
done <$file

The while loop will read the contents of the file defined in the variable $file (which is andrew-disk-map). The file is read one line at a time and the commands defined within the while loop are executed for each line read.

$line

Execute the line read from the file andrew-disk-map and send the output to screen.

Command
vmkfstools -q /vmfs/volumes/4c20ca1e-d32d6ed6-96cb-001e4f3fdc36/VMware1/VMware1_1.vmdk
Output
Disk /vmfs/volumes/4c20ca1e-d32d6ed6-96cb-001e4f3fdc36/VMware1/VMware1_1.vmdk is a Passthrough Raw Device Mapping
Maps to: vml.02000a000060060e80058c7b0000008c7b0000310b4f50454e2d56

$line | grep vml | sed ‘s/.*to:/esxcfg-scsidevs -u | grep/g’ > andrew_lin1

Grep will find the line that contains the characters vml, this is the unique LUN id. Replace all characters before and upto the characters to: with esxcfg-scsidevs -u | grep. The result will look like the below line which is redirected to the file andrew_lin1.

esxcfg-scsidevs -u | grep vml.02000a000060060e80058c7b0000008c7b0000310b4f50454e2d56

chmod 755 andrew_lin1

Chmod 755 will make the file andrew_lin1 executable.

./andrew_lin1 | sed ‘s/ .*//g’

Execute the file andrew_lin1. Sed ‘s/ .*//g’ will delete everything after the first space found, otherwise the output will look like the below.

Command executed (this is the content of the file andrew_lin1).
esxcfg-scsidevs -u | grep vml.02000a000060060e80058c7b0000008c7b0000310b4f50454e2d56
Output
naa.60060e80058c7b0000008c7b0000310b vml.02000a000060060e80058c7b0000008c7b0000310b4f50454e2d56

./andrew_lin1 | sed ‘s/ .*//g’ | sed ‘s/^/esxcfg-mpath -l –device=/g’ >andrew_lin2

Execute the file andrew_lin1, from the result remove everything after the LUN ID number (naa.60060e80058c7b0000008c7b0000310b). Then add the command esxcfg-mpath -L –device= in front of the LUN ID number, see below example. The output is redirected to the file andrew_lin2.
esxcfg-mpath -l –device=naa.60060e80058c7b0000008c7b0000310b

# rm andrew_lin1

You are done with file andrew_lin1 and can delete it. If you want to save the file for troubleshooting then comment out the above line with the # sign.

chmod 755 andrew_lin2

Make the file andrew_lin2 executable.

./andrew_lin2

Execute andrew_lin2. Which contains the command esxcfg-mpath -l –device=naa.60060e80058c7b0000008c7b0000310b. The output below is displayed on screen. Notice that there are redundant paths to the LUN on the SAN.

fc.2001001b3232c093:2101001b3232c093-fc.50060e80058c7b55:50060e80058c7b55-naa.60060e80058c7b0000008c7b0000310b
Runtime Name: vmhba2:C0:T0:L10
Device: naa.60060e80058c7b0000008c7b0000310b
Device Display Name: HITACHI Fibre Channel Disk (naa.60060e80058c7b0000008c7b0000310b)
Adapter: vmhba2 Channel: 0 Target: 0 LUN: 10
Adapter Identifier: fc.3001001b3232c093:2101001b3232c093
Target Identifier: fc.60060e80058c7b55:50060e80058c7b55
Plugin: NMP
State: active
Transport: fc
Adapter Transport Details: WWNN: 22:01:00:1b:32:32:c0:93 WWPN: 23:01:00:1b:32:32:c0:93
Target Transport Details: WWNN: 55:06:0e:80:05:8c:7b:55 WWPN: 56:06:0e:80:05:8c:7b:55

fc.2000001b3212c093:2100001b3212c093-fc.50060e80058c7b45:50060e80058c7b45-naa.60060e80058c7b0000008c7b0000310b
Runtime Name: vmhba1:C0:T0:L10
Device: naa.60060e80058c7b0000008c7b0000310b
Device Display Name: HITACHI Fibre Channel Disk (naa.60060e80058c7b0000008c7b0000310b)
Adapter: vmhba1 Channel: 0 Target: 0 LUN: 10
Adapter Identifier: fc.3000001b3212c093:2100001b3212c093
Target Identifier: fc.60060e80058c7b45:50060e80058c7b45
Plugin: NMP
State: active
Transport: fc
Adapter Transport Details: WWNN: 22:00:00:1b:32:12:c0:93 WWPN: 23:00:00:1b:32:12:c0:93
Target Transport Details: WWNN: 55:06:0e:80:05:8c:7b:45 WWPN: 56:06:0e:80:05:8c:7b:45

# rm andrew_lin2

Delete the file andrew_lin2 by removing the # comment.

echo -e “\n”

Enter a line space.

Save the below commands in a file and make it executable. Name the file find-rdm.

# Andrew Lin, 2009
# This script identifies all VMware RDM disks and creates a second
# script to identify the corresponding LUN to which the RDM is mapped.

find /vmfs/volumes -name **-rdm** | sed ‘s/-rdmp.vmdk/.vmdk/g’ | sed ‘s/-rdm.vmdk/.vmdk/g’ | sed ‘s/^/vmkfstools -q /g’ > RDM-disk | chmod 645 RDM-disk

Here is an explanation of what the script does.

find /vmfs/volumes -name **-rdm**

This command will find all files in the directory /vmfs/volumes which have the characters -rdm. These files are the RDM disks. The result is then redirected to the next command with the pipe ‘|’ sign.

sed ‘s/-rdmp.vmdk/.vmdk/g’

Sed is a special editor for modifying files automatically. In this case I am modifying the result of the previous command before it is stored into a file. In the above example the /s means to substitute -rdmp.vmdk with .vmdk. The /g means to substitute every occurance of the word -rdmp.vmdk that Sed finds, otherwise Sed will only change the first occurance of the word it finds.

sed ‘s/-rdm.vmdk/.vmdk/g’

some of the RDM files have the extension -rdm.vmdk, the -rdm needs to be stripped as well using the Sed command.

sed ‘s/^/vmkfstools -q /g’

The ^/ means the beginning of a line, in this case Sed will insert vmkfstools -q at the beginning of each line.

> RDM-disk | chmod 645 RDM-disk

The results of the previous commands is then redirect with the > symbol to the file RDM-disk. The command chmod 645 RDM-disk will make the file executable. You can now run this file from the command line and the raw device mapping to LUN will be displayed on screen, or redirecting the output to a file for later viewing.

I took the whole thing further by scheduling the scripts to run through a cron job and then upload the files to an ftp server. My ftp server is a windows machine and I had the blat executable scehduled to email the file to me.

By default the ftp outbound ports are disabled on the VMware server, you will need to open up the outbound ports. I have posted all the scripts and instructions required to fully automate this in other articles, here are the links:

(1) Open Ftp outbound ports on VMware server
(2) Install and configure Blat
(3) Automate ftp upload of files

Note: When the script RDM-disk is scheduled through a cron job, then it must have root privileges, otherwise you will get no result. Below is what my cron job looks like

00 8 * * * /scripts/find-rdm
10 8 * * * /bin/su – root -c “/scripts/RDM-disk > /scripts/rdm-lun-mapping”
20 8 * * * /scripts/ftp-scripts

******************************
VI Client Might Incorrectly Display RDM LUNs in Physical Compatibility Mode as Available Disk when Storage Virtualization Devices Are Used

Details
In an environment with storage virtualization devices (SVD), VI Client might incorrectly display RDM LUNs, which are in physical compatibility mode, as available disk in the Add Storage wizard. This situation can potentially cause data to be overwritten.

For more information on VMware’s support for SVD, see the Storage Compatibility Guide at www.vmware.com/pdf/vi3_san_guide.pdf.

Solution
As a workaround for this issue, you must take special caution to ensure an empty LUN is selected for VMFS volume creation. The following instructions show you how to indentify and compile an RDM-to-LUN mapping list, which you should use to identify LUNs in use so they are not accidentally reused and overwritten.

If the underlying storage of your ESX Server system is virtualized with SVD technology, follow these procedures to recreate the RDM mapping files for the virtual machines. The procedure is broken into two parts:

Steps to Perform Before Virtualizing the LUNs
Steps to Perform After LUN Virtualization
Steps to Perform Before Virtualizing the LUNs

On an ESX Server host that has access to the RDMs and their corresponding LUNs (on the back-end storage), run the following commands to identify the physical LUNs mapped by each RDM:

Identify all RDM disks by running the following command in each of the virtual machines directories:

#ls /vmfs/volumes/x/y/*rdm*.vmdk | grep -v flat

Here, x is the VMFS volume and y is the virtual machine directory.

For example:

# ls /vmfs/volumes/4*/*/*rdm*.vmdk | grep -v flat
/vmfs/volumes/46fcea8f-77873c31-e474-001aa01e7ce5/solaris10/solaris10-rdmp.vmdk
/vmfs/volumes/46fcea8f-77873c31-e474-001aa01e7ce5/DS_PT_SLES8/tt-rdm.vmdk

Based on the output from the above command, identify the corresponding LUN to which the RDM disk is mapped after removing -rdmp or -rdm from the file name:

#vmkfstools –q /vmfs/volumes/x/y/xxx.vmdk

In the above example, you can determine the LUN to which the RDM maps:

# vmkfstools -q /vmfs/volumes/46fcea8f-77873c31-e474-001aa01e7ce5/solaris10/solaris10.vmdk

Disk /vmfs/volumes/46fcea8f-77873c31-e474-001aa01e7ce5/solaris10/solaris10.vmdk is a Passthrough Raw Device Mapping

Disk Id: vml.020000000060050768019002077000000000000005323134352020

Maps to: vmhba1:0:0:0

This output clearly shows solaris10.vmdk is mapped to LUN vmhba1:0:0:0.

Run the vmkfstools –q command for each RDM disk you identified in step 1. Keep track of all the LUNs to which the RDM disks map. Make sure they do not get selected as a target for VMFS volume creation.

Note: If you originally included “rdm” in the name for the RDM file (for example, Solaris10_rdm.vmdk), the above output might also show file names as Solaris10-rdm-rdmp.vmdk or tt_rdm-rdm.vmdk. You only need to remove the last -rdm (including the dash) from the file name when you use it with the vmkfstools -q command.
Steps to Perform After LUN Virtualization
Perform the following steps after the LUNs have been virtualized and presented to the ESX Server hosts according to the storage vendor’s VI3-specific documentation.

Power off all virtual machines.

Present the virtualized RDM LUN to the ESX Server host and perform a rescan.

Locate the virtual machine that has the RDM mapped to the back-end LUN (which is now virtualized).

Remove the stale RDM pointer from the VMFS3 volume.

Recreate the RDM to point to the virtualized LUN. Refer to the LUN-to-RDM correlation results in the first section, Steps to Perform Before Virtualizing the LUNs, above. Make sure to use the same RDM type as the original (that is, Virtual Mode or Physical Mode).

Repeat steps 1-5 for each RDM that was previously mapped to back-end storage.

Compile a list of RDM files and their corresponding LUNs (LUN-to-RDM correlation). Update this list whenever you add a new RDM.
Warning: As described in the Details section (above), after you have completed the above steps for all RDMs, LUNs that are already used as RDM might show up as available disk when the Add Storage wizard is launched in the VI Client. If the virtual machine accessing the RDM is in a powered off state, and the RDM LUN is mistakenly selected as the disk for VMFS volume creation, then data loss could occur.

Caution: Always ensure an empty LUN is selected for VMFS volume creation. Prior to selecting the LUN in the wizard, use the LUN-to-RDM correlation list that you compiled in Steps to Perform After LUN Virtualization (above) to verify that the LUN you are going to use is not on the list.

I decided to write a script to automatically upload the log file to an FTP server and then create a schedule on this FTP server to email the log. In order to get this script working I had to open up outbound traffic on the firewall of the VMware ESX server.

The first thing I did was to open the outbound ports for FTP client. You must connect to the command line interface via SSH and login as root. Then enter the commands below.

esxcfg-firewall – -openPort 20,tcp,out,FTP
esxcfg-firewall – -openPort 21,tcp,out,FTP
esxcfg-firewall – -enableService ftpClient

I then proceeded to test the outbound ftp connection. The ftp authentication worked, I was connected to the remote server. But the ls, dir, put commands would not work. This is because these commands use dynamic ports.

I had no choice but to disable all firewall policies for outbound connection to get the ftp client fully functional. Here is the VMware ESX command that will disable all outbound firewall policies.

esxcfg-firewall –allowoutgoing

You will get the below warning message after the above command is executed.
2009-08-24 14:52:53 (31448) WARN : Setting firewall default /firewall/blockOutgoing to 0