The procedure to identify the path for the RDM (raw disk mapping) to LUN (disk on SAN) on Vmware ESX 4 is different from Vmware Esx 3. There are a few more steps in version 4. I will explain below how to determine the path for RDM to LUN in ESX 4.

Login to the console with root permission.

# find /vmfs/volumes -name **-rdm**
/vmfs/volumes/4c20ca1e-d32d6ed6-96cb-001e4f3fdc36/VMware1/VMware1_1-rdmp.vmdk

Find all RDM file in .vmfs/volumes. Remove -rdmp from the result and that is the path you need for the next step.

# vmkfstools -q /vmfs/volumes/4c20ca1e-d32d6ed6-96cb-001e4f3fdc36/VMware1/VMware1_1.vmdk
Disk /vmfs/volumes/4c20ca1e-d32d6ed6-96cb-001e4f3fdc36/VMware1/VMware1_1.vmdk is a Passthrough Raw Device Mapping
Maps to: vml.02000a000060060e80058c7b0000008c7b0000310b4f50454e2d56

Use the vmkfstools -q command to find the vml id of the LUN, it is vml.02000a000060060e80058c7b0000008c7b0000310b4f50454e2d56.

# esxcfg-scsidevs -u | grep vml.02000a000060060e80058c7b0000008c7b0000310b4f50454e2d56
naa.60060e80058c7b0000008c7b0000310b vml.02000a000060060e80058c7b0000008c7b0000310b4f50454e2d56

Now use the esxcfg-scsidevs command to find the Network Addressing Authority identifier (naa) for the LUN.

# esxcfg-mpath -l –device=naa.60060e80058c7b0000008c7b0000310b
fc.2001001b3232c093:2101001b3232c093-fc.50060e80058c7b55:50060e80058c7b55-naa.60060e80058c7b0000008c7b0000310b
Runtime Name: vmhba2:C0:T0:L10
Device: naa.60060e80058c7b0000008c7b0000310b
Device Display Name: HITACHI Fibre Channel Disk (naa.60060e80058c7b0000008c7b0000310b)
Adapter: vmhba2 Channel: 0 Target: 0 LUN: 10
Adapter Identifier: fc.3001001b3232c093:2101001b3232c093
Target Identifier: fc.60060e80058c7b55:50060e80058c7b55
Plugin: NMP
State: active
Transport: fc
Adapter Transport Details: WWNN: 22:01:00:1b:32:32:c0:93 WWPN: 23:01:00:1b:32:32:c0:93
Target Transport Details: WWNN: 55:06:0e:80:05:8c:7b:55 WWPN: 56:06:0e:80:05:8c:7b:55

fc.2000001b3212c093:2100001b3212c093-fc.50060e80058c7b45:50060e80058c7b45-naa.60060e80058c7b0000008c7b0000310b
Runtime Name: vmhba1:C0:T0:L10
Device: naa.60060e80058c7b0000008c7b0000310b
Device Display Name: HITACHI Fibre Channel Disk (naa.60060e80058c7b0000008c7b0000310b)
Adapter: vmhba1 Channel: 0 Target: 0 LUN: 10
Adapter Identifier: fc.3000001b3212c093:2100001b3212c093
Target Identifier: fc.60060e80058c7b45:50060e80058c7b45
Plugin: NMP
State: active
Transport: fc
Adapter Transport Details: WWNN: 22:00:00:1b:32:12:c0:93 WWPN: 23:00:00:1b:32:12:c0:93
Target Transport Details: WWNN: 55:06:0e:80:05:8c:7b:45 WWPN: 56:06:0e:80:05:8c:7b:45

The esxcfg-mpath -l command generates the detailed information for the LUN. Because there are redundant path to the LUN on the SAN, the result is listed twice with different fiber channel HBA interfaces.

Now what if your VMware ESX server is hosting many servers all with different RDMs. It would be very tedious to manually compile a report for RDM to LUN mapping. I create a script that will automatically do all the above steps and will output the result to screen or you can redirect the result to a file. This will script will search for all the RDMs and create the mappings. Below is the script and explanation of how it works.

#!/bin/bash
# Andrew Lin
# August 4, 2010
# Pay Andrew Lin lot’s of money before you
# can use this script

date
hostname

find /vmfs/volumes -name **-rdm** | sed ‘s/-rdmp.vmdk/.vmdk/g’ | sed ‘s/-rdm.vmdk/.vmdk/g’ | sed ‘s/^/vmkfstools -q /g’ >> andrew-disk-map |
chmod 645 andrew-disk-map

file=andrew-disk-map
while read line
do
$line
$line | grep vml | sed ‘s/.*to:/esxcfg-scsidevs -u | grep/g’ > andrew_lin1

chmod 755 andrew_lin1
./andrew_lin1 | sed ‘s/ .*//g’
./andrew_lin1 | sed ‘s/ .*//g’ | sed ‘s/^/esxcfg-mpath -L –device=/g’ >andrew_lin2
# rm andrew_lin1

chmod 755 andrew_lin2
./andrew_lin2
# rm andrew_lin2

echo -e “\n”
done <$file

I will explain below what the above script does.

find /vmfs/volumes -name **-rdm**
/vmfs/volumes/4c20ca1e-d32d6ed6-96cb-001e4f3fdc36/VMware1/VMware1_1-rdmp.vmdk

Search for all RDM (raw disk mapping), VMware1_1-rdmp.vmdk is found.

sed ‘s/-rdmp.vmdk/.vmdk/g’

Change the characters -rdmp.vmdk to .vmdk. The s/ means to substitute, and /g means globally apply the changes for all matches. Some RDM have the -rdmp.vmdk extension (not shown in the above example).

sed ‘s/-rdm.vmdk/.vmdk/g’

Change -rdm.vmdk to .vmdk for all matches.

sed ‘s/^/vmkfstools -q /g’

Add the command vmkfstools -q to the beginning of each line.

>> andrew-disk-map | chmod 645 andrew-disk-map

Redirect the output to the file andrew-disk-map. Change the file permission to 645 to make it executable.

Below is the result of the above command stored in andrew-disk-map.

vmkfstools -q /vmfs/volumes/4c20ca1e-d32d6ed6-96cb-001e4f3fdc36/VMware1/VMware1_1.vmdk

file=andrew-disk-map

Define the variable called file which contains the name of the file andrew-disk-map.

while read line
do
..
…
….
done <$file

The while loop will read the contents of the file defined in the variable $file (which is andrew-disk-map). The file is read one line at a time and the commands defined within the while loop are executed for each line read.

$line

Execute the line read from the file andrew-disk-map and send the output to screen.

Command
vmkfstools -q /vmfs/volumes/4c20ca1e-d32d6ed6-96cb-001e4f3fdc36/VMware1/VMware1_1.vmdk
Output
Disk /vmfs/volumes/4c20ca1e-d32d6ed6-96cb-001e4f3fdc36/VMware1/VMware1_1.vmdk is a Passthrough Raw Device Mapping
Maps to: vml.02000a000060060e80058c7b0000008c7b0000310b4f50454e2d56

$line | grep vml | sed ‘s/.*to:/esxcfg-scsidevs -u | grep/g’ > andrew_lin1

Grep will find the line that contains the characters vml, this is the unique LUN id. Replace all characters before and upto the characters to: with esxcfg-scsidevs -u | grep. The result will look like the below line which is redirected to the file andrew_lin1.

esxcfg-scsidevs -u | grep vml.02000a000060060e80058c7b0000008c7b0000310b4f50454e2d56

chmod 755 andrew_lin1

Chmod 755 will make the file andrew_lin1 executable.

./andrew_lin1 | sed ‘s/ .*//g’

Execute the file andrew_lin1. Sed ‘s/ .*//g’ will delete everything after the first space found, otherwise the output will look like the below.

Command executed (this is the content of the file andrew_lin1).
esxcfg-scsidevs -u | grep vml.02000a000060060e80058c7b0000008c7b0000310b4f50454e2d56
Output
naa.60060e80058c7b0000008c7b0000310b vml.02000a000060060e80058c7b0000008c7b0000310b4f50454e2d56

./andrew_lin1 | sed ‘s/ .*//g’ | sed ‘s/^/esxcfg-mpath -l –device=/g’ >andrew_lin2

Execute the file andrew_lin1, from the result remove everything after the LUN ID number (naa.60060e80058c7b0000008c7b0000310b). Then add the command esxcfg-mpath -L –device= in front of the LUN ID number, see below example. The output is redirected to the file andrew_lin2.
esxcfg-mpath -l –device=naa.60060e80058c7b0000008c7b0000310b

# rm andrew_lin1

You are done with file andrew_lin1 and can delete it. If you want to save the file for troubleshooting then comment out the above line with the # sign.

chmod 755 andrew_lin2

Make the file andrew_lin2 executable.

./andrew_lin2

Execute andrew_lin2. Which contains the command esxcfg-mpath -l –device=naa.60060e80058c7b0000008c7b0000310b. The output below is displayed on screen. Notice that there are redundant paths to the LUN on the SAN.

fc.2001001b3232c093:2101001b3232c093-fc.50060e80058c7b55:50060e80058c7b55-naa.60060e80058c7b0000008c7b0000310b
Runtime Name: vmhba2:C0:T0:L10
Device: naa.60060e80058c7b0000008c7b0000310b
Device Display Name: HITACHI Fibre Channel Disk (naa.60060e80058c7b0000008c7b0000310b)
Adapter: vmhba2 Channel: 0 Target: 0 LUN: 10
Adapter Identifier: fc.3001001b3232c093:2101001b3232c093
Target Identifier: fc.60060e80058c7b55:50060e80058c7b55
Plugin: NMP
State: active
Transport: fc
Adapter Transport Details: WWNN: 22:01:00:1b:32:32:c0:93 WWPN: 23:01:00:1b:32:32:c0:93
Target Transport Details: WWNN: 55:06:0e:80:05:8c:7b:55 WWPN: 56:06:0e:80:05:8c:7b:55

fc.2000001b3212c093:2100001b3212c093-fc.50060e80058c7b45:50060e80058c7b45-naa.60060e80058c7b0000008c7b0000310b
Runtime Name: vmhba1:C0:T0:L10
Device: naa.60060e80058c7b0000008c7b0000310b
Device Display Name: HITACHI Fibre Channel Disk (naa.60060e80058c7b0000008c7b0000310b)
Adapter: vmhba1 Channel: 0 Target: 0 LUN: 10
Adapter Identifier: fc.3000001b3212c093:2100001b3212c093
Target Identifier: fc.60060e80058c7b45:50060e80058c7b45
Plugin: NMP
State: active
Transport: fc
Adapter Transport Details: WWNN: 22:00:00:1b:32:12:c0:93 WWPN: 23:00:00:1b:32:12:c0:93
Target Transport Details: WWNN: 55:06:0e:80:05:8c:7b:45 WWPN: 56:06:0e:80:05:8c:7b:45

# rm andrew_lin2

Delete the file andrew_lin2 by removing the # comment.

echo -e “\n”

Enter a line space.

I will explain how to recover a Solaris 10 server from hardware replicated SAN disk. It took me sometime to figure out how to boot up from the replicated SAN LUN (disk), and many more hours to understand why the steps I applied works.

In this example I have a SUN SPARC M3000 server with two Qlogic fiber channel cards (HBA) installed in the PCI solts. The HBAs were already configured to connect to the SAN disk (LUN). This LUN contained the replicated copy of a production Slaris 10 server. The production server had two ZFS pool residing in a single LUN.

Using the Solaris 10 installation CD boot up the Sparc server into single user mode.

boot cdrom -s

The first thing you need to do is to see if the HBAs are working. The connected status indictes that communication between the server and switch is working. There are two HBAs installed for redundancy, both connected to the same LUN.

# luxadm -e port
/devices/pci@0,600000/pci@0/pci@8/SUNW,qlc@0/fp@0,0:devctl CONNECTED
/devices/pci@1,700000/pci@0/pci@0/SUNW,qlc@0/fp@0,0:devctl CONNECTED

Now you need to find out if the SAN disk is visible from the server. Even though both HBAs are connected to the same SAN disk, you will see two separate SAN disks in the results below. It just means there are two paths to the SAN.

# luxadm probe
No Network Array enclosures found in /dev/es

Found Fibre Channel device(s):
Node WWN:50060e80058c7b10 Device Type:Disk device
Logical Path:/dev/rdsk/c1t50060E80058C7B10d1s2
Node WWN:50060e80058c7b00 Device Type:Disk device
Logical Path:/dev/rdsk/c2t50060E80058C7B00d1s2

In the above example the first LUN is c1t50060E80058C7B10d1s2. This is the logical device name which is a symbolic link to the physical device name stored in the /devices directory. Logical device names contain the controller number(c2), target number (t50060E80058C7B10), disk number (d1), and slice number (s2).

The next step is to find out how the disk is partitioned, the format command will give you that information. You need this information to understand how to boot up the disk.

# format
Searching for disks…done

AVAILABLE DISK SELECTIONS:
0. c1t50060E80058C7B10d1 1066>/pci@0,600000/pci@0/pci@8/SUNW,qlc@0/fp@0,0/ssd@w50060e80058c7b10,1

1. c2t50060E80058C7B00d1 1066>/pci@1,700000/pci@0/pci@0/SUNW,qlc@0/fp@0,0/ssd@w50060e80058c7b00,1

Select the first disk 0.

Specify disk (enter its number): 0
selecting c1t50060E80058C7B10d1
[disk formatted]

FORMAT MENU:
disk – select a disk
type – select (define) a disk type
partition – select (define) a partition table
current – describe the current disk
format – format and analyze the disk
repair – repair a defective sector
label – write label to the disk
analyze – surface analysis
defect – defect list management
backup – search for backup labels
verify – read and display labels
save – save new disk/partition definitions
inquiry – show vendor, product and revision
volname – set 8-character volume name
! – execute , then return
quit

Display the labels and slices (partitions). In Solaris each slice is treated as a separate physical disk. In the below example you can tell that the disk is labeled as VTOC (Volume Table of Contents) because you can see the cylinders. VTOC is also known as SMI label. If the disk was labeled with EFI (Extensible Firmware Interface), then you would see sectors instead of cylinders. Partition 0 (slice 0) holds the operating system files, the boot disk. Please note that you cannot boot from a disk with EFI label. Slice 2 is the entire physical disk because it contains all cylinders, 0 – 65532.

format> verify

Primary label contents:

Volume name = <
ascii name =
pcyl = 65535
ncyl = 65533
acyl = 2
nhead = 15
nsect = 1066
Part Tag Flag Cylinders Size Blocks
0 root wm 0 – 3356 25.60GB (3357/0/0) 53678430
1 unassigned wm 0 0 (0/0/0) 0
2 backup wm 0 – 65532 499.66GB (65533/0/0) 1047872670
3 unassigned wm 0 0 (0/0/0) 0
4 unassigned wm 0 0 (0/0/0) 0
5 unassigned wm 0 0 (0/0/0) 0
6 unassigned wm 0 0 (0/0/0) 0
7 unassigned wm 3357 – 65532 474.07GB (62176/0/0) 994194240

Here is what we know so far, we know that the disk name is c1t50060E80058C7B10d1. Slice 0 on this disk contains the boot files. The physical path for this disk is /pci@0,600000/pci@0/pci@8/SUNW,qlc@0/fp@0,0/ssd@w50060e80058c7b10,1. Now we need to find out the physical path for Slice 0.

I know that the disk contains ZFS filesystems because it is a replica of the production disk. When a ZFS filesystem is moved to a different SPARC server it must first be imported because the hostid is different.

List the ZFS pool contained on the disk using the zpool import command. There are two ZFS pools in the below example, epool and rpool. Take a note of the status and action.

# zpool import
pool: epool
id: 16865366839830765202
state: ONLINE
status: The pool was last accessed by another system.
action: The pool can be imported using its name or numeric identifier and the ‘-f’ flag.
see: http://www.sun.com/msg/ZFS-8000-EY
config:

epool ONLINE
c2t50060E80058C7B00d1s7 ONLINE

pool: rpool
id: 10594898920105832331
state: ONLINE
status: The pool was last accessed by another system.
action: The pool can be imported using its name or numeric identifier and the ‘-f’ flag.
see: http://www.sun.com/msg/ZFS-8000-EY
config:

rpool ONLINE
c2t50060E80058C7B00d1s0 ONLINE

Import the pool with the zpool import command. The options -a will import all the ZFS pools it can find, the -f option will force the import. If you do not specify the force option, then the import may fail with the error “cannot import ‘rpool’: pool may be in use from other system, it was last accessed by server name (hostid: 123456)”. Ignore the error message about failed to create mountpoint.

# zpool import -af
cannot mount ‘/epool’: failed to create mountpoint
cannot mount ‘/rpool’: failed to create mountpoint

List the imported ZFS pools.

# zpool list
NAME SIZE USED AVAIL CAP HEALTH ALTROOT
epool 472G 260G 212G 55% ONLINE -
rpool 25.5G 5.75G 19.8G 22% ONLINE -

List the ZFS filesystems. In the example below notice the mountpoint / is mounted on the zfs filesystem rpool/ROOT/zfsboot. This is the boot partition, it resides in rpool.

# zfs list
NAME USED AVAIL REFER MOUNTPOINT
epool 260G 205G 21K /epool
rpool 7.74G 17.4G 99K /rpool
rpool/ROOT 4.74G 17.4G 21K legacy
rpool/ROOT/zfsboot 4.74G 17.4G 4.48G /
rpool/ROOT/zfsboot/var 139M 17.4G 105M /var
rpool/dump 1.00G 17.4G 1.00G -
rpool/swap 2.00G 19.4G 16K -

Change the mountpoint for rpoo/ROOT/zfsboot to /mnt so you can mount it to read the contents.

# zfs set mountpoint=/mnt rpool/ROOT/zfsboot

Confirm that the mountpoint was changed.

# zfs list
NAME USED AVAIL REFER MOUNTPOINT
epool 260G 205G 21K /epool
rpool 7.74G 17.4G 99K /rpool
rpool/ROOT 4.74G 17.4G 21K legacy
rpool/ROOT/zfsboot 4.74G 17.4G 4.48G /mnt
rpool/ROOT/zfsboot/var 139M 17.4G 105M /mnt/var
rpool/dump 1.00G 17.4G 1.00G -
rpool/swap 2.00G 19.4G 16K -

Now mount rpool/ROOT/zfsboot.

# zfs mount rpool/ROOT/zfsboot

List the logical disks.

# cd /dev/dsk
# ls
c1t50060E80058C7B10d1s0 c2t50060E80058C7B00d1s0
c1t50060E80058C7B10d1s1 c2t50060E80058C7B00d1s1
c1t50060E80058C7B10d1s2 c2t50060E80058C7B00d1s2
c1t50060E80058C7B10d1s3 c2t50060E80058C7B00d1s3
c1t50060E80058C7B10d1s4 c2t50060E80058C7B00d1s4
c1t50060E80058C7B10d1s5 c2t50060E80058C7B00d1s5
c1t50060E80058C7B10d1s6 c2t50060E80058C7B00d1s6
c1t50060E80058C7B10d1s7 c2t50060E80058C7B00d1s7

As stated earlier the physical path for the disk we are looking for is /pci@0,600000/pci@0/pci@8/SUNW,qlc@0/fp@0,0/ssd@w50060e80058c7b10,1. The boot slice is 0. We can derive from the physical path that the disk name is 50060e80058c7b10. We also know from the output of the format command that the physical disk 50060e80058c7b10 maps to the logical disk c1t50060E80058C7B10d1. Therefore we can derive that the logical boot disk is c1t50060E80058C7B10d1s0.

Now find out what physical path c1t50060E80058C7B10d1s0 is a symbolic link for and that is your complete boot path. In the below example the boot path starts at the first slash (/) right after /devices. It is /pci@0,600000/pci@0/pci@8/SUNW,qlc@0/fp@0,0/ssd@w50060e80058c7b10,1:a. You need to replace ssd@ with disk@ when entering the path into EEPROM.

# ls -l c1t50060E80058C7B10d1s0
lrwxrwxrwx 1 root root 82 Jul 5 10:21 c1t50060E80058C7B10d1s0 ->
../../devices/pci@0,600000/pci@0/pci@8/SUNW,qlc@0/fp@0,0/ssd@w50060e80058c7b10,1:a

If you have more than one ZFS pool the non root pool may not get mounted upon booting up the server, as in this example. You may get the below error.

SUNW-MSG-ID: ZFS-8000-D3, TYPE: Fault, VER: 1, SEVERITY: Major
EVENT-TIME: Mon Jul 5 11:54:14 EDT 2010
PLATFORM: SUNW,SPARC-Enterprise, CSN: PX654321, HOSTNAME: Andrew-Lin
SOURCE: zfs-diagnosis, REV: 1.0
EVENT-ID: 33e5a9f1-49ac-6ebc-f2a9-dff25dea6b86
DESC: A ZFS device failed. Refer to http://sun.com/msg/ZFS-8000-D3 for more information.
AUTO-RESPONSE: No automated response will occur.
IMPACT: Fault tolerance of the pool may be compromised.
REC-ACTION: Run ‘zpool status -x’ and replace the bad device.

http://sun.com/msg/ZFS-8000-D3fameserverq9{root}: zpool status -x
pool: epool
state: UNAVAIL
status: One or more devices could not be opened. There are insufficient
replicas for the pool to continue functioning.
action: Attach the missing device and online it using ‘zpool online’.
see: http://www.sun.com/msg/ZFS-8000-3C
scrub: none requested
config:

NAME STATE READ WRITE CKSUM
epool UNAVAIL 0 0 0 insufficient replicas
c3t60060E8005652C000000652C00002100d0s7 UNAVAIL 0 0 0 cannot open

The above error is caused by the zpool.cache file. This file contains the old paths of the disks from the previous server. The default behavior of Solaris 10 is to read the path from the zpool.cache file to speed up the boot sequence. You should delete this file and the system will recreate a fresh one during the boot up sequence.

Below are the steps to rename the zpool.cache file.

# cd /mnt/etc/zfs
# ls
zpool.cache
# mv zpool.cache zpool.cache.old

Now you need to reverse the changed you applied to the mountpoint earlier. Make sure that you change directory out of /mnt to /, otherwise the set mountpoint command will fail with the error device busy. Ignore the cannot mount ‘/’: directory is not empty message.

# zfs set mountpoint=/ rpool/ROOT/zfsboot
cannot mount ‘/’: directory is not empty
property may be set but unable to remount filesystem

Confirm that the mount points were changed.

# zfs list
NAME USED AVAIL REFER MOUNTPOINT
epool 260G 205G 21K /epool
rpool 7.74G 17.4G 99K /rpool
rpool/ROOT 4.74G 17.4G 21K legacy
rpool/ROOT/zfsboot 4.74G 17.4G 4.48G /
rpool/ROOT/zfsboot/var 139M 17.4G 105M /var
rpool/dump 1.00G 17.4G 1.00G -
rpool/swap 2.00G 19.4G 16K -

Shutdown the server.

# init 0

Now set the boot device in EEPROM.

setenv boot-device /pci@0,600000/pci@0/pci@8/SUNW,qlc@0/fp@0,0/disk@w50060e80058c7b10,1:a

The server is ready to be booted with the boot command.

{0} ok boot

First list the NIC installed and active on the Solaris 9 server.

# ifconfig -a
lo0: flags=1000849 mtu 8232 index 1
inet 127.0.0.1 netmask ff000000
hme0: flags=1000843 mtu 1500 index 2
inet 162.10.1.11 netmask ffff0000 broadcast 162.10.255.255
ether 8:0:20:c4:8c:87

There is one network card installed and active, hme0.

# ndd -get /dev/hme link_mode
1

Interpretation:
0 — half-duplex
1 — full-duplex

# ndd -get /dev/hme link_speed
1

Interpretation:
0 — 10 Mbit
1 — 100 Mbit
1000 — 1 Gbit

To query a different NIC, such as hme1, set the “instance” to 1, and then perform the link_mode and link_speed queries above.

# ndd -set /dev/hme instance 1

The ndd commands above must be run as root.

The first thing to do is determine the available disk and slice. Use the format command to list the disks and slices.

# format
Searching for disks…done

AVAILABLE DISK SELECTIONS:
0. c0t0d0
/pci@0,600000/pci@0/pci@0/scsi@0/sd@0,0
1. c0t1d0
/pci@0,600000/pci@0/pci@0/scsi@0/sd@1,0
2. c3t60060E8005652C000000652C00002100d0
/scsi_vhci/ssd@g60060e8005652c000000652c00002100
Specify disk (enter its number):

# zpool create epool c3t60060E8005652C000000652C00002100d0s7

This will create the zfs pool called epool on disk c3t60060E8005652C000000652C00002100d0, slice 7.

Confirm that the pool was created with the command zpool list.

bash-3.00# zpool list
NAME SIZE USED AVAIL CAP HEALTH ALTROOT
epool 472G 174K 472G 0% ONLINE -
rpool 25.5G 5.41G 20.1G 21% ONLINE -

Create a snapshot of /export and /export/home. The -r option created a recursive snapshot, so you do not have to manually create the snapshot for /export/home as well.

# zfs snapshot -r rpool/export@snapshot

Confirm that the snapshots were crearted, they are rpool/export@snapshot and rpool/export/home@snapshot.

# zfs list
NAME USED AVAIL REFER MOUNTPOINT
epool 72K 465G 21K /epool
rpool 7.41G 17.7G 97K /rpool
rpool/ROOT 4.41G 17.7G 21K legacy
rpool/ROOT/zfsboot 4.41G 17.7G 4.35G /
rpool/ROOT/zfsboot/var 69.2M 17.7G 69.2M /var
rpool/dump 1.00G 17.7G 1.00G -
rpool/export 44K 17.7G 23K /export
rpool/export@snapshot 0 – 23K -
rpool/export/home 21K 17.7G 21K /export/home
rpool/export/home@snapshot 0 – 21K -
rpool/swap 2G 19.7G 16K -

Now restore the snapshots to the new zfs pool called epool.

# zfs send rpool/export@snapshot | zfs receive epool/export
# zfs send rpool/export/home@snapshot | zfs receive epool/export/home

Confirm that the snapshots were restored to epool.

# zfs list
NAME USED AVAIL REFER MOUNTPOINT
epool 134K 465G 23K /epool
epool/export 44K 465G 23K /epool/export
epool/export@snapshot 0 – 23K -
epool/export/home 21K 465G 21K /epool/export/home
epool/export/home@snapshot 0 – 21K -
rpool 7.41G 17.7G 97K /rpool
rpool/ROOT 4.41G 17.7G 21K legacy
rpool/ROOT/zfsboot 4.41G 17.7G 4.35G /
rpool/ROOT/zfsboot/var 69.2M 17.7G 69.2M /var
rpool/dump 1.00G 17.7G 1.00G -
rpool/export 44K 17.7G 23K /export
rpool/export@snapshot 0 – 23K -
rpool/export/home 21K 17.7G 21K /export/home
rpool/export/home@snapshot 0 – 21K -
rpool/swap 2G 19.7G 16K -

Now you need to redirect the mount points for /export and /export/home. They are still pointing to rpool/export and rpool/export/home. List the mount points with the zfs mount command.

# zfs mount
rpool/ROOT/zfsboot /
rpool/ROOT/zfsboot/var /var
epool /epool
rpool/export /export
rpool/export/home /export/home
rpool /rpool
epool/export /epool/export
epool/export/home /epool/export/home

Unmount /export and /export/home becasue they are currently mounted.

# zfs unmount /export/home
# zfs unmount /export

Now redirect the mount point /export to epool/export.

# zfs set mountpoint=/export epool/export

Redirect the mount point /export/home to epool/export/home.

# zfs set mountpoint=/export/home epool/export/home

Confirm the changes.

# zfs mount
rpool/ROOT/zfsboot /
rpool/ROOT/zfsboot/var /var
epool /epool
rpool /rpool
epool/export /export
epool/export/home /export/home

Reboot the server for the changes to take effect.

After the server restarts list the filesystems to confirm the changes.

# zfs list
NAME USED AVAIL REFER MOUNTPOINT
epool 219K 465G 21K /epool
epool/export 84K 465G 23K /export
epool/export@snapshot 21K – 23K -
epool/export/home 40K 465G 22K /export/home
epool/export/home@snapshot 18K – 21K -
rpool 7.42G 17.7G 99K /rpool
rpool/ROOT 4.42G 17.7G 21K legacy
rpool/ROOT/zfsboot 4.42G 17.7G 4.35G /
rpool/ROOT/zfsboot/var 69.3M 17.7G 69.3M /var
rpool/dump 1.00G 17.7G 1.00G -
rpool/export 75K 17.7G 23K /export
rpool/export@snapshot 16K – 23K -
rpool/export/home 36K 17.7G 21K /export/home
rpool/export/home@snapshot 15K – 21K -
rpool/swap 2G 19.7G 16K -

You need to delete the old /export and /export/home which are pointing to rpool/export and rpool/export/home respectively.

# zfs destroy -r rpool/export

Confirm the filesystems were deleted.

# zfs list
NAME USED AVAIL REFER MOUNTPOINT
epool 219K 465G 21K /epool
epool/export 84K 465G 23K /export
epool/export@snapshot 21K – 23K -
epool/export/home 40K 465G 22K /export/home
epool/export/home@snapshot 18K – 21K -
rpool 7.42G 17.7G 99K /rpool
rpool/ROOT 4.42G 17.7G 21K legacy
rpool/ROOT/zfsboot 4.42G 17.7G 4.35G /
rpool/ROOT/zfsboot/var 69.3M 17.7G 69.3M /var
rpool/dump 1.00G 17.7G 1.00G -
rpool/swap 2G 19.7G 16K -

Delete the snapshots to free up space in the zfs pool rpool.

# zfs destroy epool/export@snapshot
# zfs destroy epool/export/home@snapshot

Confirm the snapshots were deleted.

# zfs list
NAME USED AVAIL REFER MOUNTPOINT
epool 153K 465G 21K /epool
epool/export 45K 465G 23K /export
epool/export/home 22K 465G 22K /export/home
rpool 7.42G 17.7G 99K /rpool
rpool/ROOT 4.42G 17.7G 21K legacy
rpool/ROOT/zfsboot 4.42G 17.7G 4.35G /
rpool/ROOT/zfsboot/var 69.3M 17.7G 69.3M /var
rpool/dump 1.00G 17.7G 1.00G -
rpool/swap 2G 19.7G 16K -

I will explain how to create a ZFS root pool snapshot and the steps to restore it to new hardware. This is a tried and tested procedure which I created. I could not find a complete documentation to do this on the internet, this took me a few days to figure out.

First create a NFS share on the remote server to store the snapshots. Let’s call this server remote-server.

For example create a filesystem.

# zfs create rpool/snaps

Share rpool/snaps.

# zfs set sharenfs=rw rpool/snaps

Check to see if share created properly.

# share
- /rpool/snaps rw “” “”

To create a recursive snapshot of the root pool on the server you wish to backup, logon as root. For this exercise let’s call this server prod-server.

List the current ZFS filesystems.

# zfs list
NAME USED AVAIL REFER MOUNTPOINT
rpool 7.43G 41.1G 97K /rpool
rpool/ROOT 4.43G 41.1G 21K legacy
rpool/ROOT/s10s_u8wos_08a 4.43G 41.1G 4.36G /
rpool/ROOT/s10s_u8wos_08a/var 69.1M 41.1G 69.1M /var
rpool/dump 1.00G 41.1G 1.00G –
rpool/export 44K 41.1G 23K /export
rpool/export/home 21K 41.1G 21K /export/home
rpool/swap 2G 43.1G 16K –

Create recursive snapshots of the root pool.

# zfs snapshot –r rpool@backup

List the ZFS filesystems, you should see the snapshots.

# zfs list
NAME USED AVAIL REFER MOUNTPOINT
rpool 7.43G 41.1G 97K /rpool
rpool@backup 0 – 97K –
rpool/ROOT 4.43G 41.1G 21K legacy
rpool/ROOT@backup 0 – 21K –
rpool/ROOT/s10s_u8wos_08a 4.43G 41.1G 4.36G /
rpool/ROOT/s10s_u8wos_08a@backup 244K – 4.36G –
rpool/ROOT/s10s_u8wos_08a/var 69.2M 41.1G 69.1M /var
rpool/ROOT/s10s_u8wos_08a/var@backup 40.5K – 69.1M –
rpool/dump 1.00G 41.1G 1.00G –
rpool/dump@backup 0 – 1.00G –
rpool/export 44K 41.1G 23K /export
rpool/export@backup 0 – 23K –
rpool/export/home 21K 41.1G 21K /export/home
rpool/export/home@backup 0 – 21K –
rpool/swap 2.00G 43.1G 16K –
rpool/swap@backup 0 – 16K –

Send the root pool snapshot to remote-server.

#Zfs send –Rv rpool@backup > /net/remote-server/rpool/snaps/rpool.backup

On the server you wish to restore the snapshot to, boot up with the Solaris installation CD in single user mode.

ok boot cdrom –s

Configure the NIC

# ifconfig bge0 10.55.99.22 netmask 255.255.0.0 up

Mount NFS share on remote-server using remote server’s IP.

# mount -f nfs 10.55.99.33:/rpool/snaps /mnt

If the disk drive is new then ensure it is labeled as SMI and a slice 0 exist. This slice needs to be big enough to restore to. Use format –e command to create the slices and label the disk.

#format –e
Select disk number, partition, print.
A SMI labeled disk should look similar to the below example.

Current partition table (original):
Total disk cylinders available: 65533 + 2 (reserved cylinders)

Part Tag Flag Cylinders Size Blocks
0 root wm 0 – 9262 49.43GB (9263/0/0) 103652970
1 unassigned wm 0 0 (0/0/0) 0
2 backup wm 0 – 65532 349.67GB (65533/0/0) 733314270
3 unassigned wm 0 0 (0/0/0) 0
4 unassigned wm 0 0 (0/0/0) 0
5 unassigned wm 0 0 (0/0/0) 0
6 unassigned wm 0 0 (0/0/0) 0
7 unassigned wm 9263 – 65532 300.25GB (56270/0/0) 629661300

Create the ZFS root pool. C0t0d0 is the disk and the S0 at the end is slice 0.

#zpool create -f -o failmode=continue -R /a -m legacy -o cachefile=/etc/zfs/zpool.cache rpool c0t0d0s0

Check to see if the pool was created.

# zpool list
NAME SIZE USED AVAIL CAP HEALTH ALTROOT
rpool 49.2G 5.45G 43.8G 11% ONLINE –

Restore the root pool snapshot. This step will take some time depending on the size of the snapshot.

# cat /mnt/rpool.backup | zfs receive -Fdu rpool

Check to see if the restore was successful.

# zfs list
NAME USED AVAIL REFER MOUNTPOINT
rpool 5.43G 128G 97K /a/rpool
rpool@backup 0 – 97K –
rpool/ROOT 4.43G 128G 21K legacy
rpool/ROOT@backup 0 – 21K –
rpool/ROOT/s10s_u8wos_08a 4.43G 128G 4.36G /a
rpool/ROOT/s10s_u8wos_08a@backup 0 – 4.36G –
rpool/ROOT/s10s_u8wos_08a/var 69.1M 128G 69.1M /a/var
rpool/ROOT/s10s_u8wos_08a/var@backup 0 – 69.1M –
rpool/dump 1.00G 128G 1.00G –
rpool/dump@backup 0 – 1.00G –
rpool/export 44K 128G 23K /a/export
rpool/export@backup 0 – 23K –
rpool/export/home 21K 128G 21K /a/export/home
rpool/export/home@backup 0 – 21K –
rpool/swap 16K 128G 16K –
rpool/swap@backup 0 – 16K –

Set the bootfs property on the root pool BE.

#zpool set bootfs=rpool/ROOT/s10s_u8wos_08a rpool

Determine the platform name, you will need it for the next step.

# uname -i
SUNW,SPARC-Enterprise

Install the boot blocks on the new disk, insert the result of uname -i. On a SPARC server.

#installboot -F zfs /usr/platform/SUNW,SPARC-Enterprise/lib/fs/zfs/bootblk /dev/rdsk/ c0t0d0s0

On an x86 server. Note, I only tested my procedures on a SPARC server, but it should work on a x86 server as well.

# installgrub /boot/grub/stage1 /boot/grub/stage2 /dev/rdsk/c0t0d0s0

You are done, reboot the system.

#init 6

If the server does not boot up, then ensure that the disk you choose is the default boot device. If you need to find out how to do that then see this article, http://www.gamescheat.ca/2010/03/changing-the-default-boot-device-on-a-sparc-server/.

List the bootable devices with the devalias command,

ok devalias
cdrom /pci@0,600000/pci@0/pci@0/scsi@0/disk@4,0:f
net /pci@0,600000/pci@0/pci@1/pci@0/network@4
disk /pci@0,600000/pci@0/pci@0/scsi@0/disk@0
name aliases

List the current boot device.

ok printenv boot-device
boot-device = disk

Change the boot device from disk to network.

ok setenv boot-device net
boot-device = net

Verify the change.

ok printenv boot-device
boot-device = net

Save the change.

ok reset-all

After the server rests, boot from he device

ok boot

# swap -l
swapfile dev swaplo blocks free
dev/dsk/c0t0d0s1 32,9 16 8425712 8425712

The output has five columns:
path
The path name for the swap area.

dev
The major/minor device number in decimal if it is a block special device; zeroes otherwise.

swaplo
The swaplow value for the area in 512-byte blocks.

blocks
The swaplen value for the area in 512-byte blocks.

free
The number of 512-byte blocks in this area that are not currently allocated.

To find the status of total swap usage use the -s option.

# swap -s
total: 180928k bytes allocated + 28456k reserved = 209384k used, 10691424k available

List the status of all the swap areas. The output has five columns:

path
The path name for the swap area.

dev
The major/minor device number in decimal if it is a block special device; zeroes otherwise.

swaplo
The swaplow value for the area in 512-byte blocks.

blocks
The swaplen value for the area in 512-byte blocks.

free
The number of 512-byte blocks in this area that are not currently allocated.

The total amount of swap space in bytes currently allocated for use as backing store.

reserved
The total amount of swap space in bytes not currently allocated, but claimed by memory mappings for possible future use.

used
The total amount of swap space in bytes that is either allocated or reserved.

available
The total swap space in bytes that is currently available for future reservation and allocation.

These numbers include swap space from all configured swap areas as listed by the -l option, as well swap space in the form of physical memory.

Add the ip address and server name to the host file.

vi /etc/hosts
10.0.0.10 server-name

Select the interface you want, your server may have more than one network interfaces.

vi /etc/hostname.bge0
10.0.0.10 netmask 255.255.255.0

Change the node name

vi /etc/nodename
server-name

Add the gateway ip address

vi /etc/defaultrouter
10.0.0.1

Add the DNS servers ip addresses

vi /etc/resolv.conf
nameserver 168.20.0.10
nameserver 168.20.10.22

Add the network address

vi /etc/netmasks
10.0.0.0 255.255.255.0

Now reboot the server.

Samba version 3.0.35
PID Username Group Machine
——————————————————————-
1962 andrew users mypc (162.10.21.100)

Service pid machine Connected at
——————————————————-
IPC$ 1962 mypc Thu Feb 11 15:06:31 2010
andrew 1962 mypc Thu Feb 11 15:06:46 2010

I needed to configure Samba installed on my Solaris 10 server and came across this great article that was posted on support.moonpoint.com/os/unix/solaris/samba-solaris10-docusp.php. Below is a copy of the instructions.

If you need to share files between a Windows and a Unix or Linux system, you can use Samba, which is free software that implements the same networking protocol that Microsoft Windows use to share files between systems. If you don’t already have it on your system, you can download it from Samba – opening windows to a wider world.
If you have a Solaris 10 system, the Samba software is likely already on the system and you will likely just need to configure it. If you type ls /usr/sfw/bin/smb* and see smblicent, smbstatus, etc. then you already have it on the system. If not, try find / -name smbclient to look for it.

If you have an account already on the Solaris system from which you wish to share files or folders you can use it and can skip to Configuring Samba support for account. But supposing you wish to create a new account specifically for sharing files to a Windows system, then follow the steps below. This explanation was written specifically for sharing files from a Xerox DocuSP system, but is generally applicable to setting up Samba on any Sun Solaris 10 system.

Adding a New Account

While logged into the system from another account, right-click somewhere on the desktop and choose Hosts then Terminal Console.

Switch to the root account, if you aren’t already logged into that account with su – root.

Add a new user account with the useradd command.

# useradd -u 40022 -g staff -c “DocuSP folder sharing” -m -s /usr/bin/bash docusp01

-u specifies the userid. It should be one not already used on the system. You can type cat /etc/passwd to see all of the accounts on the system. The UID value is the number after the second colon in each line.

-g specifies the group the account belongs to and can be staff, if you don’t have another group you wish to use.

-c allows you to put a comment in the entry.

-m specifies that a home directory be created for the new user. In this case it will be /export/home/docusp01. It is in the form base_dir/account_name where base_dir is the base directory for the new home directories and account_name is the name for the new account.

-s specifies the shell to use for the new account, which in this case the BASH shell, which is located in /usr/bin.

The final parameter on the line is the account name to create, i.e. docusp01 in this case. It will need to be 8 characters or less in length.

Set a password for the account.
# passwd docusp01

If you need to later change any parameters for the account, you can use the usermod command.

Now that you have created the account you wish to use, you can configure Samba support for that account.

Configuring Samba support for account

You need to edit /etc/sfw/smb.conf (You may wish to create a backup copy first), e.g. cp -p /etc/sfw/smb.conf /etc/sfw/smb.conf.old.

Change the line workgroup = WORKGROUP to match whatever workgroup or domain you use for your Windows systems, e.g. workgroup = SOMECOMPANY.

You can also change server string to be whatever you wish. I changed it from the default of server string = XXP240250 to the following:

server string = Xerox DocuColor 250

Choose which systems should have access to shared folders on the Solaris system by adding a hosts allow line. There is one in the smb.conf file to show you how to configure this line, but it is commented out.

# hosts allow = 192.168.1. 192.168.2.0./24 192.168.3.0/255.255.255.0 127.0.0.1

If my systems are on a 192.168.0.0/24 subnet, i.e. systems have an address of 192.168.0.x and use a subnet mask of 255.255.255.0, I could allow access from any system on the subnet by using 192.168.0.0/24 or 192.168.0. If I only want to allow access from two specific systems, say 192.168.0.25 and 192.168.0.36, plus the Solaris system itself for testing using the loopback address 127.0.0.1, I would add the following line:
hosts allow = 192.168.0.25 192.168.0.36 127.0.0.1

If you wish to set a maximum size for the log file that records information about Samba connections, find #max log size = 50, remove the comment character, # and change the number to whaterver you wish to use, e.g. 16384 for 16 MB.

I also uncommented the line that specifies where the log file is located, but changed the line from:

#log file = /usr/local/samba/var/log.%m

To the following line instead:

log file = /var/log/samba/log.%m

I also created a samba directory under /var/log to hold the Samba logs and then changed its protection so only root can view the logs.

# mkdir /var/log/samba
# chmod 700 /var/log/samba

At the end of the smb.conf file, you can add information about the folder you wish to share. In this case for the DocuSP system, I want to share the folder /var/spool/XRXnps/saved. I want to make it browseable from the Windows systems to which I am granting access, but I don’t want them to be able to add files or modify files in the directory. And I only want to grant acces to the new account, docusp01, that I created, so I added the following lines:

# DocuSP folder shared read-only to Windows systems
[DocuSP]
comment = DocuSP XRXnps saved
path = /var/spool/XRXnps/saved
valid users = docusp01
browseable = yes
public = yes
writable = no
printable = no

The valid users = docusp01 will limit access to only the docusp01 account I created.

I need to grant access to the /var/spool/XRXnps/saved directory. Xerox recommends granting read, write, execute access to everyone for this directory using the command below:

# chmod 777 /var/spool/XRXnps/saved

I issued that command, though I am still restricting access through /etc/smb.conf. The default permission on the directory prior to issuing that command is shown below:

# ls -ld /var/spool/XRXnps/saved
drwxr-xr-x 43 root other 2048 Mar 23 12:55 /var/spool/XRXnps/saved

When I checked while setting up the system, Samba appeared to be running, i.e., I saw smbd running and when I checkd the services on the system, I saw Samba listed:

# ps -ef | grep smb | grep -v grep
root 1005 1 0 Mar 23 ? 0:00 /usr/sfw/sbin/smbd -D
root 1007 1005 0 Mar 23 ? 0:00 /usr/sfw/sbin/smbd -D
# svcs | grep samba
legacy_run Mar_23 lrc:/etc/rc3_d/S90samba

So I stopped and restarted it.

# /etc/init.d/samba stop
# /etc/init.d/samba start

Once I did that, I saw the system appear as SIP-8 in the workgroup used by the company for whom I was setting it up, when I checked on what systems were in the workgroup. But you may not always see the system show up immediately.

C:\>net view /domain:frostinc
Server Name Remark

——————————————————————————-
\\FROSTINC26 Cindy’s Dell Dimension 3000
\\PCWORKSTATION Gateway Profile 4
\\SIP-8 Xerox DocuColor 250
The command completed successfully.

But to allow access from a Windows PC using the account I created, I needed to set a Samba password for the docusp01 account I created on the Solaris system. You can use the smbpasswd command to create a Samba password for the account that can be used to access the shared folder from a Windows system. This can be the same as the one you use for logging into the account under Solaris, but doesn’t necessarily have to be the same as the password for the account.

# /usr/sfw/bin/smbpasswd -a docusp01
New SMB: password:

The -a specifies that you are adding a new account password rather than changing an existing one. When I first ran the command I received the error message shown below.

# /usr/sfw/bin/smbpasswd docusp01
New SMB password:

Retype new SMB password:

startsmbfilepwent_internal: file /opt/XRXnps/XRXsamba/private/smbpasswd did not exist. Couldn’t create new one. Error was: No such file or directoryUnable to open passdb database.
Failed to find entry for user docusp01.
Failed to modify password entry for user docusp01

In the /etc/sfw/smb.conf file, the lines below appeared:

# You may wish to use password encryption. Please read
# ENCRYPTION.txt, Win95.txt and WinNT.txt in the Samba documentation.
# Do not enable this option unless you have read those documents
encrypt passwords = yes

smb passwd file = /opt/XRXnps/XRXsamba/private/smbpasswd

Once I realized that the directory XRXsamba did not exist, I created it and the private subdirectory beneath it.

# ls /opt/XRXnps/XRXsamba
/opt/XRXnps/XRXsamba: No such file or directory
# mkdir /opt/XRXnps/XRXsamba
# mkdir /opt/XRXnps/XRXsamba/private

I was then able to use the smbpasswd command without the prior error occurring.

# /usr/sfw/bin/smbpasswd -a docusp01
New SMB password:

Retype new SMB password:

startsmbfilepwent_internal: file /opt/XRXnps/XRXsamba/private/smbpasswd did not
exist. File successfully created.
Added user docusp01.

And when I looked in the smbd.log file, I saw an entry indicating the encrypted password file had been created.

# cat /var/log/samba/log.smbd
[2007/03/25 22:27:58, 0] passdb/pdb_smbpasswd.c:(195)
startsmbfilepwent_internal: file /opt/XRXnps/XRXsamba/private/smbpasswd did not exist. File successfully created.

I was then able to access the shared folder from one of the Windows PCs to which I had granted access to the shared folder by listing its IP address in /etc/smb.conf. To access the shared folder on the Solaris system from the PC, I used the username of SIP-8\docusp01 and the password for that account.

C:\>net view \\sip-8
Shared resources at \\sip-8

Xerox DocuColor 250

Share name Type Used as Comment

——————————————————————————-
DocuSP Disk (UNC) DocuSP XRXnps saved
The command completed successfully.

And from the Solaris system, I could issue the smbstatus command and see the access from the Windows PC.

# /usr/sfw/bin/smbstatus
creating lame upcase table
creating lame lowcase table

Samba version 3.0.11
PID Username Group Machine
——————————————————————-
8446 docusp01 staff pcworkstation (192.168.0.26)

Service pid machine Connected at
——————————————————-
DocuSP 8446 pcworkstation Sun Mar 25 22:43:58 2007

No locked files

And I saw entries appear in /var/log/samba/log.pcworkstation for the connection from the PC.

I could also issue the smbclient command from the Solaris system to view shared directories on the Windows PC.

# /usr/sfw/bin/smbclient -L 192.168.0.25 -U Smith

For the smbclient command, the -L parameter specified the address of the PC and the -U parameter specified an account on that Windows PC, for which I received a password prompt

I also set up Samba on a second DocuSP system, SIP-3. When I created a shortcut on the Windows PC’s desktop, I had to specify the IP address of the Solaris DocuSP system instead of SIP-3, e.g. I had to use \\192.168.0.36\docusp as the location. When prompted for the userid and password, I entered \\SIP-3\docusp-1 as the username and the password for that account set up through smbpasswd on the Solaris system. It was accepted and I was able to access files on the DocuSP system through the shortcut. But the system wasn’t showing up when I entered the command net view /domain:somecompany.

You can also manage Samba via the web console, the url for this interface is http://servername:901

Many SEOs suggest that the first thing you should do is to get your site listed in DMOZ to help boot your page rank. There are just as many SEOs who claim that getting your site listed in DMOZ is worthless. Strangely there are many sites listed in the DMOZ that should have never been approved. There are broken links, links that redirect to different site, and sites that looks like crap.