RH: Multipathing NetApp LUN on RHEL 5.3
- Make sure you have the packages needed by RHEL, otherwise install them.
rpm -q device-mapper
rpm -q device-mappermultipath
yum install device-mapper
yum install device-mapper-multipath - Install QLogic Drivers if needed, or utilize RHEL drivers. In my case I am using HP C-Class blades with Qlogic HBA cards. HP drivers can be found at the HP site, driver is called hp_sansurfer. I am utilizing RHEL built in drivers, but you can install the HP/QLogic drivers as follows:
rpm -Uvh hp_sansurfer-5.0.1b45-1.x86_64.rpm
- If Qlogic HBA, install the SanSurfer CLI, this is very useful program for doing things with QLogic HBA cards, it can be downloaded at QLogic website, install as follows:
rpm -Uvh scli-1.7.3-14.i386.rpm
- Install NetApp Host Utilities Kit, the package is a tar.gz file, you can find it at the now site http://now.netapp.com.Open it and run the install shell script
netapp_linux_host_utilities_5_0.tar.gz
- Once Everything is installed on the host, create the LUN and ZONE it from the NetApp, Brocade(SAN Fabric),Host
To find your WWPNs, use the scli as follows:
# scli –i all
// Use the WWPN numbers for the iGroup and Brocade Aliases - Once it has been Zoned and mapped correctly, verify if your RHEL host can see it.
// Rescan HBA for new SAN Luns
# modprobe -r qla2xxx
# modprobe qla2xxx
// Check the kernel can see it
# cat /proc/scsi/scsi
# fdisk –lu - Utilize NetApp tools to see LUN connectivity
// Check your host and utilities see the LUNs
[root@server ~]# sanlun lun show
controller: lun-pathname device filename adapter protocol lun size lun state
NETAPPFILER: /vol/servervol/serverlun /dev/sdf host6 FCP 100g (107374182 400) GOOD
NETAPPFILER: /vol/servervol/serverlun /dev/sda host4 FCP 100g (107374182 400) GOOD
NETAPPFILER: /vol/servervol/serverlun /dev/sde host6 FCP 100g (107374182 400) GOOD
NETAPPFILER: /vol/servervol/serverlun /dev/sdc host5 FCP 100g (107374182 400) GOOD
NETAPPFILER: /vol/servervol/serverlun /dev/sdd host5 FCP 100g (107374182 400) GOOD
NETAPPFILER: /vol/servervol/serverlun /dev/sdb host4 FCP 100g (107374182 400) GOOD - Utilize NetApp tools to check multipathing, not set yet
[root@server ~]# sanlun lun show -p
NETAPPFILER:/vol/servervol/serverlun (LUN 0) Lun state: GOOD
Lun Size: 100g (107374182400) Controller_CF_State: Cluster Enabled
Protocol: FCP Controller Partner: NETAPPFILER2
Multipath-provider: NONE
——— ———- ——- ———— ——————————————— —————
sanlun Controller Primary Partner
path Path /dev/ Host Controller Controller
state type node HBA port port
——— ———- ——- ———— ——————————————— —————
GOOD primary sdf host6 0c —
GOOD secondary sda host4 — 0c
GOOD secondary sde host6 — 0c
GOOD secondary sdc host5 — 0d
GOOD primary sdd host5 0d —
GOOD primary sdb host4 0c —Time to configure multipathing
- Start the multipath daemon
# service multipathd start
- Find you WWID, this will be needed in the configuration if you want to alias it.
Comment out the blacklist in the default /etc/multipath.conf, otherwise you will NOT see anything.#blacklist {
# devnode “*”
#}// Show your devices and paths, and record the WWID of the LUN
# multipath -v3
…
…
===== paths list =====
uuid hcil dev dev_t pri dm_st chk_st vend/pr
360a98000486e576748345276376a4d41 4:0:0:0 sda 8:0 1 [undef][ready] NETAPP,
360a98000486e576748345276376a4d41 4:0:1:0 sdb 8:16 4 [undef][ready] NETAPP,
360a98000486e576748345276376a4d41 5:0:0:0 sdc 8:32 1 [undef][ready] NETAPP,
360a98000486e576748345276376a4d41 5:0:1:0 sdd 8:48 4 [undef][ready] NETAPP,
360a98000486e576748345276376a4d41 6:0:0:0 sde 8:64 1 [undef][ready] NETAPP,
360a98000486e576748345276376a4d41 6:0:1:0 sdf 8:80 4 [undef][ready] NETAPP,
…
… - Now you are ready to configure /etc/multipath.conf
Exclude (blacklist) all the devices that do not correspond to any
LUNs configured on the storage controller and which are mapped to
your Linux host. There are 2 methods:
Block by WWID
Block by devnode
In this case I am blocking by devnode since I am using HP and know my devnode RegEx
Also configure the device and alias(optional).
The full /etc/multipath.conf will look like this:defaults
{
user_friendly_names yes
max_fds max
queue_without_daemon no
}
blacklist {
###devnode “*”
devnode “^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*”
devnode “^hd[a-z]”
devnode “^cciss!c[0-9]d[0-9]*” # Note the cciss, usual in HP
}
multipaths {
multipath {
wwid 360a98000486e57674834527533455570 # You found this
alias netapp # This is how you want to name the device in your host
# server LUN on NETAPPFILER
}
}
devices
{
device
{
vendor “NETAPP”
product “LUN”
getuid_callout “/sbin/scsi_id -g -u -s /block/%n”
prio_callout “/sbin/mpath_prio_ontap /dev/%n”
features “1 queue_if_no_path”
hardware_handler “0”
path_grouping_policy group_by_prio
failback immediate
rr_weight uniform
rr_min_io 128
path_checker directio
flush_on_last_del yes
}
} - Restart multipath and make sure it starts automatically:
// Restart multipath
# service multipathd restart
// Add to startup
# chkconfig –add multipathd
# chkconfig multipathd on - Verify multipath is working
//RHEL tools
[root@server scli]# multipath -l
netapp (360a98000486e576748345276376a4d41) dm-2 NETAPP,LUN
[size=100G][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=0][active]
\_ 4:0:1:0 sdb 8:16 [active][undef]
\_ 5:0:1:0 sdd 8:48 [active][undef]
\_ 6:0:1:0 sdf 8:80 [active][undef]
\_ round-robin 0 [prio=0][enabled]
\_ 4:0:0:0 sda 8:0 [active][undef]
\_ 5:0:0:0 sdc 8:32 [active][undef]
\_ 6:0:0:0 sde 8:64 [active][undef]//NetApp utilities Tool
[root@server scli]# sanlun lun show -p
NETAPPFILER:/vol/servervol/serverlun (LUN 0) Lun state: GOOD
Lun Size: 100g (107374182400) Controller_CF_State: Cluster Enabled
Protocol: FCP Controller Partner: NETAPPFILER2
DM-MP DevName: netapp (360a98000486e576748345276376a4d41) dm-2
Multipath-provider: NATIVE
——— ———- ——- ———— ——————————————— —————
sanlun Controller Primary Partnerstate type node HBA port port
——— ———- ——- ———— ——————————————— —————
GOOD primary sdb host4 0c —
GOOD primary sdd host5 0d —
GOOD primary sdf host6 0c —
GOOD secondary sda host4 — 0c
GOOD secondary sdc host5 — 0d
GOOD secondary sde host6 — 0c
… - Now you can access the LUN by using the mapper
[root@server scli]# ls -l /dev/mapper
total 0
crw——- 1 root root 10, 63 Sep 12 12:32 control
brw-rw—- 1 root disk 253, 2 Sep 16 10:54 netapp
brw-rw—- 1 root disk 253, 0 Sep 12 16:32 VolGroup00-LogVol00
brw-rw—- 1 root disk 253, 1 Sep 12 12:32 VolGroup00-LogVol01 - Format it to your liking and mount it
# mkdir /mnt/netapp
# mkfs -t ext3 /dev/mapper/netapp
# mount /dev/mapper/netapp /mnt/netapp/
//verify it mounted
# mount
…
…
/dev/mapper/netapp on /mnt/netapp type ext3 (rw)
… - If you want it to be persistent after reboots put it on /etc/fstab and make sure multipathd start automatically.
# cat /etc/fstab
…
…
/dev/mapper/netapp /mnt/netapp ext3 defaults 0 0 - If possible reboot to check it mounts correctly after reboots.