Mounting iSCSI LUN on RedHat 6

NetApp preparing

iSCSI service is started by default. You can check the status nad get required later info by:

netapp> iscsi status        
iSCSI service is running
netapp> iscsi nodename
iSCSI target nodename:

Preparing RedHat 6 client

Install iscsi-initiator-utils package if not installed already. Check for /etc/iscsi/initiatorname.iscsi file contain your _unique_ nodename. This file generated during iscsi-initiator-utils RPM installation.

# cat /etc/iscsi/initiatorname.iscsi

If you are unhappy with content, new string can be generated again using iscsi-iname command.

Discover your netapp by:

# iscsiadm --mode discoverydb --type sendtargets --portal netappdnsname --discover
Starting iscsid:                                           [  OK  ],2000,2000,1009
# iscsiadm --mode node,2000,2000,1009

Netapp steps

Create initiator group using info above, create volume with lun, map lun:

netapp> igroup create -i -t linux vorh6t01
netapp> igroup show
    vorh6t01 (iSCSI) (ostype: linux): (not logged in)
netapp> vol create vorh6t -s none aggr0 20g
Creation of volume 'vorh6t' with size 20g on containing aggregate
'aggr0' has completed.
netapp> qtree create /vol/vorh6t/luns
netapp> lun create -s 20g -t linux -o noreserve /vol/vorh6t/luns/L0
netapp> lun map /vol/vorh6t/luns/L0 vorh6t01 0
netapp> lun show
        /vol/vorh6t/luns/L0           20g (21474836480)   (r/w, online, mapped)
netapp> lun show -m
LUN path                            Mapped to          LUN ID  Protocol
/vol/vorh6t/luns/L0                 vorh6t01                0     iSCSI

Back to client

Verify, that iSCSI services will started automatically after reboot and restart them (this will also rescan devices):

# chkconfig --list | grep scsi
iscsi           0:off   1:off   2:off   3:on    4:on    5:on    6:off
iscsid          0:off   1:off   2:off   3:on    4:on    5:on    6:off
# /etc/init.d/iscsi stop ; /etc/init.d/iscsid stop ; /etc/init.d/iscsid start ; /etc/init.d/iscsi start 
Stopping iscsi:                                            [  OK  ]
Stopping iscsid: 
Starting iscsid:                                           [  OK  ]
Starting iscsi:                                            [  OK  ]

Check by # cat /proc/scsi/scsi if you see new added LUN. Ooops! Three LUNs ! That is because we have three TCP/IP sessions over different paths to storage. It is looks like multipath should be used.

Install device-mapper-multipath package if not yet installed. Check /dev/disk/by-id/ for wwid of your disk:

# ll /dev/disk/by-id/
lrwxrwxrwx 1 root root  9 Aug 30 13:45 scsi-360a980006466583957346d3769486e48 -> ../../sdd

Edit /etc/multipath.conf and add usefull alias for it:

# cat /etc/multipath.conf
defaults {
        user_friendly_names yes

multipaths {
        multipath {
                wwid    360a980006466583957346d3769486e48
                alias   nlun0

Restart multipath and rebuild it's configureation:

# /etc/init.d/multipathd restart
Stopping multipathd daemon:                                [FAILED]
Starting multipathd daemon:                                [  OK  ]
# multipath -F
# multipath 
create: nlun0 (360a980006466583957346d3769486e48) undef NETAPP,LUN
size=20G features='3 queue_if_no_path pg_init_retries 50' hwhandler='0' wp=undef
`-+- policy='round-robin 0' prio=2 status=undef
  |- 12:0:0:0 sdc 8:32 undef ready running
  |- 13:0:0:0 sde 8:64 undef ready running
  `- 14:0:0:0 sdd 8:48 undef ready running

Create VG, LV, FS and mount, add it to /etc/fstab with _netdev option:

# pvcreate --dataalignment=4k /dev/mapper/nlun0
  Writing physical volume data to disk "/dev/mapper/nlun0"
  Physical volume "/dev/mapper/nlun0" successfully created
# vgcreate datavg /dev/mapper/nlun0
  Volume group "datavg" successfully created
# lvcreate -n export -L16g /dev/datavg 
  Logical volume "export" created
# mkfs.ext4 -j -m0 /dev/datavg/export
# mkdir /export ; mount -o discard /dev/datavg/export /export
# grep ext4 /etc/fstab
/dev/datavg/export /export ext4 _netdev,discard 2 2

Restart server to see it mount everything automatically.


 # dd if=/dev/zero of=/export/10g.file bs=1024k count=10240
10240+0 records in
10240+0 records out
10737418240 bytes (11 GB) copied, 60.3137 s, 178 MB/s
# sync
# dd if=/export/10g.file of=/dev/null bs=1024k
10240+0 records in
10240+0 records out
10737418240 bytes (11 GB) copied, 177.623 s, 60.5 MB/s

netapp> priv set diag;lun alignment show
Warning: These diagnostic commands are for use by NetApp
         personnel only.
                Multiprotocol type: linux
                Alignment: aligned
                Write alignment histogram percentage: 100, 0, 0, 0, 0, 0, 0, 0
                Read alignment histogram percentage: 99, 0, 0, 0, 0, 0, 0, 0
                Partial writes percentage: 0
                Partial reads percentage: 0
netapp> df -g vorh6t
Filesystem               total       used      avail capacity  Mounted on
/vol/vorh6t/              16GB       10GB        5GB      64%  /vol/vorh6t/
/vol/vorh6t/.snapshot        4GB        0GB        4GB       0%  /vol/vorh6t/.snapshot

# df /export
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/datavg-export   16G   11G  5.6G  65% /export
# rm /export/10g.file 
rm: remove regular file `/export/10g.file'? y
# sync
# df /export
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/datavg-export   16G  172M   16G   2% /export

netapp> df -g vorh6t
Filesystem               total       used      avail capacity  Mounted on
/vol/vorh6t/              16GB        0GB       15GB       1%  /vol/vorh6t/
/vol/vorh6t/.snapshot        4GB        0GB        4GB       0%  /vol/vorh6t/.snapshot

As you can see, even promissed thin provisioning works !

Updated on Sun Sep 2 13:38:31 IDT 2012 More documentations here