core-iscsi howto v1.1 11/30/2005

	Topics covered:

	I) Introduction
	II) Requirements
	III) Glossary
	1) Installation
		A) Getting iscsi-initiator-core
		B) Installing iscsi-initiator-core
		C) Getting iscsi-initiator-core-tools
		D) Installing iscsi-initiator-core-tools
	2) Configuring iscsi-initiator-core
		A) /etc/initiatorname.iscsi
		B) /etc/sysconfig/initiator
		C) /etc/sysconfig/initiator_auth
		D) /etc/sysconfig/iscsi_device_maps
	3) Typical iSCSI Channel Management Scenarios
		A) Using /etc/rc.d/init.d/initiator
		B) Starting an iSCSI Channel
		C) Stopping an iSCSI Channel
		D) Pausing an iSCSI Channel
	4) Advanced iSCSI Channel Management Scenarios
		A) Trunking
		B) dm-multipath
		C) udev
	Appendix A) Manual Pages
		A) man 5 initiator-ctl
		B) man 5 initiator
		C) man 5 initiator_auth
		D) man 5 iscsi_device_maps

I) Introduction

	iSCSI is changing the way that networked machines access block level storage over IP
	driven networks.  This document is going to explain, step-by-step how you can create
	your own fully functional iSCSI Initiator Node, how to define the SAN topography for
	iSCSI Channels, logical units (LUNS) that contain storage resources defined within those
	iSCSI Channels, and typical management scenarios iSCSI SAN adminstators will encounter.
	This is exercise starts by installing the iscsi-initiator-core.org storage stack and
	iscsi-initiator-core-tools suite.  Getting iscsi-initiator-core.org module loaded,
	configuration files defined, and finally logging into a iSCSI Target Node.
	
	This document is broken into multiple sections that attempt to describe the following
	steps to achieve this goal:
	
	Installation, Configuration, Management, Advanced Scenarios and Manual Pages.
	
	This document is best read from start to finish the first time (come on, its a short
	document :-), and can be used as a reference though the manual page after the first
	iSCSI Target Node bringup has completed successfully.

II) System Requirements

	32-64 bit, Big/Little Endian, UP/SMP machines running Linux 2.6.11+

III) Glossary

	iSCSI: Internet Small Computer Systems Interface

	IETF: Internet Engineering Task Force

	SAN: Storage Area Network

	iSCSI Initiator Node:
	
		A) iSCSI Channel
		B) iSCSI Logical Units
		C) Persisant iSCSI LUN Mapping

	iSCSI Target Node:

		A) iSCSI Network Portals
		B) iSCSI Target Portal Groups
		C) iSCSI Logical Units


Section 1) Getting iscsi-initiator-core and iscsi-initiator-core-tools installed

	A) Installing core-iscsi

	   The core-iscsi stack is distributed in source form as a standalone sourcetree
	   that can complile against most modern 2.6 kernels.  The tarball is located at the
	   following location:
	   
	   http://www.kernel.org/pub/linux/kernel/people/nab/iscsi-initiator-core/
	   
	   After unpacking a release, change directory to the tarball and run the following:

	   	# make initiator KERNEL_DIR=/usr/src/linux-kernel-source-2.6
		# make install

	   This will install the core-iscsi stack into your /lib/modules/$KERNEL directory,
	   so the module (.ko) will be available to modprobe.

	B) Installing core-iscsi-tools

	   The core-iscsi tools package is distributed in source form as a standalone
	   sourcetree that contains configuration file examples, management scripts, and
	   userspace utilites.  The tarball is located at the following location:

	   http://www.kernel.org/pub/linux/utils/storage/iscsi/
	   
	   After unpacking a core-iscsi-tools release, change directory to the tarball
	   and run the following:

	   	# make install

	   This will install the core-iscsi-tools package into various locations on your
	   local system.

Section 2) Configuring iscsi-initiator-core

	A) /etc/initiatorname.iscsi
	
	   The first step for configuring your newly installed iSCSI Initiator Core Stack
	   to to generate a unique identifer that seperates your iSCSI Initiator Node from
	   other iSCSI Initiator Nodes.  This is known as an iSCSI Initiator Name.  The
	   iSCSI standard defines that this name stays with the iSCSI Initiator Node for its
	   lifetime.  A unique name for your iSCSI Initiator Node helps other iSCSI Nodes in
	   the iSCSI SAN to determine how the topography is laided out.

	   iscsi-initiator-core-tools generates a unique name for the iSCSI Initiator Node
	   and stores this string in /etc/initiatorname.iscsi.  This file is read each time the
	   iscsi-initiator-core stack is loaded into the kernel and remains persisant throughout
	   normal operation.

	B) /etc/sysconfig/initiator
	
	   The second step is defining which iSCSI Network Portals that your iSCSI Initiator Node
	   will be connecting to iSCSI Target Nodes.  This information is containing in the plain
	   text configuration file located at /etc/sysconfig/initiator.  This file also contains
	   information on various SCSI dependant parameters, as well as iSNS server location.

	   Each iSCSI Channel entry (there can be multiple CHANNEL="$CHANNEL_ID lines) defines one
	   iSCSI Target Node.  Consider the following example for iSCSI Channel 0:

	   Example 1)
	
	   # Arguments:
	   #        (REQUIRED 1st arg) iSCSI Channel (1 iSCSI Session for N Logical Units per CHANNEL=) 
	   #        (REQUIRED 2nd arg) iSCSI IPv4 Connection Count for iSCSI Session
	   #        (OPTIONAL 3rd arg) Network Interconnect Device Name (optional)
	   #        (REQUIRED 4th arg) IPv4 iSCSI Target Node Address
	   #        (REQUIRED 5th arg) iSCSI Target Service Port
	   #        (REQUIRED 6th arg) Network Interconnect         
	   #        (REQUIRED 7th arg) iSCSI Parameter Keys (seperated by ;)
	   #        (REQUIRED 8th arg) iSCSI Channel Attributes (seperated by ;)
	   #        (OPTIONAL 9th arg) Target IQN Name

	   CHANNEL="0 1 eth0 192.168.0.1 3260 0 AuthMethod=CHAP,None;MaxRecvDataSegmentLength=8192 nopout_timeout=5"

	   This example defines iSCSI Channel 0 that creates an iSCSI Session with one (1)
	   iSCSI TCP Connection to 192.168.0.1 at port 3260.

	   Important things to notice in this example for iSCSI parameters for iSCSI Channel
	   Zero (0)
	   
	   	1) In this example authetication is enabled with 'AuthMethod=CHAP,None'
	   	2) The iSCSI PDU Data Payload size is set to 8k with 'MaxRecvDataSegmentLength=8192'

	   Important things to notice in this example for iSCSI Channel attributes:
	   	
		1) The nopout ping is sent every 5 seconds.
		2) Wait 10 seconds until a nopout ping response is received before taking
		   recovery action.

	  Example 2)
	  
	  Now lets consider a more complex example for iSCSI Channel 1:

	   # Arguments:
	   #        (REQUIRED 1st arg) iSCSI Channel (1 iSCSI Session for N Logical Units per CHANNEL=)
	   #        (REQUIRED 2nd arg) iSCSI IPv4 Connection Count for iSCSI Session
	   #        (OPTIONAL 3rd arg) Network Interconnect Device Name (optional)
	   #        (REQUIRED 4th arg) IPv4 iSCSI Target Node Address
	   #        (REQUIRED 5th arg) iSCSI Target Service Port
	   #        (REQUIRED 6th arg) Network Interconnect
	   #        (REQUIRED 7th arg) iSCSI Parameter Keys (seperated by ;)
	   #        (REQUIRED 8th arg) iSCSI Channel Attributes (seperated by ;)
	   #        (OPTIONAL 9th arg) Target IQN Name

	   CHANNEL="0 1 eth0 192.168.0.1 3260 0 AuthMethod=CHAP,None;MaxRecvDataSegmentLength=8192 nopout_timeout=5"
	   CHANNEL="1 2 eth0 192.168.0.2 3260 0 AuthMethod=None;MaxRecvDataSegmentLength=32768;ImmediateData=No nopout_timeout=10, nopout_response_timeout=10"
	   CHANNEL="1 2 eth1 192.168.1.2 3260 0"

	   This second example builds on Example 2 and defines another iSCSI Channel 1 within
	   CHANNEL="1 that establishes an iSCSI Session with four (4) iSCSI/TCP Connections to
	   192.168.[0,1].2 at port 3260.

	   Important things to notice in this example is the three (3) iSCSI parameters for
	   iSCSI Channel 1.

           	1) Authentication is disabled with AuthMethod=None.
	   	2) The iSCSI PDU Data Payload size is set to 32k with 'MaxRecvDataSegmentLength=32768'.
	   	3) Immediate WRITE data is disabled.

	   Important things to notice in this example for iSCSI Channel attributes:

		1) The nopout ping is sent every 5 seconds.
		2) Wait 10 seconds until a nopout ping response is received before taking recovery action.
	
	    ---------------------------------------------------------------------------------------------------

	KEY TOPIC - As you define your iSCSI SAN Topography, assign each new iSCSI Target Node in your SAN
	            an iSCSI Channel within /etc/sysconfig/initiator.  This allows for simple iSCSI Initiator
		    Node Management who's configuration file can be copied between multiple initiators and
		    globally define iSCSI Channel 0 as a specific iSCSI Target Node with multiple communication
		    pathways.
	    
	    ---------------------------------------------------------------------------------------------------
	
	C) /etc/sysconfig/initiator_auth

	    The third step for defining your first iSCSI Target Node is to define any secrets
	    used during iSCSI login authentication for each of the iSCSI Channels defined in
	    /etc/sysconfig/initiator_auth.  Note that authetication is DISABLED by default and is
	    enabled by defining 'AuthMethod=CHAP,None' for a given iSCSI Channel in
	    /etc/sysconfig/initiator. (See example B)

		# Channel ID
		#     | Mutual Authentication
		#     | |  Client username
		#     | |    |   Client password
		#     | |    |         |       Server username
		#     | |    |         |              |       Server password
		#     | |    |         |              |              |
		AUTH="0 1 someuser somepass034 someuser_ma somepass014_ma"
		AUTH="1 0 another_user 12345"

	    Following the iSCSI Channels defined in example B), here we defined iSCSI Channel 0
	    use mutal authetication with the listed user/password combinations.  Additionally this
	    example defines iSCSI Channel 0 use only client authetication with the listed user/password
	    combination.

	    Note that these users DO NOT have to add to the local Linux system and are only used for
	    iSCSI connection authentication.

	   ---------------------------------------------------------------------------------------------------

	KEY TOPIC - Authentication is defined in /etc/sysconfig/initiator_auth on a per iSCSI Channel basis.
		    iSCSI Authentication is disabled by default, and is enabled by setting iSCSI Parameter
		    'AuthMethod=CHAP,None' for a given iSCSI Channel within /etc/sysconfig/initiator.

	   ---------------------------------------------------------------------------------------------------

	D) /etc/sysconfig/iscsi_device_maps

	    The persistent mapping of iSCSI Logical Units to mountpoints is defined within the
	    last iSCSI Initiator configuration file /etc/sysconfig/iscsi_device_maps.  As with
	    example B) and C), the iSCSI Channel abstraction defined without the previous two
	    iSCSI Initiator configuration files define where those iSCSI Channel's Logical Units
	    are mounted.  Considering the following example:

	    	#
		#       iSCSI Channel ID 
		#       | iSCSI Logical Unit Number
		#       | | Parition Number
		#       | | |     
		#       | | |   iSCSI device mount point
		#       | | |    |
		DEVICE="0 0 1 disk0a" # Channel 0, LUN 0, Partition 1 -> /iqn.target.example/disk0a
		DEVICE="0 0 2 disk0b" # Channel 0, LUN 0, Partition 2 -> /iqn.target.example/disk0b
		DEVICE="0 1 1 disk1a" # Channel 0, LUN 1, Partition 1 -> /iqn.target.example/disk1a
		DEVICE="0 2 1 disk2a" # Channel 0, LUN 2, Partition 1 -> /iqn.target.example/disk2a
		DEVICE="1 0 0 dvd0"   # Channel 1, LUN 0, Partition 0 -> /iqn.target.example/dvd0
		DEVICE="1 1 1 disk0a" # Channel 1, LUN 1, Partition 1 -> /iqn.target.example/disk0a
		DEVICE="1 1 2 disk0b" # Channel 1, LUN 1, Partition 2 -> /iqn.target.example/disk0b
		DEVICE="1 2 1 disk1a" # Channel 1, LUN 2, Partition 1 -> /iqn.target.example/disk1a

	    Just as with the previous examples, we define the same iSCSI Channels with ID Zero
	    (0) and One (1).  We define four (4) DEVICE entires for each iSCSI Channel for various
	    iSCSI Logical Unit mappings:
	    
	    Logical Units for iSCSI Channel Zero (0): (ie: the First iSCSI Target Node)

   	    	1) iSCSI LUN 0's First Partition mounts at: /iqn.2005-04.com.example.target.mipsEE:sn.1234567890/disk0a
	    	2) iSCSI LUN 0's Second Partition mounts at: /iqn.2005-04.com.example.target.mipsEE:sn.1234567890/disk0b
	    	3) iSCSI LUN 1's First Partition mounts at: /iqn.2005-04.com.example.target.mipsEE:sn.1234567890/disk1a
	    	4) iSCSI LUN 2's First Partition mounts at: /iqn.2005-04.com.example.target.mipsEE:sn.1234567890/disk2a

	    Important things to notice in this example for iSCSI parameters for iSCSI Channel
	    Zero (0):

		1) iSCSI LUN 0 defines two partitions.
		2) iSCSI LUN 1 and LUN 2 define two partitions.

	    Logical Units for iSCSI Channel One (1): (ie: The Second iSCSI Target Node)

	        1) iSCSI LUN 0's image mounts at: /iqn.2005-04.com.example.target.gekko:sn.0987654321/dvd0
		2) iSCSI LUN 1's First Partition mounts at: /iqn.2005-04.com.example.target.gekko:sn.0987654321/disk0a
		3) iSCSI LUN 1's Second Partition mounts at: /iqn.2005-04.com.example.target.gekko:sn.0987654321/disk0b
		4) iSCSI LUN 2's First Partition mounts at: /iqn.2005-04.com.example.target.gekko:sn.0987654321/disk1a

            Important things to notice in this example for iSCSI parameters for iSCSI Channel
	    One (1):

	    	1) iSCSI LUN 0 is a DVD ROM image, as defined with the partition number of 0.
		2) iSCSI LUN 1 defines two partitions.
		3) iSCSI LUN 2 defines one partitions.

	    ---------------------------------------------------------------------------------------------------

	KEY TOPIC - As you define the topography that your iSCSI Initiator Node sees, you will want to be able
		    to persisantly map iSCSI LUNs to mountpoints.  /etc/sysconfig/iscsi_device_maps uses the
		    same iSCSI Channel abstractions as /etc/sysconfig/initiator and /etc/sysconfig/initiator_auth,
		    and defines what /iqn.name/directory iSCSI Logical Units of storage devices from iSCSI Channels
		    should be mounted.

	    ---------------------------------------------------------------------------------------------------

Section 3) Typical iSCSI Channel Management Scenarios with /etc/rc.d/init.d/initiator

	  A) /etc/rc.d/init.d/initiator script explanation
	
		'/etc/rc.d/init.d/initiator <action>' run with NO iscsi_channel_id arguement will
		preform 'execute <action>' on each defined iSCSI Channel.
		
		'/etc/rc.d/init.d/initiator <action> <channel_id>'
		executes <action> on only the passed <channel_id>

	  B) Starting an iSCSI Channel entry with '/etc/rc.d/init.d/initiator start'

		Following the previous examples in section 2 for iSCSI Channel 0 in
		/etc/sysconfig/initiator, it is now time to connect to the iSCSI Target Node
		and register iSCSI Logical Units.  The process of starting an iSCSI Session with
		an iSCSI Target Node is started with:

			/etc/rc.d/init.d/initiator start 0

		Starting an iSCSI Channel will cause the following sequence of events to occur:
			1) Creating an iSCSI Session to the network portal IP address(es) into a iSCSI
			   Target Node defined in /etc/sysconfig/initiator
			2) Performing a iSCSI LUN scan on the connected iSCSI Target Node
			3) Mapping of all iSCSI Logical Units defined in
			   /etc/sysconfig/iscsi_device_maps

	   C) Stopping an iSCSI Channel entry with '/etc/rc.d/init.d/initiator stop'

		Shutting down an iSCSI Channel will unmap all of the iSCSI LUNs as defined within
		/etc/sysconfig/iscsi_device_maps (see section 2 D), and preform an LUN removal and
		logout of the iSCSI Session.  Note that if any iSCSI LUNs are currently in use, the
		shutdown attempt with fail until the iSCSI LUNs are no longer in use, or
		iscsi-unmap-channel is called manually.

			/etc/rc.d/init.d/initiator stop 0

		Stopping an iSCSI Channel will cause the following sequence of events to occur.

			1) Unmapping of all iSCSI Logical Unit mountpoints (fail if BUSY)
			2) Removal of iSCSI Logical Units from SCSI Layer
			3) Logout of iSCSI Session on iSCSI Channel.

	   D) Pausing an iSCSI Channel with '/etc/rc.d/init.d/initiator pause'

		At any time during the lifetime of an iSCSI Channel, all IO may be paused with:

			/etc/rc.d/init.d/initiator pause 0

		Pausing an iSCSI Channel will cause the following event to occur:

			1) Logout of iSCSI Sessions on the specifed iSCSI Channel.

		A iSCSI Channel may be unpaused at any time with '/etc/rc.d/init.d/initiator start',
		see section 3 B) above.

	   E) Getting iSCSI Channel status with '/etc/rc.d/init.d/initiator status'

	   	At any time you may view the status of Sessions/Connections of an iSCSI Channel,
		as well as its assoicated iSCSI Logical Units with:

			/etc/rc.d/init.d/initiator status 0

	  ---------------------------------------------------------------------------------------------------

	KEY TOPICS - This chapter defined the basic iSCSI Channel operations: Start, Stop, Pause, Status.
		     These are the first and essitental operations for interacting with iSCSI Target Nodes
		     on your iSCSI Initiator within your iSCSI Storage Area Network.  Once these basic iSCSI
		     Channel primatives have been grasp, the reader should be able to run and perform basic
		     administration of a iSCSI Initiator Node.

Section 4) Advanced iSCSI Channel Management Scenarios

	TODO - see 'man initiator-ctl'

Section 5) Persistent LUN Mapping

	The concept of using scripts to locate registered iSCSI Logical Units, and automatically mount them
	to user defined mountpoints is something that comes in handy during day-to-day usage of iSCSI
	Initiator nodes.  Note that these scripts are called automatically within /etc/rc.d/init.d/initiator,
	but may be called outside of the typical iSCSI Channel management scenarios mentioned above.
	The following three scripts that read /etc/sysconfig/iscsi_device_maps are:

	A) iscsi-map-channel $CHANNEL

		'iscsi-map-channel 0' will locate entries starting with CHANNEL="0 in iscsi_device_maps and
		call /sbin/mount.

	B) iscsi-unmap-channel $CHANNEL $FORCE

		'iscsi-unmap-channel 0' will locate entries starting with CHANNEL="0 in iscsi_device_maps
		and call /sbin/umount.  Note that the 2nd and optional parameter (ie: iscsi-unmap-channel 0 1)
		will call /sbin/fuser on the mounted device and forcefully unmount the iSCSI LUN.

	C) iscsi-remap-channel $CHANNEL

		'iscsi-renamp-channel 0' is a simple helper script that will unmap and then remap LUNs located
		on iSCSI Channel ID: 0.

	---------------------------------------------------------------------------------------------------

	KEY TOPICS - This chapter focused on the usage of the iSCSI Channel Mapping scripts that use
		     /etc/sysconfig/iscsi_device_maps to automatically mount iSCSI LUNs to defined mount
		     points.  

Section 6) Manual Pages
	
	man 5 initiator - /etc/rc.d/init.d/initiator script and /etc/sysconfig/initiator configuration file

	man 5 initiator_auth - /etc/sysconfig/initiator_auth configuration file

	man 5 iscsi_device_maps -  - /etc/sysconfig/iscsi_device_maps configuration file