Configuration Guide -Server Connection - Fujitsu

FUJITSU StorageETERNUS AF, ETERNUS DX

Configuration Guide -Server Connection-

(Fibre Channel) for Linux

P3AM-3842-32ENZ0

This page is intentionally left blank.

Preface

This manual briefly explains the operations that need to be performed by the user in order to connect anETERNUS AF/DX to a server running Linux via a Fibre Channel interface.

This manual should be used in conjunction with any other applicable user manuals, such as those for theETERNUS AF/DX, server, OS, Fibre Channel cards, and drivers.

Use the default values for parameters that are not described in this manual.

Refer to "Configuration Guide -Server Connection- Notations" for the notations used in this manual such asproduct trademarks and product names. For storage systems that are supported by the OS, refer to the ServerSupport Matrix of the ETERNUS AF/DX.

32nd Edition

February 2022

The Contents and Structure of this Manual

This manual is composed of the following fourteen chapters and an appendix.

• "Chapter 1 Workflow" (page 9)

This chapter describes how to connect the ETERNUS AF/DX storage systems to a server running Linux.

• "Chapter 2 Checking the Server Environment" (page 13)

This chapter describes which servers can be connected to ETERNUS AF/DX storage systems.

• "Chapter 3 Notes" (page 15)

This chapter describes issues that should be noted when connecting the ETERNUS AF/DX storage systemsand server.

• "Chapter 4 Setting Up the ETERNUS AF/DX" (page 20)

This chapter describes how to set up an ETERNUS AF/DX.

• "Chapter 5 Setting Up the Fibre Channel Switches" (page 21)

This chapter describes how to set up the Fibre Channel switches.

• "Chapter 6 Installing Drivers and Setting Up the Server" (page 23)

This chapter describes how to install the Fibre Channel card drivers and set up the server.

• "Chapter 7 Connecting the Server to the ETERNUS AF/DX storage systems" (page 24)

This chapter describes how to connect the server and ETERNUS AF/DX storage systems.

• "Chapter 8 Setting Up the Server to Recognize the Logical Units" (page 25)

This chapter describes how to make the server recognize the LUNs of the ETERNUS AF/DX.

• "Chapter 9 Setting Up and Checking the Multipath Driver" (page 28)

This chapter describes how to set multipaths for multipath connection.

• "Chapter 10 Setting a File System" (page 29)

This chapter describes how to create a file system.

• "Chapter 11 Storage Migration" (page 30)

This chapter describes the procedures that are related to Storage Migration.

• "Chapter 12 Non-disruptive Storage Migration" (page 37)

This chapter describes the procedures that are related to Non-disruptive Storage Migration.

3FUJITSU Storage ETERNUS AF, ETERNUS DX Configuration Guide -Server Connection- (Fibre Channel) for Linux

Copyright 2022 FUJITSU LIMITEDP3AM-3842-32ENZ0

• "Chapter 13 Settings for When the Number of LUNs Exceeds 256 in Red Hat Enterprise Linux" (page 42)

This chapter describes how to set the number of LUNs that can be recognized by the OS if it exceeds 256 inRed Hat Enterprise Linux.

• "Chapter 14 Setup When Using SUSE Linux Enterprise Server 15 SP3" (page 47)

This chapter describes how to avoid problems when using SUSE Linux Enterprise Server 15 SP3.

The restoration procedure for when a path degrades in a Veritas Volume Manager environment is describedin the appendix.

Fibre Channel Card Names

Refer to "Server Support Matrix" to check the Fibre Channel cards.

• "Fujitsu Emulex Fibre Channel cards" refers to Fibre Channel cards with the following product IDs.

- PG-FC105

- PG-FC106

- PG-FC107

- PG-FC201

- PG-FC202/PG-FC202L



- PY-FC201/PY-FC201L


- PY-FC221/PY-FC222





- PG-FCD201

- PG-FCD202

- MC-0JFC11

- MC-0JFC21

- MC-0JFC31

- MC-0JFC41

- MC-0JFC71/MC-0JFC72


- MC-0JFC91

- MC-0JFCA1

- MC-0JFCF1/MC-0JFCF2

- MC-0JFCG1/MC-0JFCG2

- MC-0JFCM1/MC-0JFCM2

- MC-0JFCN1/MC-0JFCN2

- S26361-F2624-E1

- S26361-F2843-E1/S26361-F2843-E201

Preface



- S26361-F3141-E1

- S26361-F3141-E10/S26361-F3141-E210

- S26361-F3306-E601/BX600-FC42E

- S26361-F3306-L1/S26361-F3306-L201

- S26361-F3961-L1/S26361-F3961-L201

- S26361-F3961-L2/S26361-F3961-L202

- S26361-F4044-L501/E1/E201

- S26361-F4044-L502/E2/E202

- S26361-F4994-L501/S26361-F4994-L502

- S26361-F5596-L501/E1/E201

- S26361-F5596-L502/E2/E202

- MC-FC82E

- MC-FC162E

• "Fujitsu QLogic Fibre Channel cards" refers to Fibre Channel cards with the following product IDs.

- PG-FCD101

- PG-FCD102











- S26361-F3023-E1

- S26361-F3631-L1/S26361-F3631-L201

- S26361-F3631-L2/S26361-F3631-L202

- S26361-F4043-L501/E1/E201

- S26361-F4043-L502/E2/E202

- S26361-F5313-L501/E1/E201

- S26361-F5313-L502/E2/E202

- S26361-F5580-L501/E1/E201

- S26361-F5580-L502/E2/E202



- MC-0JFCB1/MC-0JFCB2

- MC-0JFCC1/MC-0JFCC2

- MC-0JFCD1/MC-0JFCD2

- MC-0JFCE1/MC-0JFCE2

- MC-0JFCP1/MC-0JFCP2

Preface



- MC-0JFCQ1/MC-0JFCQ2

- MC-0JFCK1/MC-0JFCK2

- MC-0JFCL1

Preface



Table of Contents

Chapter 1 Workflow 9

Chapter 2 Checking the Server Environment 13

2.1 Hardware ..................................................................................................................................... 13

2.2 Operating System (OS) ................................................................................................................. 13

2.3 Fibre Channel Cards...................................................................................................................... 13

2.4 ETERNUS Multipath Driver ............................................................................................................ 13

2.5 Veritas Volume Manager (VxVM).................................................................................................. 13

2.6 device-mapper multipath............................................................................................................. 13

2.7 Veritas Cluster Server (VCS) .......................................................................................................... 14

Chapter 3 Notes 15

3.1 Connection Notes ......................................................................................................................... 153.1.1 SCSI Devices and Linux........................................................................................................................................15

3.2 Red Hat Enterprise Linux Notes.................................................................................................... 15

3.3 Oracle Linux Notes ....................................................................................................................... 15

3.4 SUSE Linux Enterprise Server Notes .............................................................................................. 15

3.5 Fibre Channel Card Notes ............................................................................................................. 16

3.6 Fibre Channel Switch Notes .......................................................................................................... 16

3.7 Server Startup and Power Supply Control Notes............................................................................ 16

3.8 System Design Sheet Notes .......................................................................................................... 16

3.9 Veritas Volume Manager (VxVM) Notes........................................................................................ 17

3.10 Notes for Veritas Volume Manager Dynamic Multipathing (VxVM DMP) .................................... 17

3.11 Veritas Cluster Server (VCS) Notes .............................................................................................. 17

3.12 Notes on Connecting the ETERNUS AF/DX to a Server ................................................................. 17

3.13 Path Addition Notes ................................................................................................................... 17

3.14 SAN Boot Notes .......................................................................................................................... 18

3.15 Notes on Creating and Mounting File Systems ........................................................................... 18

3.16 Notes on Using Thin Provisioning Volumes ................................................................................ 19



3.17 Notes on Changing the max_sectors_kb Value ........................................................................... 19

Chapter 4 Setting Up the ETERNUS AF/DX 20

Chapter 5 Setting Up the Fibre Channel Switches 21

Chapter 6 Installing Drivers and Setting Up the Server 23

Chapter 7 Connecting the Server to the ETERNUS AF/DX storage systems 24

Chapter 8 Setting Up the Server to Recognize the Logical Units 25

8.1 Displaying the Recognized Logical Units ...................................................................................... 258.1.1 For Red Hat Enterprise Linux or Oracle Linux ......................................................................................................25

8.1.2 For SUSE Linux Enterprise Server.........................................................................................................................26

Chapter 9 Setting Up and Checking the Multipath Driver 28

9.1 Setting Up and Checking the ETERNUS Multipath Driver............................................................... 28

9.2 Setting Up and Checking the device-mapper multipath ............................................................... 28

Chapter 10 Setting a File System 29

Chapter 11 Storage Migration 30

Chapter 12 Non-disruptive Storage Migration 37

Chapter 13 Settings for When the Number of LUNs Exceeds 256 in Red Hat Enter-prise Linux 42

13.1 For Red Hat Enterprise Linux 8.x or Red Hat Enterprise Linux 7.x............................................... 42

13.2 For Red Hat Enterprise Linux 6.x ................................................................................................ 45

13.3 For Red Hat Enterprise Linux 5.x ................................................................................................ 46

Chapter 14 Setup When Using SUSE Linux Enterprise Server 15 SP3 47

Appendix A Restoration Procedure for when a Path Degrades in a Veritas VolumeManager Environment 51

Table of Contents



Chapter 1Workflow

This chapter describes how to connect the ETERNUS AF/DX to a server running Linux.

Procedure varies according to whether a single-path or multipath configuration is used.

When a Fibre Channel switch is used to make the connection, refer to the "Configuration Guide -ServerConnection- (Fibre Channel) Fibre Channel Switch Settings" and use the procedure described.

The workflow is shown below.

Usable Multipath Drivers

• ETERNUS Multipath Driver

• Veritas Volume Manager

• device-mapper multipath

Required Documents

• "Server Support Matrix"

• "Server Support Matrix for FC-SWITCH"

• "Configuration Guide -Server Connection- Storage System Settings" that corresponds to the ETERNUS AF/DXto be connected

• "Configuration Guide -Server Connection- (Fibre Channel) Fibre Channel Switch Settings"

• "Configuration Guide -Server Connection- (Fibre Channel) for Linux Driver Settings for Fibre Channel Cards"

• "Configuration Guide -Server Connection- (Fibre Channel/FCoE/iSCSI/SAS) for Linux device-mapper multi-path"

• "ETERNUS Web GUI User's Guide"

• Manuals supplied with the server, Fibre Channel cards, Fibre Channel card driver, multipath driver, andVxVM



Workflow

Setting Up the ETERNUS AF/DXSet the various parameters required to operate the ETERNUS AF/DX.

• "Chapter 4 Setting Up the ETERNUS AF/DX" (page 20)

• Checking the ETERNUS Web GUI operational procedures

- "ETERNUS Web GUI User's Guide"

• Setting up the ETERNUS AF/DX

- "Configuration Guide -Server Connection- Storage System Settings" that corresponds to the ETER-NUS AF/DX to be connected

Setting Up the Fibre Channel SwitchesIf a Fibre Channel switch is to be used, set it up now.

• "Chapter 5 Setting Up the Fibre Channel Switches" (page 21)

• Setting up the Fibre Channel switches

- "Configuration Guide -Server Connection- (Fibre Channel) Fibre Channel Switch Settings"

• Checking the Fibre Channel switch connection requirements

- "Server Support Matrix for FC-SWITCH"

Installing the DriverInstall the appropriate driver for the Fibre Channel card being used.

Applicable document varies according to the server, OS, and Fibre Channel Card type being used. Se-lect from the following list.

• "Chapter 6 Installing Drivers and Setting Up the Server" (page 23)

• Installing Fibre Channel cards, installing drivers, and performing setup

- "Configuration Guide -Server Connection- (Fibre Channel) for Linux Driver Settings for Fibre Chan-nel Cards"

• Checking the Fibre Channel card driver versions

- "Server Support Matrix"

When the ETERNUS Multipath Driver is used When Veritas Volume Manager orthe device-mapper multipath isused. Or, when the Multipath

Driver is not used.

Chapter 1 Workflow



Installing the ETERNUS Multipath DriverInstall the ETERNUS Multipath Driver.

• "Chapter 9 Setting Up and Checking the Multipath Driver" (page28)

• Installing and setting up the Multipath Driver

- Manuals provided with the Multipath Driver

• Checking the corresponding Multipath Driver


Connecting the Server to the ETERNUS AF/DX Storage SystemsConnect the server and the ETERNUS AF/DX storage systems using the Fibre Channel cables.

• "Chapter 7 Connecting the Server to the ETERNUS AF/DX storage systems" (page 24)

Chapter 1 Workflow



Setting Up the Server to Recognize the Logical UnitsSet up the server so that it can recognize the LUNs (logical unit numbers) of the ETERNUS AF/DX.

• "Chapter 8 Setting Up the Server to Recognize the Logical Units" (page 25)

When the ETERNUS MultipathDriver is used or when the Mul-

tipath Driver is not used

When Veritas Volume Manageris used

When the device-mapper multi-path is used

Installing Veritas VolumeManagerInstall the Veritas Volume Man-ager software.

• Installing and setting up theVeritas Volume Manager

- Documentation providedwith the product

• Checking the correspondingMultipath Driver


Setting Up and Checking thedevice-mapper multipathSet up and check the device-mapper multipath.

• "Chapter 9 Setting Up andChecking the Multipath Driv-er" (page 28)

• Setting up the device-mappermultipath

- "Configuration Guide -Serv-er Connection- (Fibre Chan-nel/FCoE/iSCSI/SAS) for Li-nux device-mapper multi-path"

Creating a File SystemCreate a file system.

• "Chapter 10 Setting a File System" (page 29)

Chapter 1 Workflow



Chapter 2Checking the Server Environment

Connection to servers is possible in the following environments.

Check the "Server Support Matrix" for server environment conditions.

2.1 Hardware

Refer to the "Server Support Matrix".

2.2 Operating System (OS)


2.3 Fibre Channel Cards


2.4 ETERNUS Multipath Driver


2.5 Veritas Volume Manager (VxVM)


2.6 device-mapper multipath




2.7 Veritas Cluster Server (VCS)


Chapter 2 Checking the Server Environment2.7 Veritas Cluster Server (VCS)



Chapter 3Notes

Note the following issues when connecting the ETERNUS AF/DX to a server.

3.1 Connection Notes

3.1.1 SCSI Devices and LinuxThe ETERNUS AF/DX storage systems are recognized by the server as a set of Fibre Channel connected SCSIdisks. The server then recognizes the ETERNUS AF/DX storage systems' disks as LUNs (logical units). For Li-nux, SCSI disk devices have the following definitions in "/dev":

Device (LUN) Partition

sda sda1, sda2,. . . . sda15

sdb sdb1, sdb2,. . . . sdb15

:

sdz sdzl, sdz2,. . . . sdz15

sdaa-sdaz sdaa1, sdaa2,. . . . sdaz15

sdba-sdbz sdba1, sdba2,. . . . sdbz15

sdca-sdcz sdca1, sdca2,. . . . sdcz15

sdda-sddx sdda1, sdda2,. . . . sddx15

:

sdia-sdiv sdia1, sdia2,. . . . sdiv15

3.2 Red Hat Enterprise Linux Notes

For details involving RHEL, refer to the Red Hat web-site and any documentation available.

3.3 Oracle Linux Notes

For details involving Oracle Linux, refer to the Oracle web-site and any documentation available.

3.4 SUSE Linux Enterprise Server Notes

For details involving SLES, refer to the Novell web-site and any documentation available.



3.5 Fibre Channel Card Notes

When a Fibre Channel card is installed in the server after installation of the OS, it should be automaticallyrecognized the next time the OS is restarted. Proceed as follows when a response is requested:

• For Red Hat Enterprise Linux

The kudzu starts automatically during the system reboot. Select "Ignore".

• For SUSE Linux Enterprise Server

The "New hardware detection" window appears during the system reboot. Select "No".

• For RHEL6.3 or Oracle Linux 6.3 with a QLogic Fibre Channel card

If the Fibre Channel connection is lost, "LOOP DOWN" may not be recorded on the log in the host.

3.6 Fibre Channel Switch Notes

• When a Fibre Channel switch (ETERNUS SN200 series or Brocade series) is to be used between the serverand ETERNUS AF/DX, follow the preparation-in-advance and Fibre Channel switch setup procedures givenin the "Configuration Guide -Server Connection- (Fibre Channel) Fibre Channel Switch Settings".

• When a 4Gbit/s QLogic Fibre Channel card is connected to a Fibre Channel switch (ETERNUS SN200 series orBrocade series) and the SFP produces an incorrect linkup, change the BIOS settings of the Fibre Channelcard. Contact your maintenance engineer for SFP details.

3.7 Server Startup and Power Supply Control Notes

Before turning the server on, check that the ETERNUS AF/DX storage systems and Fibre Channel switches areall "Ready". If the server is turned on and they are not "Ready", the server will not be able to recognize theETERNUS AF/DX storage systems.

Also, when the ETERNUS AF/DX power supply is being controlled by a connected server, make sure that theETERNUS AF/DX does not shut down before the connected servers. Similarly, the Fibre Channel switches mustalso be turned off after the connected servers have been shut down. If turned off, data writes from the run-ning server cannot be saved to the ETERNUS AF/DX storage systems, and already saved data may also beaffected.

3.8 System Design Sheet Notes

The system design sheet is a spreadsheet program work sheet that is used to simplify the process of instal-ling the ETERNUS AF/DX. It is important that the system details be recorded after first installing the systemand also each time the system is subsequently modified, expanded, or has maintenance work performed onit. Creating a system design sheet makes installation and maintenance of the system easy.

Chapter 3 Notes3.5 Fibre Channel Card Notes



3.9 Veritas Volume Manager (VxVM) Notes

• If the ETERNUS AF/DX storage systems are connected to a configuration that uses the VxVM, the VeritasArray Support Library (ASL) will also need to be installed. Confirm and acquire the required ASL from thefollowing Veritas web-site.

https://sort.veritas.com/asl

• Install Veritas Array Support Library (ASL) after installing VxVM.

• For VxVM 5.1 or later

ASL does not need to be installed.

• If a path degrades when Veritas Volume Manager is used, some volumes may not be successfully recog-nized after path restoration regardless of the Linux version or VxVM version that is installed. In this case,volume recognition is required after restoring the LUN using the utility provided by each Fibre Channel cardvendor. Refer to "Appendix A Restoration Procedure for when a Path Degrades in a Veritas Volume Manag-er Environment" (page 51) for more details.

3.10 Notes for Veritas Volume Manager Dynamic Multipathing(VxVM DMP)

When using the VxVM DMP function, do not use it with a single path configuration.

3.11 Veritas Cluster Server (VCS) Notes

Veritas Cluster Server should be installed according to the directions given in the documentation providedwith Veritas Cluster Server.

3.12 Notes on Connecting the ETERNUS AF/DX to a Server

When connecting the ETERNUS AF/DX to a server using a Fibre Channel cable, make sure to confirm that allthe settings on the ETERNUS AF/DX are complete. The WWN Automatic Acquisition function should not beused because it requires the server to be connected. Otherwise, LUNs in the ETERNUS AF/DX may not be rec-ognized properly.

3.13 Path Addition Notes

When using device-mapper multipath and the OS is not rebooted after adding the paths, the added pathsmay not be automatically recognized. In that case, execute the following command.

# multipath -r

Chapter 3 Notes3.9 Veritas Volume Manager (VxVM) Notes



3.14 SAN Boot Notes

When using an ETERNUS AF/DX LUN (Logical Unit) as the boot disk, check the "Server Support Matrix" for theserver environment being used, as well as any SAN boot documentation (such as in a system installationguide) provided with the server.

When an ETERNUS AF/DX LUN is to be used as a boot disk, it must be provided with sufficient capacity toinstall the OS.

3.15 Notes on Creating and Mounting File Systems

If the environment meets all the following conditions, the file system creation command may fail due to anerror.

• RHEL7.3 or later, or SLES12 SP2 or later is used

• ETERNUS DX S3 series or ETERNUS AF series is used with firmware version earlier than V10L70

• A TPV or an FTV with an Allocation setting of "Thick" is used

Add the following options according to the file system type and execute the command.

• If the file system is ext3/ext4

-E discard option

Example:

# mkfs.ext4 -E discard <Device name>

• If the file system is xfs

-K option

Example:

# mkfs.xfs -K <Device name>

Do not specify the "discard" option when mounting these file systems.

Chapter 3 Notes3.14 SAN Boot Notes



3.16 Notes on Using Thin Provisioning Volumes

If the environment meets all of the following conditions, the area of Thin Provisioning Volumes may not bereleased properly.

• RHEL7.3 or later, SLES12 SP2 or later, Oracle Linux 6.9, Oracle Linux 6.10, or Oracle Linux 7.3 or later isused

• A TPV or an FTV with an Allocation setting of "Thin" is used

To avoid this, perform the following procedure.

1 Locate the configuration file (99-eternus.rules) in /etc/udev/rules.d/ and add the following content tothat file.

ACTION=="add", SUBSYSTEM=="scsi_disk", ATTRS{vendor}=="FUJITSU", ATTRS{model}=="ETERNUS_DXH", ATTR{thin_provisioning}=="1", ATTR{provisioning_mode}="unmap"ACTION=="add", SUBSYSTEM=="scsi_disk", ATTRS{vendor}=="FUJITSU", ATTRS{model}=="ETERNUS_DXM", ATTR{thin_provisioning}=="1", ATTR{provisioning_mode}="unmap"ACTION=="add", SUBSYSTEM=="scsi_disk", ATTRS{vendor}=="FUJITSU", ATTRS{model}=="ETERNUS_DXL", ATTR{thin_provisioning}=="1", ATTR{provisioning_mode}="unmap"ACTION=="add", SUBSYSTEM=="scsi_disk", ATTRS{vendor}=="FUJITSU", ATTRS{model}=="ETERNUS_DX400", ATTR{thin_provisioning}=="1", ATTR{provisioning_mode}="unmap"ACTION=="add", SUBSYSTEM=="scsi_disk", ATTRS{vendor}=="FUJITSU", ATTRS{model}=="ETERNUS_DX8000", ATTR{thin_provisioning}=="1", ATTR{provisioning_mode}="unmap"

2 Restart the OS.

3.17 Notes on Changing the max_sectors_kb Value

Change /sys/block/sdX/queue/max_sectors_kb to a value of 7992 or less when tuning the maximum I/O size.

Chapter 3 Notes3.16 Notes on Using Thin Provisioning Volumes



Chapter 4Setting Up the ETERNUS AF/DX

Set up the ETERNUS AF/DX storage systems using ETERNUS Web GUI.

ETERNUS AF/DX setup can be performed independently of server setup. For details on how to perform thesesettings, refer to the following manuals.


• "ETERNUS Web GUI User's Guide"



Chapter 5Setting Up the Fibre Channel Switches

Perform the settings required to connect the ETERNUS AF/DX storage systems and server via the Fibre Chan-nel switch, according to "Configuration Guide -Server Connection- (Fibre Channel) Fibre Channel Switch Set-tings".

If the access path is set with ETERNUS SF Storage Cruiser, the Host Response settings are set to the defaultvalues.

If the Host Response settings are changed from the default values, set the Host Response again.

The following examples show configurations in which a server is connected to a Fibre Channel switch withzoning.

The following example shows a configuration for multiple servers.

Port2Port0 Port6Port4




ZONE1 ZONE2 ZONE3

ZONE4

Name : Server#1

Port1Port0

Name : Server#2

Port1Port0

Cascading

line

FC switch FC switch

CM0 CM1

Name : RAID#1

ETERNUS AF/DX

Port1 Port1Port0 Port0



The following example shows a configuration for a single server.





Cascading

line

ZONE1

ZONE2

ZONE3 ZONE4

FC switch FC switch

Server

Port0 Port1

CM0 CM1

ETERNUS AF/DX

Port0 Port0 Port1Port1

Chapter 5 Setting Up the Fibre Channel Switches



Chapter 6Installing Drivers and Setting Up the Server

Install the Fibre Channel Card driver, and then set up the server environment.

Refer to "Configuration Guide -Server Connection- (Fibre Channel) for Linux Driver Settings for Fibre ChannelCards" for detailed setting procedures.

• When Emulex Fibre Channel cards or Fujitsu Emulex Fibre Channel cards are used, refer to the "Setup Pro-cedure for Emulex Fibre Channel Cards" chapter.

• When QLogic Fibre Channel cards or Fujitsu QLogic Fibre Channel cards are used, refer to the "Setup Proce-dure for QLogic Fibre Channel Cards" chapter.

• When Brocade Fibre Channel cards are used, refer to the "Operations When Using Brocade Fibre ChannelCards" chapter.



Chapter 7Connecting the Server to the ETERNUS AF/DX stor-age systems

After all the required drivers have been installed, shut down the OS and make sure the server is turned off.

Connect the shutdown server to the ETERNUS AF/DX storage systems using Fibre Channel cables, then restartthe OS.

The server and ETERNUS AF/DX storage systems must be connected for the procedures that follow.



Chapter 8Setting Up the Server to Recognize the LogicalUnits

At boot up, Linux automatically recognizes all SCSI disks and assigns them successive device names startingfrom "sda". When assigning the device names, any internal SCSI disks are assigned first, followed by the as-signment of the ETERNUS AF/DX storage systems' LUNs. Example: If there is one internal SCSI disk, it is as-signed a SCSI device name as follows:

SCSI device name by-id name (Example) Explanation

/dev/sda /dev/disk/by-id/scsi-36003005700ac3fc013746ef6315e42c4 Internal SCSI disk

/dev/sdb or later /dev/disk/by-id/scsi-3600000e00d0000000000042200000000/dev/disk/by-id/scsi-3600000e00d0000000000042200010000:

ETERNUS AF/DX storagesystems' LUN

8.1 Displaying the Recognized Logical Units

The driver version and information of each LUN's SCSI device name, etc. are displayed in the console messagewhen Linux is booted up. Although this console message disappears soon from the screen, it can be re-dis-played using the following commands.

8.1.1 For Red Hat Enterprise Linux or Oracle Linux• "dmesg" command for LUN display

Use the "dmesg" command in order to confirm whether Linux has recognized the LUNs in the ETERNUSAF/DX storage systems.

The following shows an example:

#dmesg|less :Emulex LightPulse FC SCSI/IP 4.21qPCI:Found IRQ 10 for device 03:06.0!lpfc0:031:Link Up Event received Data:1 1 1 2PCI:Found IRQ 5 for device 04:02.0IRQ routing conflict for 04:02.0,have irq 10,want irq 5!lpfc0:031:Link Up Event received Data:1 1 1 2scsi3:Emulex LPFC SCSI on PCI bus 03 device 30 irq 10scsi4:Emulex LPFC SCSI on PCI bus 04 device 10 irq 10Vender: FUJITSU Model: ETERNUS_DX400 Rev: 0000Type: Direct-Access ANSI SCSI revision: 03Vender: FUJITSU Model: ETERNUS_DX400 Rev: 0000Type: Direct-Access ANSI SCSI revision: 03



• "cat" command for LUN display

Use the "cat" command to display the "/proc/scsi/scsi" file, which contains a list of recognized SCSI devices.Check each path of the allocated LUNs.


# cat /proc/scsi/scsi Attached devices:Host: scsi2 Channel: 00 Id: 05 Lun: 00Vender: HP Model: SAFTE; U160/M BP Rev: 1023Type: processor ANSI SCSI revision: 02Host: scsi2 Channel: 02 Id: 00 Lun: 00Vender: MegaRAID Model: LD0 RAID0 8677R Rev: EType: Direct-Access ANSI SCSI revision: 02Host: scsi3 Channel: 00 Id: 00 Lun: 00Vender: FUJITSU Model: ETERNUS_DX400 Rev: 0000Type: Direct-Access ANSI SCSI revision: 03Host: scsi3 Channel: 00 Id: 00 Lun: 01Vender: FUJITSU Model: ETERNUS_DX400 Rev: 0000Type: Direct-Access ANSI SCSI revision: 03

8.1.2 For SUSE Linux Enterprise Server• Boot message for LUN display

Open the /var/log/boot.msg file to confirm that Linux has recognized the ETERNUS AF/DX storage systems'LUNs.

The following shows an example (Messages can be scrolled (back and forth) using the arrow keys).

<6>qla2x00_set_info starts at address - c3600060<6>qla2x00: Found VID-1077 DID-2422 SSVID-1077 SSDID-134<6>scsi(2): Found a QLA2422 @ bus l, device 0xa, 1rq 29, iobase 0xf9o0d000

. . .

<6>scsi2 : Qlogic QLA2422 PCI to Fibre Channel Host Adapter: bus l device 10 irq 29<4> Firmware version: 4.00.12, Driver version 7.07.01 (1)<4><6>scsi3 : Qlogic QLA2422 PCI to Fibre Channel Host Adapter: bus l device 10 irq 30<4> Firmware version: 4.00.12, Driver version 7.07.01 (2)<4><4> Vendor: FUJITSU Model: ETERNUS_DX400 Rev: 0000<4> Type: Direct-Access ANSI SCSI revision: 05<4> Vendor: FUJITSU Model: ETERNUS_DX400 Rev: 0000<4> Type: Direct-Access ANSI SCSI revision: 05<6>scsi(2:0:0:0): Enabled tagged queuing, queue depth 20.<6>scsi(2:0:0:1): Enabled tagged queuing, queue depth 20.<4>Attached scsi disk sdb at scsi2, channel 0, id 0, lun 0<4>Attached scsi disk sdb at scsi2, channel 0, id 0, lun 1<4>SCSI device sdb: 4096000 512-byte hdwr sectors (2097 MB)<6> sdb: unknown partition table<4>SCSI device sdb: 4096000 512-byte hdwr sectors (2097 MB)<6> sdb: unknown partition table

(1) and (2) indicate the driver versions.

Chapter 8 Setting Up the Server to Recognize the Logical Units8.1 Displaying the Recognized Logical Units



• "cat" command for LUN display

Use the "cat" command to display the "/proc/scsi/scsi" file, which contains a list of recognized SCSI devices.Check each path of the allocated LUNs.


# cat /proc/scsi/scsi Attached devices:Host: scsi0 Channel: 00 Id: 04 Lun: 00 Vender: FUJITSU Model: MAP3735NC Rev: 5207 Type: Direct-Access ANSI SCSI revision: 03Host: scsi0 Channel: 00 Id: 08 Lun: 00 Vender: SDR Model: GEM318 Rev: 0 Type: Processor ANSI SCSI revision: 02Host: scsi1 Channel: 00 Id: 05 Lun: 00 Vender: SEAGATE Model: DAT 9SP40-000 Rev: 9M38 Type: Sequential-Access ANSI SCSI revision: 03Host: scsi2 Channel: 00 Id: 00 Lun: 00 Vender: FUJITSU Model: ETERNUS_DX400 Rev: 0000 Type: Direct-Access ANSI SCSI revision: 05Host: scsi2 Channel: 00 Id: 00 Lun: 01 Vender: FUJITSU Model: ETERNUS_DX400 Rev: 0000 Type: Direct-Access ANSI SCSI revision: 05

Chapter 8 Setting Up the Server to Recognize the Logical Units8.1 Displaying the Recognized Logical Units



Chapter 9Setting Up and Checking the Multipath Driver

9.1 Setting Up and Checking the ETERNUS Multipath Driver

For the setup and check procedures, refer to the manual that is provided with the ETERNUS Multipath Driver.

9.2 Setting Up and Checking the device-mapper multipath

Set up and check the device-mapper multipath.

For the procedures, refer to "Configuration Guide -Server Connection- (Fibre Channel/FCoE/iSCSI/SAS) for Linuxdevice-mapper multipath".



Chapter 10Setting a File System

After Linux has recognized the ETERNUS AF/DX storage systems' LUNs, access for data storage will be possibleafter completing the following steps:

• Setting up Partitions

• Formatting Partitions

• Mounting Partitions

If by-id names are being used with Red Hat Enterprise Linux AS v.4 Update 4 or Red Hat Enterprise LinuxES v.4 Update 4 or later, then the accessible device names will be by-id names.



Chapter 11Storage Migration

This chapter explains how to configure the server for performing Storage Migration.

When Storage Migration is performed, configure the settings so that the ETERNUS AF/DX LUNs can be usedfrom the server (RHEL).

■ Setting Procedure Outline

Execute the operation to start Storage Migration from ETERNUS Web GUI. Perform all other operations in theserver (RHEL).

In this procedure, ETERNUS Multipath Driver or device-mapper is used as the multipath driver.

There are two methods for the Storage Migration procedure: one for when editing fstab, and one for whenediting bindings. When using the ETERNUS AF/DX as a Raw device, use the method for editing the bindingsto perform Storage Migration.

● When editing fstab

1 Unmount the target LUN.

2 Edit the "/etc/fstab" file, and change the target LUN so that it will not mount.

3 Edit the "/etc/multipath.conf" file, and change the device ID for the ETERNUS AF/DX.

4 Turn off the server.

5 Start Storage Migration, change the connection destination to the migration destination ETERNUSAF/DX, and then reboot the server.

6 Check the targeted LUN.

7 Edit the "/etc/fstab" file, change the target LUN so that it will mount, and then mount the target LUN.

● When editing bindings

1 Unmount the target LUN.




5 Start Storage Migration, change the connection destination to the migration destination ETERNUSAF/DX, and then reboot the server.


7 Edit the "/var/lib/multipath/bindings" file, and change the WWID of the alias to the value for the migra-tion destination ETERNUS AF/DX.



8 Apply the file changes to the system.

9 Mount the target LUN.

10 Edit the "/etc/fstab" file, and change the target LUN so that it will mount.

■ Example Setting Procedure

The following procedure shows an example of editing the bindings in RHEL 5.10.

1 Stop the processes that are accessing the target LUN, and unmount the target LUN.


# vi /etc/fstab

/dev/VolGroup00/LogVol00 / ext3 defaults 1 1LABEL=/boot /boot ext3 defaults 1 2tmpfs /dev/shm tmpfs defaults 0 0devpts /dev/pts devpts gid=5,mode=620 0 0sysfs /sys sysfs defaults 0 0proc /proc proc defaults 0 0/dev/VolGroup00/LogVol01 swap swap defaults 0 0# /dev/mapper/mpath1p1 /mnt/lun0 ext3 defaults 0 0 fComment out this line.# /dev/mapper/mpath2p1 /mnt/lun1 ext2 defaults 0 0 fComment out this line.# /dev/mapper/mpath5p1 /mnt/lun4 ext3 defaults 0 0 fComment out this line.# /dev/mapper/mpath6p1 /mnt/lun5 ext4 defaults 0 0 fComment out this line.# /dev/vg1/lv1 /mnt/lvm0 ext4 defaults 0 0 fComment out this line.

#


# vi /etc/multipath.conf

# This is a basic configuration file with some examples, for device mapper# multipath.(Omitted)devices { device { vendor "FUJITSU"# product "ETERNUS_DX400" fDelete this line. product "ETERNUS_DXH" fAdd this line. (Change "ETERNUS_DX400" to "ETERNUS_DXH".) prio_callout "/sbin/mpath_prio_alua %d" path_grouping_policy group_by_prio path_selector "round-robin 0" failback immediate no_path_retry 10 path_checker tur }}

#


5 Start Storage Migration from ETERNUS Web GUI and then change the connection destination to the mi-gration destination ETERNUS AF/DX and reboot the server.

Chapter 11 Storage Migration




# multipath -ll

mpath16 (3600000e00d2b0000002b010100740000) dm-9 FUJITSU,ETERNUS_DXH[size=20G][features=1 queue_if_no_path][hwhandler=0][rw]\_ round-robin 0 [prio=50][active] \_ 5:0:0:7 sdi 8:128 [active][ready] \_ 6:0:0:7 sdq 65:0 [active][ready]mpath9 (3600000e00d2b0000002b010100590000) dm-2 FUJITSU,ETERNUS_DXH[size=20G][features=1 queue_if_no_path][hwhandler=0][rw]\_ round-robin 0 [prio=50][active] \_ 5:0:0:0 sdb 8:16 [active][ready] \_ 6:0:0:0 sdj 8:144 [active][ready]mpath15 (3600000e00d2b0000002b010100730000) dm-8 FUJITSU,ETERNUS_DXH[size=20G][features=1 queue_if_no_path][hwhandler=0][rw]\_ round-robin 0 [prio=50][active] \_ 5:0:0:6 sdh 8:112 [active][ready] \_ 6:0:0:6 sdp 8:240 [active][ready]mpath14 (3600000e00d2b0000002b010100720000) dm-7 FUJITSU,ETERNUS_DXH[size=20G][features=1 queue_if_no_path][hwhandler=0][rw]\_ round-robin 0 [prio=50][active] \_ 5:0:0:5 sdg 8:96 [active][ready] \_ 6:0:0:5 sdo 8:224 [active][ready]mpath13 (3600000e00d2b0000002b010100710000) dm-6 FUJITSU,ETERNUS_DXH[size=20G][features=1 queue_if_no_path][hwhandler=0][rw]\_ round-robin 0 [prio=50][active] \_ 5:0:0:4 sdf 8:80 [active][ready] \_ 6:0:0:4 sdn 8:208 [active][ready]mpath12 (3600000e00d2b0000002b0101005c0000) dm-5 FUJITSU,ETERNUS_DXH[size=20G][features=1 queue_if_no_path][hwhandler=0][rw]\_ round-robin 0 [prio=50][active] \_ 5:0:0:3 sde 8:64 [active][ready] \_ 6:0:0:3 sdm 8:192 [active][ready]mpath11 (3600000e00d2b0000002b0101005b0000) dm-4 FUJITSU,ETERNUS_DXH[size=20G][features=1 queue_if_no_path][hwhandler=0][rw]\_ round-robin 0 [prio=50][active] \_ 5:0:0:2 sdd 8:48 [active][ready] \_ 6:0:0:2 sdl 8:176 [active][ready]mpath10 (3600000e00d2b0000002b0101005a0000) dm-3 FUJITSU,ETERNUS_DXH[size=20G][features=1 queue_if_no_path][hwhandler=0][rw]\_ round-robin 0 [prio=50][active] \_ 5:0:0:1 sdc 8:32 [active][ready] \_ 6:0:0:1 sdk 8:160 [active][ready]

#(Continued on the next page)




(Continued from the previous page)# cat /var/lib/multipath/bindings

# Multipath bindings, Version : 1.0# NOTE: this file is automatically maintained by the multipath program.# You should not need to edit this file in normal circumstances.## Format:# alias wwid#mpath0 36003005700e979401efd5fe60f79a19bmpath1 36000b5d0006a0000006a0ba000040000mpath2 36000b5d0006a0000006a0ba000050000mpath3 36000b5d0006a0000006a0ba000060000mpath4 36000b5d0006a0000006a0ba000070000mpath5 36000b5d0006a0000006a0ba0001c0000mpath6 36000b5d0006a0000006a0ba0001d0000mpath7 36000b5d0006a0000006a0ba0001e0000mpath8 36000b5d0006a0000006a0ba0001f0000mpath9 3600000e00d2b0000002b010100590000 fAfter the migrationmpath10 3600000e00d2b0000002b0101005a0000 fAfter the migrationmpath11 3600000e00d2b0000002b0101005b0000 fAfter the migrationmpath12 3600000e00d2b0000002b0101005c0000 fAfter the migrationmpath13 3600000e00d2b0000002b010100710000 fAfter the migrationmpath14 3600000e00d2b0000002b010100720000 fAfter the migrationmpath15 3600000e00d2b0000002b010100730000 fAfter the migrationmpath16 3600000e00d2b0000002b010100740000 fAfter the migration

#

# ls -l /dev/mapper

total 0crw------- 1 root root 10, 60 Jul 15 14:25 controlbrw-rw---- 1 root disk 253, 3 Jul 15 14:26 mpath10brw-rw---- 1 root disk 253, 10 Jul 15 14:26 mpath10p1brw-rw---- 1 root disk 253, 4 Jul 15 14:26 mpath11brw-rw---- 1 root disk 253, 5 Jul 15 14:26 mpath12brw-rw---- 1 root disk 253, 6 Jul 15 14:26 mpath13brw-rw---- 1 root disk 253, 11 Jul 15 14:26 mpath13p1brw-rw---- 1 root disk 253, 7 Jul 15 14:26 mpath14brw-rw---- 1 root disk 253, 12 Jul 15 14:26 mpath14p1brw-rw---- 1 root disk 253, 8 Jul 15 14:26 mpath15brw-rw---- 1 root disk 253, 9 Jul 15 14:26 mpath16brw-rw---- 1 root disk 253, 2 Jul 15 14:26 mpath9brw-rw---- 1 root disk 253, 13 Jul 15 14:26 mpath9p1brw-rw---- 1 root disk 253, 14 Jul 15 14:26 vg1-lv1brw-rw---- 1 root disk 253, 0 Jul 15 14:26 VolGroup00-LogVol00brw-rw---- 1 root disk 253, 1 Jul 15 14:25 VolGroup00-LogVol01

#




Based on these results, the following applies to before Storage Migration starts and after it starts.

mpath1 36000b5d0006a0000006a0ba000040000 mpath9 3600000e00d2b0000002b010100590000

mpath2 36000b5d0006a0000006a0ba000050000 mpath10 3600000e00d2b0000002b0101005a0000

mpath3 36000b5d0006a0000006a0ba000060000 mpath11 3600000e00d2b0000002b0101005b0000

mpath4 36000b5d0006a0000006a0ba000070000 mpath12 3600000e00d2b0000002b0101005c0000

mpath5 36000b5d0006a0000006a0ba0001c0000 mpath13 3600000e00d2b0000002b010100710000

mpath6 36000b5d0006a0000006a0ba0001d0000 mpath14 3600000e00d2b0000002b010100720000

mpath7 36000b5d0006a0000006a0ba0001e0000 mpath15 3600000e00d2b0000002b010100730000

mpath8 36000b5d0006a0000006a0ba0001f0000 mpath16 3600000e00d2b0000002b010100740000

Before Storage Migration starts After Storage Migration starts

7 Edit the "/var/lib/multipath/bindings" file, and change the WWID of the alias to the value for the migra-tion destination ETERNUS AF/DX.

# vi /var/lib/multipath/bindings

# Multipath bindings, Version : 1.0# NOTE: this file is automatically maintained by the multipath program.# You should not need to edit this file in normal circumstances.## Format:# alias wwid#mpath0 36003005700e979401efd5fe60f79a19b# mpath1 36000b5d0006a0000006a0ba000040000 fDelete this line.# mpath2 36000b5d0006a0000006a0ba000050000 fDelete this line.# mpath3 36000b5d0006a0000006a0ba000060000 fDelete this line.# mpath4 36000b5d0006a0000006a0ba000070000 fDelete this line.# mpath5 36000b5d0006a0000006a0ba0001c0000 fDelete this line.# mpath6 36000b5d0006a0000006a0ba0001d0000 fDelete this line.# mpath7 36000b5d0006a0000006a0ba0001e0000 fDelete this line.# mpath8 36000b5d0006a0000006a0ba0001f0000 fDelete this line.# mpath9 3600000e00d2b0000002b010100590000 fDelete this line.# mpath10 3600000e00d2b0000002b0101005a0000 fDelete this line.# mpath11 3600000e00d2b0000002b0101005b0000 fDelete this line.# mpath12 3600000e00d2b0000002b0101005c0000 fDelete this line.# mpath13 3600000e00d2b0000002b010100710000 fDelete this line.# mpath14 3600000e00d2b0000002b010100720000 fDelete this line.# mpath15 3600000e00d2b0000002b010100730000 fDelete this line.# mpath16 3600000e00d2b0000002b010100740000 fDelete this line.mpath1 3600000e00d2b0000002b010100590000 fAdd this line.mpath2 3600000e00d2b0000002b0101005a0000 fAdd this line.mpath3 3600000e00d2b0000002b0101005b0000 fAdd this line.mpath4 3600000e00d2b0000002b0101005c0000 fAdd this line.mpath5 3600000e00d2b0000002b010100710000 fAdd this line.mpath6 3600000e00d2b0000002b010100720000 fAdd this line.mpath7 3600000e00d2b0000002b010100730000 fAdd this line.mpath8 3600000e00d2b0000002b010100740000 fAdd this line.

#




8 Execute the "multipath -r" command to apply the file changes to the system.

# multipath -r

3600000e00d2b0000002b010100590000: rename mpath9 to mpath1: mpath1 (3600000e00d2b0000002b010100590000) FUJITSU,ETERNUS_DXH[size=20G][features=0][hwhandler=0][n/a]\_ round-robin 0 [prio=50][undef] \_ 5:0:0:0 sdb 8:16 [active][ready] \_ 6:0:0:0 sdj 8:144 [active][ready]3600000e00d2b0000002b0101005a0000: rename mpath10 to mpath2: mpath2 (3600000e00d2b0000002b0101005a0000) FUJITSU,ETERNUS_DXH[size=20G][features=0][hwhandler=0][n/a]\_ round-robin 0 [prio=50][undef] \_ 5:0:0:1 sdc 8:32 [active][ready] \_ 6:0:0:1 sdk 8:160 [active][ready]3600000e00d2b0000002b0101005b0000: rename mpath11 to mpath3: mpath3 (3600000e00d2b0000002b0101005b0000) FUJITSU,ETERNUS_DXH[size=20G][features=0][hwhandler=0][n/a]\_ round-robin 0 [prio=50][undef] \_ 5:0:0:2 sdd 8:48 [active][ready] \_ 6:0:0:2 sdl 8:176 [active][ready]3600000e00d2b0000002b0101005c0000: rename mpath12 to mpath4: mpath4 (3600000e00d2b0000002b0101005c0000) FUJITSU,ETERNUS_DXH[size=20G][features=0][hwhandler=0][n/a]\_ round-robin 0 [prio=50][undef] \_ 5:0:0:3 sde 8:64 [active][ready] \_ 6:0:0:3 sdm 8:192 [active][ready]3600000e00d2b0000002b010100710000: rename mpath13 to mpath5: mpath5 (3600000e00d2b0000002b010100710000) FUJITSU,ETERNUS_DXH[size=20G][features=0][hwhandler=0][n/a]\_ round-robin 0 [prio=50][undef] \_ 5:0:0:4 sdf 8:80 [active][ready] \_ 6:0:0:4 sdn 8:208 [active][ready]3600000e00d2b0000002b010100720000: rename mpath14 to mpath6: mpath6 (3600000e00d2b0000002b010100720000) FUJITSU,ETERNUS_DXH[size=20G][features=0][hwhandler=0][n/a]\_ round-robin 0 [prio=50][undef] \_ 5:0:0:5 sdg 8:96 [active][ready] \_ 6:0:0:5 sdo 8:224 [active][ready]3600000e00d2b0000002b010100730000: rename mpath15 to mpath7: mpath7 (3600000e00d2b0000002b010100730000) FUJITSU,ETERNUS_DXH[size=20G][features=0][hwhandler=0][n/a]\_ round-robin 0 [prio=50][undef] \_ 5:0:0:6 sdh 8:112 [active][ready] \_ 6:0:0:6 sdp 8:240 [active][ready]3600000e00d2b0000002b010100740000: rename mpath16 to mpath8: mpath8 (3600000e00d2b0000002b010100740000) FUJITSU,ETERNUS_DXH[size=20G][features=0][hwhandler=0][n/a]\_ round-robin 0 [prio=50][undef] \_ 5:0:0:7 sdi 8:128 [active][ready] \_ 6:0:0:7 sdq 65:0 [active][ready]

#

9 Mount the target LUN.




10 Edit the "/etc/fstab" file, and change the target LUN so that it will mount.

# vi /etc/fstab

/dev/VolGroup00/LogVol00 / ext3 defaults 1 1LABEL=/boot /boot ext3 defaults 1 2tmpfs /dev/shm tmpfs defaults 0 0devpts /dev/pts devpts gid=5,mode=620 0 0sysfs /sys sysfs defaults 0 0proc /proc proc defaults 0 0/dev/VolGroup00/LogVol01 swap swap defaults 0 0/dev/mapper/mpath1p1 /mnt/lun0 ext3 defaults 0 0 fUncomment this line./dev/mapper/mpath2p1 /mnt/lun1 ext2 defaults 0 0 fUncomment this line./dev/mapper/mpath5p1 /mnt/lun4 ext3 defaults 0 0 fUncomment this line./dev/mapper/mpath6p1 /mnt/lun5 ext4 defaults 0 0 fUncomment this line./dev/vg1/lv1 /mnt/lvm0 ext4 defaults 0 0 fUncomment this line.

#

After this procedure is completed, the LUNs can be used in the migration destination ETERNUS AF/DX thesame way as before the migration.




Chapter 12Non-disruptive Storage Migration

This chapter describes the procedures for connecting and disconnecting paths and provides notes for whenthe Non-disruptive Storage Migration function is used in the example device-mapper multipath environmentthat runs RHEL6.5.

■ Connecting Paths

The following procedure shows how to add a path to the migration destination storage system from the serv-er (RHEL6.5) after the migration destination storage system is connected.

1 Check the multipath status.

In the following example, each LUN has two paths.

[root@S6-12 ~]# multipath -llmpathc (36000b5d0006a0000006a0ba0001a0000) dm-3 FUJITSU,ETERNUS_DX400size=3.9G features='0' hwhandler='0' wp=rw`-+- policy='round-robin 0' prio=130 status=active |- 6:0:1:1 sdc 8:32 active ready running `- 5:0:1:1 sdo 8:224 active ready runningmpathb (36000b5d0006a0000006a0ba000190000) dm-2 FUJITSU,ETERNUS_DX400size=3.9G features='0' hwhandler='0' wp=rw`-+- policy='round-robin 0' prio=130 status=active |- 6:0:1:0 sdb 8:16 active ready running `- 5:0:1:0 sdn 8:208 active ready running[root@S6-12 ~]#

2 Connect the multipath.

Add the host affinity setting to the migration destination storage system.

3 Rescan the HBA.

[root@S6-12 ~]# for s in /sys/class/fc_host/host*/device/scsi_host/host*/scan; do echo '- - -' > $s; done[root@S6-12 ~]#



4 Confirm that the path has been connected.

In the following example, two paths are added to each LUN and they becomes a four-path configura-tion.

[root@S6-12 ~]# multipath -llmpathc (36000b5d0006a0000006a0ba0001a0000) dm-3 FUJITSU,ETERNUS_DX400size=3.9G features='0' hwhandler='0' wp=rw`-+- policy='round-robin 0' prio=130 status=active |- 6:0:1:1 sdc 8:32 active ready running |- 5:0:1:1 sdo 8:224 active ready running |- 6:0:0:1 sdam 66:96 active ready running `- 5:0:0:1 sdaa 65:160 active ready runningmpathb (36000b5d0006a0000006a0ba000190000) dm-2 FUJITSU,ETERNUS_DX400size=3.9G features='0' hwhandler='0' wp=rw`-+- policy='round-robin 0' prio=130 status=active |- 6:0:1:0 sdb 8:16 active ready running |- 5:0:1:0 sdn 8:208 active ready running |- 5:0:0:0 sdz 65:144 active ready running `- 6:0:0:0 sdal 66:80 active ready running

If the path is not connected, perform the procedure in Step 5.

5 Reload the multipath setting.

[root@S6-12 ~]# service multipathd reloadReloading multipathd: [ OK ][root@S6-12 ~]# multipath -llmpathc (36000b5d0006a0000006a0ba0001a0000) dm-3 FUJITSU,ETERNUS_DX400size=3.9G features='0' hwhandler='0' wp=rw`-+- policy='round-robin 0' prio=130 status=active |- 6:0:1:1 sdc 8:32 active ready running |- 5:0:1:1 sdo 8:224 active ready running |- 6:0:0:1 sdam 66:96 active ready running `- 5:0:0:1 sdaa 65:160 active ready runningmpathb (36000b5d0006a0000006a0ba000190000) dm-2 FUJITSU,ETERNUS_DX400size=3.9G features='0' hwhandler='0' wp=rw`-+- policy='round-robin 0' prio=130 status=active |- 6:0:1:0 sdb 8:16 active ready running |- 5:0:1:0 sdn 8:208 active ready running |- 5:0:0:0 sdz 65:144 active ready running `- 6:0:0:0 sdal 66:80 active ready running[root@S6-12 ~]#

Chapter 12 Non-disruptive Storage Migration



■ Disconnecting the Path

The following procedure shows how to delete a path of the migration destination storage system from theserver (RHEL6.5) after the migration source storage system is disconnected.

1 Check the multipath status.

In the following example, each LUN has four paths.

[root@S6-12 ~]# multipath -llmpathc (36000b5d0006a0000006a0ba0001a0000) dm-3 FUJITSU,ETERNUS_DX400size=3.9G features='0' hwhandler='0' wp=rw`-+- policy='round-robin 0' prio=130 status=active |- 6:0:1:1 sdc 8:32 active ready running |- 5:0:1:1 sdo 8:224 active ready running |- 6:0:0:1 sdam 66:96 active ready running `- 5:0:0:1 sdaa 65:160 active ready runningmpathb (36000b5d0006a0000006a0ba000190000) dm-2 FUJITSU,ETERNUS_DX400size=3.9G features='0' hwhandler='0' wp=rw`-+- policy='round-robin 0' prio=130 status=active |- 6:0:1:0 sdb 8:16 active ready running |- 5:0:1:0 sdn 8:208 active ready running |- 5:0:0:0 sdz 65:144 active ready running `- 6:0:0:0 sdal 66:80 active ready running[root@S6-12 ~]#

2 Disconnect the multipath.

Disconnect the paths between the migration source storage system and the server.

3 Reload the multipath setting.

[root@S6-12 ~]# service multipathd reloadReloading multipathd: [ OK ][root@S6-12 ~]#




4 Confirm that the path has been disconnected.

In the following example, two paths are deleted from each LUN which then becomes a two-path config-uration.

[root@S6-12 ~]# multipath -llmpathc (36000b5d0006a0000006a0ba0001a0000) dm-3 FUJITSU,ETERNUS_DX400size=3.9G features='0' hwhandler='0' wp=rw`-+- policy='round-robin 0' prio=130 status=active |- 5:0:0:1 sdaa 65:160 active ready running `- 6:0:0:1 sdam 66:96 active ready runningmpathb (36000b5d0006a0000006a0ba000190000) dm-2 FUJITSU,ETERNUS_DX400size=3.9G features='0' hwhandler='0' wp=rw`-+- policy='round-robin 0' prio=130 status=active |- 5:0:0:0 sdz 65:144 active ready running `- 6:0:0:0 sdal 66:80 active ready running[root@S6-12 ~]#

■ Notes

If the ALUA setting differs between the migration source and migration destination storage systems, all thepaths in the storage system where the ALUA setting is "ACTIVE / ACTIVE" and the priority paths in the storagesystem where the ALUA setting is "ACTIVE-ACTIVE / PREFERRED_PATH" are set as the priority path group whilethe path is connected. After the path is disconnected, the path group specified in the ALUA setting of themigration destination storage system is used.

The following shows the path state before the migration, while the path is connected, and after the path isdisconnected.

• Before the migration

Because the migration source is "ACTIVE / ACTIVE", both paths become the priority path group.

mpathe (36000b5d0006a0000006a0ba0001c0000) dm-5 FUJITSU,ETERNUS_DX400size=3.9G features='0' hwhandler='0' wp=rw`-+- policy='round-robin 0' prio=130 status=active |- 2:0:1:3 sde 8:64 active ready running `- 1:0:1:3 sdq 65:0 active ready running

• While the path is connected

Two "ACTIVE / ACTIVE" paths in the migration source storage system and a single priority "ACTIVE-ACTIVE /PREFERRED_PATH" path in the migration destination storage system become the priority path group.

mpathe (36000b5d0006a0000006a0ba0001c0000) dm-5 FUJITSU,ETERNUS_DX400size=3.9G features='0' hwhandler='0' wp=rw|-+- policy='round-robin 0' prio=130 status=active| |- 2:0:1:3 sde 8:64 active ready running| |- 1:0:1:3 sdq 65:0 active ready running| `- 1:0:0:3 sdac 65:192 active ready running`-+- policy='round-robin 0' prio=10 status=enabled `- 2:0:0:3 sdao 66:128 active ready running




• After the path is disconnected

Because the migration destination is "ACTIVE-ACTIVE / PREFERRED_PATH", one of the priority paths be-comes the priority path group.

mpathe (36000b5d0006a0000006a0ba0001c0000) dm-5 FUJITSU,ETERNUS_DX400size=3.9G features='0' hwhandler='0' wp=rw|-+- policy='round-robin 0' prio=130 status=active| `- 1:0:0:3 sdac 65:192 active ready running`-+- policy='round-robin 0' prio=10 status=enabled `- 2:0:0:3 sdao 66:128 active ready running




Chapter 13Settings for When the Number of LUNs Exceeds256 in Red Hat Enterprise Linux

This chapter describes how to set the number of LUNs that can be recognized by Red Hat Enterprise Linux if itexceeds 256. The procedure differs depending on the version of Red Hat Enterprise Linux that is being used.

The server load, and the I/O load to the ETERNUS AF/DX may increase due to an increase in the number ofLUNs that are recognized by the server.

In a state where intermittent high load conditions (*1) are occurring, if one of the paths fails due to apath failure between the server and storage system, multipath reconfiguration may take three hours ormore after recovery from the path failure.

*1 : The state where the resource usage of the server and ETERNUS AF/DX is extremely high such asthe following.

• When a server side resource monitoring is performed with the "sar" command, the Idle state is0%.

• When a monitoring is performed with PMCC, the physical disk usage of the ETERNUS AF/DX is100%.

13.1 For Red Hat Enterprise Linux 8.x or Red Hat Enterprise Linux7.x

This section describes a case in which the UEFI mode or the legacy mode (BIOS-compatibility mode) is usedfor the stored OS area.

After the setting is complete, the settings for the HBA parameters (Emulex only) and the ETERNUS AF/DX arerequired.

Refer to the following documents for details about the settings for the HBA parameters and the ETERNUSAF/DX.





■ When the UEFI Mode Is Used for the Stored OS Area

This section describes an example to get 1024 LUNs recognized by Red Hat Enterprise Linux 8.0.

1 Add "scsi_mod.max_luns" and "scsi_mod.max_report_luns" to the end of the "GRUB_CMDLINE_LINUX"line in the "/etc/default/grub" file.

:GRUB_CMDLINE_LINUX="crashkernel=auto rd.lvm.lv=rhel/root rd.lvm.lv=rhel/swap rhgb quiet scsi_mod.max_luns=1024 scsi_mod.max_report_luns=1024":

• In the underlined portion, specify the number of LUNs to get recognized by the server.

• For the maximum number of LUNs that can be set in the ETERNUS AF/DX, refer to the manuals ofeach storage system.

2 Execute the following command:

# grub2-mkconfig -o /boot/efi/EFI/redhat/grub.cfg

3 Check the contents of "/boot/efi/EFI/redhat/grubenv".

# cat /boot/efi/EFI/redhat/grubenv | grep scsikernelopts=root=/dev/mapper/rhel-root ro crashkernel=auto resume=/dev/mapper/rhel-swap rd.lvm.lv=rhel/root rd.lvm.lv=rhel/swap rhgb quiet scsi_mod.max_luns=1024 scsi_mod.max_report_luns=1024#

4 Reboot the server.

Chapter 13 Settings for When the Number of LUNs Exceeds 256 in Red Hat Enterprise Linux13.1 For Red Hat Enterprise Linux 8.x or Red Hat Enterprise Linux 7.x



■ When the Legacy Mode (BIOS-Compatibility Mode) Is Used for the Stored OS Area


1 Add "scsi_mod.max_luns" and "scsi_mod.max_report_luns" to the end of the "GRUB_CMDLINE_LINUX"line in the "/etc/default/grub" file.

:GRUB_CMDLINE_LINUX="crashkernel=auto rd.lvm.lv=rhel/root rd.lvm.lv=rhel/swap rhgb quiet scsi_mod.max_luns=1024 scsi_mod.max_report_luns=1024":



2 Execute the following command:

# grub2-mkconfig -o /boot/grub2/grub.cfg

3 Check the contents of "/etc/grub2.cfg".

# cat /boot/grub2/grub.cfg | grep scsi linux16 /vmlinuz-3.10.0-862.el7.x86_64 root=/dev/mapper/rhel-root ro crashkernel=auto rd.lvm.lv=rhel/root rd.lvm.lv=rhel/swap rhgb quiet scsi_mod.max_luns=1024 scsi_mod.max_report_luns=1024 linux16 /vmlinuz-0-rescue-6421e10563d2430daf5184aa9572c1b4 root=/dev/mapper/rhel-root ro crashkernel=auto rd.lvm.lv=rhel/root rd.lvm.lv=rhel/swap rhgb quiet scsi_mod.max_luns=1024 scsi_mod.max_report_luns=1024#

4 Reboot the server.

Chapter 13 Settings for When the Number of LUNs Exceeds 256 in Red Hat Enterprise Linux13.1 For Red Hat Enterprise Linux 8.x or Red Hat Enterprise Linux 7.x



13.2 For Red Hat Enterprise Linux 6.x


1 Add "scsi_mod.max_luns" and "scsi_mod.max_report_luns" to the end of the "kernel" line in the "/etc/grub.conf" file.

:kernel /vmlinuz-2.6.32-573.1.1.el6.x86_64 ro root=/dev/mapper/vg_rx3s71rh67x64-lv_root nomodeset rd_NO_LUKS rd_LVM_LV=vg_rx3s71rh67x64/lv_root LANG=en_US.UTF-8 rd_LVM_LV=vg_rx3s71rh67x64/lv_swap rd_NO_MD SYSFONT=latarcyrheb-sun16 crashkernel=128M KEYBOARDTYPE=pc KEYTABLE=jp106 rd_NO_DM rhgb quiet scsi_mod.max_luns=1024 scsi_mod.max_report_luns=1024:



• Because the "kernel" line is long, make sure to add them to the correct location.

2 Update the initial RAM disk and reboot the server.





Chapter 13 Settings for When the Number of LUNs Exceeds 256 in Red Hat Enterprise Linux13.2 For Red Hat Enterprise Linux 6.x



13.3 For Red Hat Enterprise Linux 5.x


This procedure is available for Red Hat Enterprise Linux 5.7 or later.

1 Write "options scsi_mod max_luns" or "max_report_luns" to the "/etc/modprobe.conf" file.

:alias scsi_hostadapter megaraid_sasalias scsi_hostadapter1 ata_piixalias scsi_hostadapter2 qla2xxxoptions scsi_mod max_luns=512 max_report_luns=512:


• If the ETERNUS Multipath Driver or the device-mapper multipath is used, the maximum number ofLUNs that can be set is 512.

• If the ETERNUS Multipath Driver or the device-mapper multipath is not used, the maximum num-ber of LUNs that can be set is equal to the maximum number of LUNs supported by each storagesystem. For the maximum number of LUNs, refer to the manuals of each storage system.

2 Update the initial RAM disk and reboot the server.





Chapter 13 Settings for When the Number of LUNs Exceeds 256 in Red Hat Enterprise Linux13.3 For Red Hat Enterprise Linux 5.x



Chapter 14Setup When Using SUSE Linux Enterprise Server15 SP3

This chapter describes how to avoid problems when using SUSE Linux Enterprise Server 15 SP3. In SUSE LinuxEnterprise Server 15 SP3, if the OS is started with a number of LUNs that have been recognized, one of thefollowing problems may occur.

• Problem 1

Although the OS started successfully, the following message appears.

Failed to update device symlinks: No space left on device

• Problem 2

The OS failed to start up.

The number of LUNs that triggers these problems varies depending on the hardware configuration of theserver.

If these problems occur, work around the problem as follows.

■ Problem 1

1 Create a directory using the following command.

# mkdir /usr/lib/dracut/modules.d/99mymodule

2 Create the following file under the directory /usr/lib/dracut/modules.d/99mymodule created in Step 1.

module-setup.sh



3 Add the following strings in the created module-setup.sh file and save it.

#!/bin/bash # called by dracutcheck() { require_binaries grep awk mount} # called by dracutdepends() { return 0} # called by dracutinstall() { inst_multiple grep awk mount inst_hook pre-udev 99 "$moddir/remount-dev.sh"}


remount-dev.sh

5 Add the following strings in the created remount-dev.sh file and save it.

#!/bin/bash

NR_INODES=$((`grep MemTotal /proc/meminfo | awk '{print $2}'`/4/2))mount -o remount,nr_inodes=$NR_INODES /dev

6 Execute the following command to recreate "initrd".

# mkinitrd

7 Execute the following command to shut down the OS.

# shutdown -h now

8 Start the OS.

9 Confirm that the OS starts successfully and all the assigned LUNs are recognized.

Chapter 14 Setup When Using SUSE Linux Enterprise Server 15 SP3



■ Problem 2

1 Shut down the OS.

2 Release the LUNs that are assigned to the OS.

Operate the ETERNUS AF/DX to release the host affinity setting.

3 Create a directory using the following command.

# mkdir /usr/lib/dracut/modules.d/99mymodule


module-setup.sh

5 Add the following strings in the created module-setup.sh file and save it.

#!/bin/bash # called by dracutcheck() { require_binaries grep awk mount} # called by dracutdepends() { return 0} # called by dracutinstall() { inst_multiple grep awk mount inst_hook pre-udev 99 "$moddir/remount-dev.sh"}


remount-dev.sh

7 Add the following strings in the created remount-dev.sh file and save it.

#!/bin/bash

NR_INODES=$((`grep MemTotal /proc/meminfo | awk '{print $2}'`/4/2))mount -o remount,nr_inodes=$NR_INODES /dev




8 Execute the following command to recreate "initrd".

# mkinitrd

9 Execute the following command to shut down the OS.

# shutdown -h now

10 Assign LUNs to the OS.

Operate the ETERNUS AF/DX to create host affinity.

11 Start the OS.

12 Confirm that the OS starts successfully and all the assigned LUNs are recognized.




Appendix ARestoration Procedure for when a Path Degradesin a Veritas Volume Manager Environment

If a path degrades in a Veritas Volume Manager environment, some volumes may not be recognized properlyafter path restoration.

In this case, volume recognition is required after restoring LUN recognition using the Change LUN State Utili-ty provided by each Fibre Channel card vendor.

1 Check the Fibre Channel card's controller number and status.

Use the "vxdmpadm" command to check the controller number and controller status of the Fibre Chan-nel card that corresponds to the unrecognized volume.

(1) Check the controller number of the Fibre Channel card in which there is a path connected to theETERNUS AF/DX.

# vxdmpadm listctlr all

Example (for ETERNUS DX410 S2)

# vxdmpadm listctlr allCTLR-NAME ENCLR-TYPE STATE ENCLR-NAME=========================================================c0 Disk ENABLED Diskc6 FJ_DX400 ENABLED fj_dx400c7 FJ_DX400 ENABLED fj_dx400

In this example, the controller numbers of the Fibre Channel cards connected to the ETERNUSDX410 S2 are "c6" and "c7".

(2) Check the status of volume paths connected to each Fibre Channel card using the controller num-bers that were checked in Step (1).

In this step, check the controller number corresponding to the volume for which "DISABLED" is dis-played.

# vxdmpadm getsubpaths ctlr=c

The underlined portion indicates the controller number.




# vxdmpadm getsubpaths ctlr=c6NAME STATE PATH-TYPE DMPNODENAME ENCLR-TYPE ENCLR-NAME=========================================================================sdb ENABLED - sdb FJ_DX400 fj_dx400sdc ENABLED - sdc FJ_DX400 fj_dx400sdd ENABLED - sdd FJ_DX400 fj_dx400

# vxdmpadm getsubpaths ctlr=c7NAME STATE PATH-TYPE DMPNODENAME ENCLR-TYPE ENCLR-NAME=========================================================================sde DISABLED - sdb FJ_DX400 fj_dx400sdf ENABLED - sdc FJ_DX400 fj_dx400sdg ENABLED - sdd FJ_DX400 fj_dx400

In this example, "DISABLED" is displayed for the "sde" volume.

The controller number of the Fibre Channel card in which the volume cannot be recognized is "c7".

2 Obtain and apply the Change LUN State Utility.

(1) Obtain the Change LUN State Utility provided by each Fibre Channel card vendor.

• For QLogic Fibre Channel cards

Obtain the Change LUN State Utility from the QLogic web-site (https://www.marvell.com/).

• For Emulex Fibre Channel cards

Obtain the Change LUN State Utility from the Emulex web-site (https://www.broadcom.com/).

Refer to each vendor web-site for details about the Change LUN State Utility.

(2) Apply the Change LUN State Utility that is obtained in Step (1) according to the procedure speci-fied by each vendor to change the state of the LUN to recognized.

Refer to each vendor web-site for details about how to apply the Change LUN State Utility.

3 Enable Veritas Volume Manager.

Execute the "vxdctl" command to enable Veritas Volume Manager.

# vxdctl enable

4 Check the volume status.

Check that the status of a volume that was "DISABLED" in Step (2) is changed to "ENABLED".

# vxdmpadm getsubpaths ctlr=c

The underlined portion indicates the controller number.

Appendix A Restoration Procedure for when a Path Degrades in a Veritas Volume Manager Environment




# vxdmpadm getsubpaths ctlr=c7NAME STATE PATH-TYPE DMPNODENAME ENCLR-TYPE ENCLR-NAME==========================================================================sde ENABLED - sdb FJ_DX400 fj_dx400sdf ENABLED - sdc FJ_DX400 fj_dx400sdg ENABLED - sdd FJ_DX400 fj_dx400

Appendix A Restoration Procedure for when a Path Degrades in a Veritas Volume Manager Environment



FUJITSU Storage ETERNUS AF, ETERNUS DX Configuration Guide -Server Connection-(Fibre Channel) for Linux

P3AM-3842-32ENZ0

Date of issuance: February 2022Issuance responsibility: FUJITSU LIMITED

• The content of this manual is subject to change without notice.

• This manual was prepared with the utmost attention to detail. However, Fujitsu shall assume no responsibility for anyoperational problems as the result of errors, omissions, or the use of information in this manual.

• Fujitsu assumes no liability for damages to third party copyrights or other rights arising from the use of any informationin this manual.

• The content of this manual may not be reproduced or distributed in part or in its entirety without prior permission fromFujitsu.

Configuration Guide -Server Connection - Fujitsu

Documents

Transcript of Configuration Guide -Server Connection - Fujitsu