Multipathing, RAC et ASM

Voici un petit cas concret rencontré chez un « célèbre client » Parisien où nous avons fait la découverte sur l’un des environnements ISO PROD en RAC que certains disques ASM ne bénéficiaient pas de multipathing pourtant si cher à la haute disponibilité.
Vous trouverez donc ci-après un petit rappel de la configuration du multipathing sous Linux RedHat pour architecture RAC.

  • Positionnement de l’environnement ASM
[root@rac1-iso log]# su -
[root@rac1-iso ~]# . oraenv
ORACLE_SID = [root] ? +ASM1
The Oracle base has been set to /app/oracle
  •  Arrêt du clusterware Oracle pour le noeud :
[root@rac1-iso ~]# crsctl stop crs
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rac1-iso'
CRS-2673: Attempting to stop 'ora.crsd' on 'rac1-iso'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'rac1-iso'
CRS-2673: Attempting to stop 'ora.DG_GRID.dg' on 'rac1-iso'
CRS-2673: Attempting to stop 'ora.serv.db' on 'rac1-iso'
CRS-2673: Attempting to stop 'ora.registry.acfs' on 'rac1-iso'
CRS-2673: Attempting to stop 'ora.cvu' on 'rac1-iso'
...
CRS-4133: Oracle High Availability Services has been stopped.
  • Contrôle des [minor,major] des disques ASM
[root@rac1-iso ~]# ls -l /dev/oracleasm/disks/
total 0
brw-rw---- 1 oracle dba  8,   1 Nov  8 12:23 ASM_ARCH_01
brw-rw---- 1 oracle dba  8,  17 Nov  8 11:47 ASM_ARCH_02
brw-rw---- 1 oracle dba 65, 161 Nov  8 12:23 ASM_ARCH_03
brw-rw---- 1 oracle dba 65, 177 Nov  8 11:47 ASM_ARCH_04
...
  • Contrôle du device associé au disque ASM
[root@rac1-iso ~]# oracleasm querydisk -d ASM_ARCH_01
Disk "ASM_ARCH_01" is a valid ASM disk on device [8, 1]
[root@rac1-iso ~]# oracleasm querydisk -p ASM_ARCH_01
Disk "ASM_ARCH_01" is a valid ASM disk
/dev/mapper/eva4400-6-iso-arch001p1: LABEL="ASM_ARCH_01" TYPE="oracleasm"
/dev/sdbe1: LABEL="ASM_ARCH_01" TYPE="oracleasm"
/dev/sdac1: LABEL="ASM_ARCH_01" TYPE="oracleasm"
/dev/sda1: LABEL="ASM_ARCH_01" TYPE="oracleasm"
/dev/sdcg1: LABEL="ASM_ARCH_01" TYPE="oracleasm"
  • Recherche du device à partir du [minor,major]
[root@rac1-iso ~]# ls -l /dev/sd* | grep "8,   1"
brw-r----- 1 root disk 8, 1 Oct 26 11:47 /dev/sda1
  • Arrêt de la librairie oracleasm:
[root@rac1-iso ~]# /etc/init.d/oracleasm stop
Dropping Oracle ASMLib disks:                              [  OK  ]
Shutting down the Oracle ASMLib driver:                    [  OK  ]
  • Modification du fichier /etc/sysconfig/oracleasm :

   ORACLEASM_SCANEXCLUDE= »sd »

  • Relance de la librairie oracleasm
[root@rac1-iso ~]# /etc/init.d/oracleasm start
Initializing the Oracle ASMLib driver:                     [  OK  ]
Scanning the system for Oracle ASMLib disks:               [  OK  ]
  • Contrôle du changement des devices associés aux disques ASM
[root@rac1-iso ~]# ls -l /dev/oracleasm/disks/
total 0
brw-rw---- 1 oracle dba 253, 40 Nov  8 12:30 ASM_ARCH_01
brw-rw---- 1 oracle dba 253, 38 Nov  8 12:30 ASM_ARCH_02
brw-rw---- 1 oracle dba 253, 60 Nov  8 12:30 ASM_ARCH_03
brw-rw---- 1 oracle dba 253, 58 Nov  8 12:30 ASM_ARCH_04
...
[root@rac1-iso ~]# oracleasm querydisk -d ASM_ARCH_01
Disk "ASM_ARCH_01" is a valid ASM disk on device [253, 40]
[root@rac1-iso ~]# oracleasm querydisk -p ASM_ARCH_01
Disk "ASM_ARCH_01" is a valid ASM disk
/dev/mapper/eva4400-6-iso-arch001p1: LABEL="ASM_ARCH_01" TYPE="oracleasm"
/dev/sdbe1: LABEL="ASM_ARCH_01" TYPE="oracleasm"
/dev/sdac1: LABEL="ASM_ARCH_01" TYPE="oracleasm"
/dev/sda1: LABEL="ASM_ARCH_01" TYPE="oracleasm"
/dev/sdcg1: LABEL="ASM_ARCH_01" TYPE="oracleasm"
  • Recherche du device à partir du [minor,major] après changement :
[root@rac1-iso ~]# ls -l /dev/mapper/ | grep 253 | grep '40 '
brw-rw---- 1 root disk 253, 40 Oct 26 11:46 eva4400-6-iso-arch001p1
  • Vérification du multipathing :
[root@rac1-iso ~]# multipath -l eva4400-6-iso-arch001
eva4400-6-iso-arch001 (36001438007f2d6180000400000120000) dm-9 HP,HSV300
[size=100G][features=1 queue_if_no_path][hwhandler=0][rw]
_ round-robin 0 [prio=0][active]
 _ 2:0:0:1  sda  8:0    [active][undef]
 _ 3:0:1:1  sdcg 69:64  [active][undef]
_ round-robin 0 [prio=0][enabled]
 _ 2:0:1:1  sdac 65:192 [active][undef]
 _ 3:0:0:1  sdbe 67:128 [active][undef]
  • Relance du clusterware Oracle pour le noeud:
[root@rac1-iso ~]# . oraenv
ORACLE_SID = [+ASM1] ? +ASM1
The Oracle base remains unchanged with value /app/oracle
[root@rac1-iso ~]# crsctl start crs
CRS-4123: Oracle High Availability Services has been started.

Et voilà !