Toutes les horloges des différents composants EXADATA doivent être impérativement synchronisées entre elles.
Dans le cas contraire, nous pouvons être confrontés par exemple à ce type d’incidents :
- Ressource CRS avec un statut UNKNOWN ou OFFLINE
- Reboot d’un noeud déclenché par le CRS
- Installation défectueuse de binaires Oracle sur le remote serveur
- etc …
Nous allons donc détailler au cours de cet article les différentes étapes de mise à jour de la configuration NTP sur un EXADATA 1/4 rack (version > X3.2).
La configuration NTP peut être mise à jour en mode Rolling afin de ne pas interrompre l’activité applicative.
Database Node
- dbnode1 :192.168.36.70
- dbnode2 : 192.168.36.71
Arrêt du service NTP
# service ntpd stop
Mise àjour du fichier de configuration /etc/ntp.conf
# Permit time synchronization with our time source, but do not # permit the source to query or modify the service on this system. restrict default ignore restrict -6 default ignore ... restrict 192.168.35.48 mask 255.255.255.255 nomodify notrap noquery server 192.168.35.48 prefer iburst burst minpoll 4 maxpoll 4 restrict 192.168.35.49 mask 255.255.255.255 nomodify notrap noquery server 192.168.35.49 iburst burst minpoll 4 maxpoll 4 #### END Generated by Exadata ####
Démarrage du service NTP
# service ntpd start
Switch Infiniband
- sw-iba0 : 192.168.36.81
- sw-ibb0 : 192.168.36.82
Depuis un des 2 Database Nodes, se connecter au Switch Infiniband avec le user ilom-admin
# ssh -l ilom-admin 192.168.36.81 password
Ne pas éditer le fichier de configuration /etc/ntp.conf
Oracle(R) Integrated Lights Out Manager Version ILOM 3.0 r47111 Copyright (c) 2012, Oracle and/or its affiliates. All rights reserved. -> set /SP/clock timezone=CET -> set /SP/clients/ntp/server/1 address=192.168.35.48 -> set /SP/clients/ntp/server/2 address=192.168.35.49 -> set /SP/clock usentpserver=enabled
Switch Ethernet
- sw-adm0 : 192.168.36.80
Depuis un des 2 Database Nodes, se connecter au Switch Ethernet
# ssh admin@192.168.36.80 Password: sw-adm0>enable Password: sw-adm0#
Mettre à jour la configuration NTP
sw-adm0# show running-config … ntp server 192.168.35.6 prefer ntp server 192.168.35.7 end sw-adm0# configure terminal Enter configuration commands,one per line.End with CNTL/Z. Switch(config)# no ntp server 192.168.35.6 Switch(config)# no ntp server 192.168.35.7 Switch(config)# end Switch# write memory Building configuration... Compressed configuration from 2603 bytes to 1158 bytes [OK ] sw-adm0# configure terminal Enter configuration commands,one per line.End with CNTL/Z. Switch(config)# ntp server 192.168.35.48 prefer Switch(config)# ntp server 192.168.35.49 Switch(config)# end Switch# write memory Building configuration... Compressed configuration from 2603 bytes to 1158 bytes [OK ] sw-adm0# show running-config … ntp server 192.168.35.48 prefer ntp server 192.168.35.49 end sw-adm0# copy running-config startup-config Destination filename [startup-config]? Building configuration... Compressed configuration from 14343 bytes to 3986 bytes[OK] sw-adm0# exit
Storage Server
- celadm01 : 192.168.36.88
- celadm02 : 192.168.36.89
- celadm03 : 192.168.36.90
Vérifier au préalable le statut de tous les grid disks de la cellule
# cellcli -e list griddisk attributes name,asmmodestatus,asmdeactivationoutcome
DATA_CD_00_celadm01 ONLINE Yes
DATA_CD_01_celadm01 ONLINE Yes
DATA_CD_02_celadm01 ONLINE Yes
DATA_CD_03_celadm01 ONLINE Yes
DATA_CD_01_celadm01 ONLINE Yes
DATA_CD_05_celadm01 ONLINE Yes
DATA_CD_06_celadm01 ONLINE Yes
DATA_CD_07_celadm01 ONLINE Yes
DATA_CD_08_celadm01 ONLINE Yes
DATA_CD_09_celadm01 ONLINE Yes
DATA_CD_10_celadm01 ONLINE Yes
DATA_CD_11_celadm01 ONLINE Yes
DBFS_DG_CD_02_celadm01 ONLINE Yes
DBFS_DG_CD_03_celadm01 ONLINE Yes
DBFS_DG_CD_01_celadm01 ONLINE Yes
DBFS_DG_CD_05_celadm01 ONLINE Yes
DBFS_DG_CD_06_celadm01 ONLINE Yes
DBFS_DG_CD_07_celadm01 ONLINE Yes
DBFS_DG_CD_08_celadm01 ONLINE Yes
DBFS_DG_CD_09_celadm01 ONLINE Yes
DBFS_DG_CD_10_celadm01 ONLINE Yes
DBFS_DG_CD_11_celadm01 ONLINE Yes
RECO_CD_00_celadm01 ONLINE Yes
RECO_CD_01_celadm01 ONLINE Yes
RECO_CD_02_celadm01 ONLINE Yes
RECO_CD_03_celadm01 ONLINE Yes
RECO_CD_01_celadm01 ONLINE Yes
RECO_CD_05_celadm01 ONLINE Yes
RECO_CD_06_celadm01 ONLINE Yes
RECO_CD_07_celadm01 ONLINE Yes
RECO_CD_08_celadm01 ONLINE Yes
RECO_CD_09_celadm01 ONLINE Yes
RECO_CD_10_celadm01 ONLINE Yes
RECO_CD_11_celadm01 ONLINE Yes
Désactivation de tous les grid disks de la cellule
# cellcli -e alter griddisk all inactive
Arrêt de la cellule de stockage
# cellcli -e alter cell shutdown services all
Mise à jour de la configuration NTP via l’outil ipconf
# /opt/oracle.cellos/ipconf Logging started to /var/log/cellos/ipconf.log Interface ib0 is Linked. hca: mlx4_0 Interface ib1 is Linked. hca: mlx4_0 Interface eth0 is Linked. driver/mac: igb/00:21:28:8e:ab:d8 Interface eth1 is ... Unlinked. driver/mac: igb/00:21:28:8e:ab:d9 Interface eth2 is ... Unlinked. driver/mac: igb/00:21:28:8e:ab:da Interface eth3 is ... Unlinked. driver/mac: igb/00:21:28:8e:ab:db ... The current NTP server(s): 192.168.35.6 Do you want to change it (y/n) [n]:y ... Do you want to configure basic ILOM settings (y/n) [y]: Loading basic configuration settings from ILOM … ... ILOM First NTP server. Fully qualified hostname or ip address or none [192.168.35.6]: 192.168.35.48 ILOM Second NTP server. Fully qualified hostname or ip address or none [192.168.35.7]: 192.168.35.49 ... Is this correct (y/n) [y]: y
Démarrage de la cellule de stockage
# cellcli -e alter cell startup services all
Activation des grid disks
# cellcli -e alter griddisk all active
Vérification du statut des grid disks de la cellule de stockage
# cellcli -e list griddisk attributes name,asmmodestatus,asmdeactivationoutcome
Continuez la mise à jour de la prochaine cellule de stockage une fois tous les grid disks à l’état ONLINE.
