Oracle Scripts: RAC

Showing posts with label RAC. Show all posts

Wednesday, September 11, 2013

Hangcheck Timer Not Needed for 11gR2 RAC

Hangcheck Timer FAQ (Doc ID 232355.1)

Do I need the hangcheck-timer with 11gR2 ?

Answer

-----------

The hangcheck-timer is not needed with 11gR2. This is documented in 'Oracle® Grid Infrastructure Installation Guide 11g Release 2 (11.2) for Linux' section 'Improved Input/Output Fencing Processes'.

Oracle 11gR2 Whatsnew Doc

Improved Input/Output Fencing Processes

Oracle Clusterware 11g release 2 (11.2) replaces the oprocd and Hangcheck processes with the cluster synchronization service daemon Agent and Monitor to provide more accurate recognition of hangs and to avoid false termination.

Tuesday, January 29, 2013

Auto-Start Standby Database

1. Start Standalone Grid Infrastructure - High Availability Services (HAS)

host > $CRS_HOME/bin/crsctl config has

CRS-4621: Oracle High Availability Services autostart is disabled.

host > $CRS_HOME/bin/crsctl enable has

CRS-4622: Oracle High Availability Services autostart is enabled.

2. Change auto_start to “always” and the default startup mode to “mount”

srvctl modify database -d stndbydb -s mount

crsctl modify resource ora.stndbydb.db –attr “AUTO_START=always”

Test with crs_stat -p

RAC - Relocating VIP and SCAN

Failover VIP (on the destination node)
./crs_relocate [vip resource name]

The VIP will now go where it's configured to be

Failover SCAN
srvctl relocate scan -i [LISTENER_NUMBER] -n [DESTINATION_NODE_NAME]

RAC Cluster Name

$CRS_HOME/bin/cemutlo -n

Validate RAC Networking

Validate RAC Networking
a. Record IP’s and node names
  i. Run /sbin/ifconfig
  ii. Note private, public IP’s.
  iii. Example:
   1. node name : l6312
   2. Public IP : 10.118.49.25
   3. Private IP: 10.255.255.25
   4. node name : l6313
   5. Public IP : 10.118.49.26
   6. Private IP: 10.255.255.26
  iv. Run nslookup [scan name]
   1. Note IP’s returned
b. Verify Multicast (11.2.0.2 RAC specific) on all nodes
  i. /bin/netstat –in
  ii. Look for: eth0 and eth1, MTU = 1500
  iii. /sbin/ifconfig
  iv. Look for: “MULTICAST MTU:1500”
c. Test public IP’s on all nodes
  i. /bin/ping –s 1500 –c 2 –i [IP]
   1. Ping node1 => node1
   2. Ping node2 => node2
   3. Ping node1 => node2, etc
   4. Ping node2 => node1, etc
d. Test private IP’s on all nodes
  i. /bin/ping –s 1500 –c 2 –I [IP]
   1. Ping node1 => node1
   2. Ping node2 => node2
   3. Ping node1 => node2, etc
   4. Ping node2 => node1, etc
e. Test private IP’s traceroute
  i. /bin/traceroute –s [local private IP] –r –F [remote private IP] 1472
   1. Look for ONLY 1 hop to the remote private IP
f. Test VIP’s
  i. /bin/ping –c 2 [VIP name] from all to all
   1. Ping node1 => node1 vip
   2. Ping node2 => node2 vip
   3. Ping node1 => node2 vip, etc
   4. Ping node2 => node1 vip, etc
  ii. Note: These VIP’s could be on different nodes after clusterware is up
  iii. Look for: Successful pings
g. Test DNS setup
  i. /usr/bin/nslookup
   1. nslookup from node1,node2,etc. => VIP name
   2. nslookup from node1,node2,etc. => SCAN name
h. Verify name resolution order
  i. grep ^hosts /etc/nsswitch.conf
  ii. Look for : “files dns”
i. Verify /etc/hosts (check for all cluster members)
  i. grep [node1 hostname] /etc/hosts
  ii. grep [node2 hostname] /etc/hosts
  iii. grep [node1 VIP name] /etc/hosts
  iv. grep [node2 VIP name] /etc/hosts
  v. grep [node1 IP] /etc/hosts
  vi. grep [node2 IP] /etc/hosts
  vii. grep [node1 VIP] /etc/hosts
  viii. grep [node2 VIP] /etc/hosts
j. Verify scan is not in /etc/hosts
  i. grep [scan name] /etc/hosts
  ii. grep [SCAN IP] /etc/hosts (rerun for each SCAN IP)
2. Test CRS
a. Restart nodes one at a time
  i. Verify that resources fail-over
  ii. Verify that resources actually restart on the restarted node
   1. crsctl status resource –t
    a. check for gds,vip,listener,db(s),services,etc.
  iii. Follow crs logs, look for unexpected errors

srvctl Debugging / Trace

script /tmp/trace1.out
env
crsct stat res -t or crs_stat -t
export SRVM_TRACE=TRUE
srvctl relocate service -d mydb -s R_PHRDV -i mydb1 -t mydb2
export SRVM_TRACE=
crsct stat res -t or crs_stat -t
exit

Determining Interconnect IP Addresses

1. Using the dynamic view gv$cluster_interconnects:

select * from gv$cluster_interconnects ;

INST_ID    NAME            IP_ADDRESS       IS_ SOURCE
---------- --------------- ---------------- --- -------------------------------
1          eth0            192.168.10.1     NO  Oracle Cluster Repository
1          eth1            192.168.11.1     NO  Oracle Cluster Repository
2          eth0            192.168.10.2     NO  Oracle Cluster Repository
2          eth1            192.168.11.2     NO  Oracle Cluster Repository

In the above output, the column SOURCE indicates where the private interconnect info was derived from; this column could be one of OCR, OS dependent software or cluster_interconnects parameter.

2. Using the clusterware command oifcfg:

$oifcfg getif
eth2 10.104.95.0 global public
eth0 192.168.10.0 global cluster_interconnect
eth1 192.168.11.0 global cluster_interconnect

3. Using oradebug ipc:

sqlplus “/ as sysdba”

SQL>oradebug setmypid
Statement processed.
SQL>oradebug ipc
Information written to trace file.

The above command would dump a trace to user_dump_dest. The last few lines of the trace would indicate the IP of the cluster interconnect. Below is a sample output of those lines.

From the trace file on node1:

SSKGXPT 0x5edf558 flags SSKGXPT_READPENDING socket no 9 IP 192.168.11.1 UDP 18852

From the trace file on node2:

SSKGXPT 0x5edf558 flags SSKGXPT_READPENDING socket no 9 IP 192.168.10.2 UDP 38967

Oracle Scripts