Validate RAC Networking

Validate RAC Networking
 a. Record IP’s and node names
  i. Run /sbin/ifconfig
  ii. Note private, public IP’s.
  iii. Example:
   1. node name : l6312
   2. Public IP : 10.118.49.25
   3. Private IP: 10.255.255.25
   4. node name : l6313
   5. Public IP : 10.118.49.26
   6. Private IP: 10.255.255.26
  iv. Run nslookup [scan name]
   1. Note IP’s returned
 b. Verify Multicast (11.2.0.2 RAC specific) on all nodes
  i. /bin/netstat –in
  ii. Look for: eth0 and eth1, MTU = 1500
  iii. /sbin/ifconfig
  iv. Look for: “MULTICAST MTU:1500”
 c. Test public IP’s on all nodes
  i. /bin/ping –s 1500 –c 2 –i [IP]
   1. Ping node1 => node1
   2. Ping node2 => node2
   3. Ping node1 => node2, etc
   4. Ping node2 => node1, etc
 d. Test private IP’s on all nodes
  i. /bin/ping –s 1500 –c 2 –I [IP]
   1. Ping node1 => node1
   2. Ping node2 => node2
   3. Ping node1 => node2, etc
   4. Ping node2 => node1, etc
 e. Test private IP’s traceroute
  i. /bin/traceroute –s [local private IP] –r –F [remote private IP] 1472
   1. Look for ONLY 1 hop to the remote private IP
 f. Test VIP’s
  i. /bin/ping –c 2 [VIP name]  from all to all
   1. Ping node1 => node1 vip
   2. Ping node2 => node2 vip
   3. Ping node1 => node2 vip, etc
   4. Ping node2 => node1 vip, etc
  ii. Note: These VIP’s could be on different nodes after clusterware is up
  iii. Look for: Successful pings
 g. Test DNS setup
  i. /usr/bin/nslookup
   1. nslookup from node1,node2,etc. => VIP name
   2. nslookup from node1,node2,etc. => SCAN name
 h. Verify name resolution order
  i. grep ^hosts /etc/nsswitch.conf
  ii. Look for : “files dns”
 i. Verify /etc/hosts (check for all cluster members)
  i. grep [node1 hostname] /etc/hosts
  ii. grep [node2 hostname] /etc/hosts
  iii. grep [node1 VIP name] /etc/hosts
  iv. grep [node2 VIP name] /etc/hosts
  v. grep [node1 IP] /etc/hosts
  vi. grep [node2 IP] /etc/hosts
  vii. grep [node1 VIP] /etc/hosts
  viii. grep [node2 VIP] /etc/hosts
 j. Verify scan is not in /etc/hosts
  i. grep [scan name] /etc/hosts
  ii. grep [SCAN IP] /etc/hosts (rerun for each SCAN IP)
2. Test CRS
 a. Restart nodes one at a time
  i. Verify that resources fail-over
  ii. Verify that resources actually restart on the restarted node
   1. crsctl status resource –t
    a. check for gds,vip,listener,db(s),services,etc.
  iii. Follow crs logs, look for unexpected errors