Interconnect & Cluster Software

Install the host adapters into the cluster nodes. For the procedure on installing host adapters, see the documentation that was shipped with the host adapters and node hardware. Connect the nodes with the physical interconnect and configure at least one IP address for the public network and one for the private interconnect. The networking parameter setting is important for RAC communications. The following networking parameters should be set up:

Parameter                       Description

/proc/sys/net/core/rmem_default The default setting in bytes of the socket

                                  receive buffer

/proc/sys/net/core/rmem_max     The maximum socket receive buffer size in

                                  bytes

/proc/sys/net/core/wmem_default The default setting in bytes of the socket

                                  send buffer

/proc/sys/net/core/wmem_max     The maximum socket send buffer size in

                                  bytes

The cluster software or cluster manager, also called oracm, required to run Real Application Clusters is included in the Oracle distribution.

The watchdog daemon is an Oracle-supplied module which monitors the oracm and pings to softdog through the watchdog device, /dev/watchdog, at defined intervals. It also monitors each oracm thread by receiving ping messages from them that have been registered with watchdogd.

Watchdog daemon detects the following cases:

• CM thread hang/death.

• User mode delay (scheduler / VM problem).

• Kernel hang.

In certain cases, systems with high loads experienced unnecessary reboots prior to 9.2.0.2 release. However, with the 9.2.0.2 release, functionality has changed and the watchdog is detached from the cluster manager.

In place of the watchdog daemon (watchdogd), the 9.2.0.2 version of the oracm for Linux now includes the use of a Linux kernel module called hangcheck-timer. The hangcheck-timer module monitors the Linux kernel for long operating system hangs and reboots the node if this occurs, thereby protecting the database from potential corruption. This is the new I/O fencing mechanism for RAC on Linux.

This approach offers three advantages over the watchdog implementation:

• Node resets are triggered from within the Linux kernel making them much less affected by the system load.

• The oracm service on a node can easily be stopped and reconfigured as its operation is completely independent of the kernel module.

• The features provided by the hangcheck-timer module closely follow the features found in the implementation of the cluster manager for RAC on the Windows platform.

The removal of the watchdogd means that the following parameters included in the cmcfg.ora file are no longer valid:

• WatchdogTimerMargin

• WatchdogSafetyMargin

To install the cluster manager software, use the runInstaller command from Oracle distribution.

1. Check Oracle Cluster Manager. Click Next.

2. On the public node information screen, enter the public node names and click Next.

3. On the private node information screen, enter the interconnect node names. Click Next.

4. Accept the default value (60000) for the watchdog parameter. Click Next.

5. Enter the full name of the raw device created for the node monitor for the Quorum disk information. Click Next.

6. Press Install at the summary screen.

A progress window will briefly appear, followed by the end of the installation screen. Click Exit and confirm by clicking Yes.

Note: Create the directory $ORACLE_HOME/oracm/log (as oracle) on the other nodes if it doesn't exist.

Start the cluster manager on all nodes as root:

$export ORACLE_HOME=/oracle/product/9.2.0

$ ./$ORACLE_HOME/oracm/bin/ocmstart.sh

Before starting the cluster manager, the configuration file ($ORACLE_HOME/oracm/admin/cmcfg.ora) may be edited.

The following sample parameters are for cmcfg.ora, for the RedHat Advanced Server 2.1 installation:

ClusterName=Oracle Cluster Manager, version 9i

KernelModuleName=hangcheck-timer

HeartBeat=15000

PollInterval=1000

MissCount=215

PrivateNodeNames=int-opcbr1 int-opcbr2

PublicNodeNames=opcbr1 opcbr2

ServicePort=9998

CmDiskFile=/u03/RAC/quorum.dbf

HostName=int-opcbr1

For more details, see ‘RAC Linux 9.2.0.2: configuration of cmcfg.ora and ocmargs.ora’ [Oracle Metalink Note: 222746.1].