Handling Failure and Recovery Interrupts
The transmitter and receiver modules automatically handle a parent node’s failure and recovery. However, processor or queue modules that have processes executing in them do not have the built-in capability to handle failure and recovery interrupts. Three typical examples of processes that are designed to handle failure and recovery interrupts are described below.
Routing Topology Example
If a node is maintaining a routing topology that is dynamically updated as nodes’ and links’ conditions change, a typical method of updating the topology is to add a module to the node model that is responsible for that task. You should set the failure intrpts and recovery intrpts attributes of the module’s process model to “network wide”. When any node or link changes condition in the system, the root process inside the module:
- Receives a failure or recovery interrupt;
- Calls the Kernel Procedure
op_intrpt_type()to determine which type of interrupt occurred; - Updates the routing topology to reflect the change.
The following figure shows an example code fragment from a process model that updates a routing table based on object failures and recoveries throughout the network.
Figure 200: Example Proto-C Updating Topology Based on Failure/Recovery Events
/** This state is entered when a link connected to this node or**/
/** a directly connected router fails/recovers. In both cases **/
/** the link status is changed. **/
/** Link failure leads to setting the metric for the entries in**/
/** the RIP routing table for which the next hop was reachable **/
/** through the failed interface to infinite. **/
/** Link recovery inserts the route to the directly connected **/
/** neighbor on the recovered link into the routing table. **/
/** Both cases trigger sending of RIP route update messages. **/
if (intrpt_type == OPC_INTRPT_FAIL)
{
/** Link has failed. **/
/* Handle the failure of the connected link. */
status_changed = rip_link_fail (table_index);
}
else
{
/** Link has recovered. **/
/* Handle the link recovery. */
status_changed = rip_link_recover (table_index);
}
/* Routing table has changed. Schedule generation of*/
/* route update messages. */
•
•
•
Node Failure and Recovery Examples
A node failure is usually considered a catastrophic event for the modules within the node. Transmitters automatically abort any packets in transmission and flush their transmit queue. Receivers will not accept packets while the node is disabled. In addition, a process may require notification of the node’s failure so that it can leave itself in a clean state. For example, a process within a queue module may need to flush its buffers of packets or a process may destroy its child processes. If a module requires notification of the parent node’s failure, the module’s failure intrpt attribute can be set to “local only”. The process set up to receive the interrupt will be invoked just before the node actually fails, so that it may take any actions necessary to handle the node failure. Typically, a process calls the Kernel Procedure op_intrpt_type() to determine that the interrupt is a failure interrupt. The following figure shows an example code fragment from a process model that flushes its packet buffers just before its parent node fails.
Figure 201: Example Proto-C Handling Various Interrupt Types Including Failure Interrupts
/* This process just received an interrupt. Based on the type of */
/* interrupt, take the appropriate actions. */
/* Determine which interrupt type just occurred. */
intrpt_type = op_intrpt_type ();
if (intrpt_type == OPC_INTRPT_STREAM)
{
/* Received stream intrpt. Call procedure to handle the packet. */
packet_receive_and_enqueue ();
}
else if (intrpt_type == OPC_INTRPT_SELF)
{
/* Received self interrupt. Call procedure to send a packet. */
packet_dequeue_and_send ();
}
else if (intrpt_type == OPC_INTRPT_FAIL)
{
/* This process just received a failure interrupt for its */
/* parent node and must flush the subqueues and cancel self */
/* interrupts. */
/* Flush the subqueues. */
op_q_flush ();
/* Cancel all self interrupts. */
op_intrpt_clear_self ();
}
Similarly, when a node recovers, processes within that node often need to re-initialize themselves. For example, a process could reset its counters, zero out its arrays, and transition to the state where it expects incoming packets. If a process requires notification when only its parent node recovers, then its containing module’s recovery intrpts attribute is set to “local only”, so that when the node recovers, a recovery interrupt will be delivered to the module. The process calls the Kernel Procedure op_intrpt_type() to distinguish this interrupt type from the others, such as a stream or self interrupt. The following listing shows an example code fragment from a process model which re-initializes its state variables when its parent node recovers.
Figure 202: Example Proto-C Handling Various Interrupt Types Including Recovery Interrupts
/* This process just received an interrupt. Based on the type of */
/* interrupt, take the appropriate actions. */
/* Determine which interrupt type just occurred. */
intrpt_type = op_intrpt_type ();
if (intrpt_type == OPC_INTRPT_SELF)
{
/* Received self interrupt. Re-transmit packet. */
packet_retransmit ();
}
else if (intrpt_type == OPC_INTRPT_STREAM)
{
/* Received stream interrupt. Handle the packet. */
packet_receive ();
}
else if (intrpt_type == OPC_INTRPT_RECOVER)
{
/* This process just received a recovery interrupt for */
/* its parent node and must re-initialize its state variables. */
/* Reset counters to 0. */
num_packets = 0;
num_errors = 0;
/* Set current time. */
start_time = op_sim_time ();
/* Reset array values. */
for (i = 0; i = NUM_ENTRIES; i++)
entry_array [i] = NULL_ENTRY;
}