Proposed Alarm schema

Earthworm5 Alarms

Programmer's Guide

Lucky Vidmar

May 7, 2003

Introduction

The Earthworm Alarms System is intended to satisfy the need for an easy-to-use, web-centered method for dispatching notifications about automatic and reviewed earthquakes. The initial requirements are that the system must support several delivery mechanisms such as email, pager, and QDDS. Furthermore, the user must be able to specify and modify the text of the notification messages. The system must keep an audit of all notifications that were sent, and this audit should be used to ensure that subsequent alarms are sent to the same list of recipients who received the original notification.

This document deals only with the Earthworm Alarms Manager portion with its source code residing under oracle/apps/src/alarms/web. Refer to the Earthworm Quick Review documentation for more on how reviewed notifications are handled and issued.

System Overview

At the center of the system is the Alarms Schema - a set of Oracle DB tables, views, and stored procedures, along with the requisite Earthworm API routines written in C. Please see the attached ERD and the explanation of it in this document.

The schema is accessed from two levels:

Alarms Manager -- sequence of CGI programs that support the web pages allowing the user to create or modify alarm rules, delivery mechanisms, formats, etc.
Alarms Processor - Earthworm library routines that access the rules from the schema, process them, and issue alarms.

Alarms Manager

The Earthworm Alarms Manager is comprised of a series of CGI programs written in C. They support a web interface allowing the "manager" to create or modify recipients, rules, delivery mechanisms, and formats. See System Overview Diagram .

alarms_manager - This is the entry point into the Alarms Manager. It retrieves from the DB the available lists of recipients, criteria programs, and delivery formats, and then puts up three HTML sections allowing the manager to select an existing entry from a list, or chose to insert a new one. Clicking on Submit for each of the three sections (recipients, programs, formats) invokes the appropriate processing program (alarms_process_recipient, alarms_process_prog, alarms_process_format, explained below).

alarms_process_recipient - Invoked by alarms_manager, works in two modes depending on whether it is processing a new or an existing recipient. If it is a new recipient, it puts up an HTML form allowing the manager to define the initial rules for this recipient, specifying magnitude, reviewed status, delivery method and format, as well as the criteria program desired, if any. If an existing recipient is being processed, it retrieves from the DB and then displays the known information about that recipient, and then puts up the HTML form allowing the manager to either modify previously defined rules, or define new ones. The manager can also delete one or more existing rules, or delete the recipient altogether. Clicking Submit or Delete will invoke the alarms_process_new_recipient program.

alarms_process_new_recipient - Works in two modes, depending on whether it was invoked by clicking Delete or Submit from alarms_process_recipient. If in Delete mode, it deletes the recipient and all rules and deliveries associated with the recipient from the database. Otherwise, its task is to collect all available information about a recipient and prepare for insertion into the database. It is called from alarms_process_recipient which passes it all available information about the recipient. It then writes the information into a binary structure (see configuration file for location of that file) which contains all that information so that alarms_insert_recipient can use it. After that, it puts up an HTML form requesting the manager to fill in all the information which is needed to complete the request (new email address, name of the new criteria program, etc.). Clicking Submit invokes alarms_insert_recipient program.

alarms_insert_recipient - First it reads the binary file created by alarms_process_new_recipient which contains all of the information about this recipient known at that point. Then, it processes the web parameters passed in from alarms_process_new_recipient and reconciles them with the structure read from the file. If all goes well up to here, it proceeds to insert all of the information in the structure into the database.

alarms_process_prog - Invoked by alarms_manager, works in two modes depending on whether it is processing a new or an existing criteria program. See below for more information on criteria programs. If it is a new program, it puts up an HTML form allowing the manager to fill in the information about that program. Otherwise, it retrieves from the DB and displays previously defined information allowing the manager to review or modify it. Clicking Submit or Delete calls alarms_insert_prog program.

alarms_insert_prog - Called by alarms_process_prog, works in two ways depending on whether it was invoked by clicking Submit or Delete. If it is in Delete mode, and the criteria program is not used in any rule, it deletes the program from the database. Otherwise, it processes the web parameters passed to it, and calls Earthworm DB API routines to insert the information into the database.

alarms_process_format - Invoked by alarms_manager, works in three modes depending on whether it is processing a new or an existing delivery format, or if a preview of a format is desired. In Preview mode, it displays the existing format showing all valid format tokens in a different color. See below for more information on formats. If it is a new format, it puts up an HTML form allowing the manager to fill in the information about that format. Otherwise, it retrieves from the DB and displays previously defined information allowing the manager to review or modify it. Clicking Submit or Delete calls alarms_insert_format program.

alarms_insert_format - Called by alarms_process_format, works in two ways depending on whether it was invoked by clicking Submit or Delete. If it is in Delete mode, and the format is not used in any rule, it deletes the format from the database. Otherwise, it processes the web parameters passed to it, and calls Earthworm DB API routines to insert the information into the database.

Alarms Processor

The basic flow of the Alarms Processor is to retrieve all the rules from the database, determine which ones meet the event?s criteria, and then build and execute the corresponding alarms. Optionally, we would like to allow the user to review and modify the alarm list before it is executed.

DetermineAlarms() - Retrieves the list of all rules from the database. For each rule which passes the magnitude and review flag criteria, the rule?s criteria program, if defined, is executed. See below for more information on criteria programs. If the criteria program returns successfully, a new alarm is inserted into the list of alarms to be executed. After all rules are processed, the list of alarms is sorted by priority and duplicates are eliminated.

ReviewAlarms() - Given a list of alarms, it puts up an HTML form allowing the manager to review the alarms. It returns another alarm list which can be passed to ExecuteAlarms().

ExecuteAlarms() - For each alarm from the alarm list, produces an Earthworm message of the appropriate type (e.g., TYPE_EMAIL_MSG) and writes it to the Alarms ring. For each alarm issued, it writes an entry into the AlarmsAudit table in the database, noting the time of issue. Note that the Phone delivery mechanism alarms are ignored by ExecuteAlarms. Currently, the Phone list alarms are processed within the Earthworm Quick Review system after ExecuteAlarms() is done, by a CGI program which creates a web form listing all phone numbers which the reviewer should dial.

Glossary Note - Alarms Formats

The alarms manager allows flexibility in defining and modifying the formats in which alarms messages are delivered. Using the format manager, each format is assigned two message templates: insertion and deletion. The format manager uses a token-based template to allow easy definition of new formats. Each token is bracketed by the ~ character, and each format definition must end with the ~End~ token. The system then replaces each token with the particular value associated with the event which caused the event to be issued:

The following tokens are currently recognized:

~EventID~ : Database ID assigned to the event.
~Otime@TIMEZONE~ : Origin time for the event.TIMEZONE must specify a valid value for the TIMEZONE environmental variable corresponding to the timezone for the desired display. If the @TIMEZONE is omitted, the time will be displayed in the timezone of the webserver. Example: ~Otime@US/Mountain~ will display the origin time in Mountain Time.
~Towns@<TOTAL>,<BIG>~: List of towns nearest to the event. <TOTAL> is the total number of towns to list, <BIG> is the number of big or significant towns. The population limit for big towns is specified by the administrator in the Review configuration file. Example: ~Towns@5,2~ will display five nearest towns, two of which will be "big" with population above the limit set in the configuration file.
~PrefMag~ : Preferred Magnitude.
~Md~ : Duration Magnitude.
~ML~ : Local Magnitude.
~Lat~ : Latitude in degrees with the format of +/- xx.xxxx.
~Lon~ : Longitude in degrees with the format of +/- xxx.xxxx.
~Depth~ : Depth (km).
~Nph~ : Number of phases used in the solution.
~Dmin~ : Distance to nearest station (km).
~Rms~ : rms misfit (seconds).
~ErrLat~ : Lattitudinal location error (km).
~ErrLon~ : Longitudinal location error (km).
~ErrZ~ : Vertical location error (km).
~Gap~ : Azimuthal gap (degrees).
~RevAuto~ : If the solution is automatic this token is replaced with the string Automatic, otherwise the string Reviewed is used.
~InvokedBy~ : This token is replaced with a string supplied by the program which invoked the execution of alarms. This is useful for distinguishing alarms raised when various parameters of an event are inserted into the database.

The insertion and deletion messages for the CUBE format are predefined in the system. They can modified in the source code for determine_alarms.

Glossary Note - Criteria Programs

It was realized early on that a relational database is not the best platform for storing every imaginable type of alarms rule. However, storing rules in the database is one of the specifications. So, to make life easier for the programmer, we use the notion of a criteria program (CP) which is one attribute of each rule. The basic idea is that a CP is a program written in any language, and invoked by the DetermineAlarms() function. It is passed information about the event and it signals back to the caller whether the event passes some set of criteria defined and checked by the program.

Each CP has three essential elements:

ProgramName - Full path to the location of the program executable.
ProgramDir - Temporary directory where this program has write permission. This is necessary for simple communication between the calling program and CP. ProgramDesc- human readable description (brief) of what this CP checks for (e.g., Utah Polygon, Menlo Automatics 2.5).

Criteria programs are stand-alone programs. This causes some problems with respect to communication with DetermineAlarms(). For now, we take a simple approach: call each CP using the system() command. Unfortunately, this is not as easy as it sounds. To accomplish this, DetermineAlarms() retrieves information about each rule and the associated CP (if any, note that a rule can exist without a CP). It then prepares to execute the CP by writing into the ProgramDir a binary file containing the EWEvent structure (structure which holds information about the event). Each CP is executed with two parameters: name of the working directory and name of the event file. Therefore, each CP, irrespective of the language, must be written so that it takes two character strings as parameters: name of the working directory and name of the event file.

After a CP is executed, DetermineAlarms() needs to figure out whether the CP failed or succeeded. In our simple scheme this is accomplished by reading back the EWEvent structure from the file. If the idEvent value is set to the negative of the original idEvent then the event passed the criteria check. Otherwise, it did not. Therefore, each CP, irrespective of the language, must write the EWEvent structure back to the same file from which it was read, and set the idEvent to the negative of the original if the criteria checks are successful.

Finally, after the EWEvent structure is read back by DetermineAlarms(), it is removed from the ProgramDir temporary directory.

Alarms Schema Overview

See the attached ERD for a graphical representation of the schema. Following are descriptions of each table and their entries.

AlarmsRecipient - keeps information about a recipient, such as the name, address, etc.

idRecipient - unique database identifier.
dPriority - priority designation for this recipient
sDescription - description of the recipient.

AlarmsFormat - delivery format such as CUBE, human readable format, pager formats etc. For each format, two descriptions must be defined: one defining the format of the message when an event is being inserted, and another with the text of deletion message.

idFormat - unique database identifier.
sFormatDescription - brief description or label for the format.
sFmtInsert - template for the insertion format
sFmtDelete - template for the deletion format.

CriteriaProgram

idCritProgram - unique database identifier.
sProgName - full path to the executable for the program.
sProgDir - full path to the temporary directory where event structure files are written.
sProgDescription - brief description or label for this program.

RecipientDelivery - bind table mapping the recipients and associated delivery mechanisms, such as email, pager, etc.

idRecipientDelivery - unique database identifier.
idRecipient - foreign key into the AlarmsRecipient table identifying the recipient.
tiDelivery - string indicating delivery mechanism (email, pager).
idDelivery - foreign key into one of the delivery mechanism tables.

EmailDelivery - email delivery mechanism information.

idDelivery - unique database identifier.
sAddress - full address.
sMailServer - host to be used as email server (optional).

PagerDelivery - pager delivery mechanism information.

idDelivery - unique database identifier.
sPagerNumber - pager number.
sPagerCompany - pager company and other information needed to send a page.

PhoneDelivery - phone delivery mechanism information.

idDelivery - unique database identifier.
sPhoneNumber - telephone number.

QddsDelivery - QDDS delivery mechanism information.

idDelivery - unique database identifier.
sQddsDirectory - QDDS target directory.

CustomDelivery - Custom delivery mechanism information.

idDelivery - unique database identifier.
sDescription - label (name) of this custom delivery.

AlarmsRule - central table holding the information about each rule.

idRule - unique database identifier.
dMag - minimum magnitude.
bIsAuto - reviewed or automatic event.
idFormat - foreign key into the AlarmsFormat table indicating the format to be used for sending alarms triggered by this rule.
idCritProgram - foreign key into the CriteriaProgram table indicating the program to run to check further criteria.
idRecipientDelivery - foreign key into the RecipientDelivery table indicating the recipient for whom this rule is defined, as well as the delivery format to be used for sending alarms trigger by this rule.

AlarmsAudit - keeps track of alarms that have been declared or executed. The database ids stored in this table are not strictly key-ed back to the tables in which they reside. This is necessary so that the Audit trail is independent from any changes made to the data in the referred tables (i.e., if a recipient?s email address changes, the audits for previously issued alarms should not be affected).

idAudit - unique database identifier.
idEvent -key into the Event table (part of Earthworm Core Schema) indicating the event for which this alarm was issued.
bIsAuto - boolean value indicating whether the alarm was issued for an automatic or reviewed solution.
idRecipient - key into the Recipient table indicating the recipient who was sent the alarm.
idFormat -key into the AlarmsFormat table indicating the delivery format in which the alarm was issued.
sTableName- string indicating delivery mechanism.
idDelivery - index into the appropriate delivery table (email, pager) - note that the Audit system has separate delivery tables (i.e., don?t use foreign keys into the Email or Pager tables described above). This is needed so that audits remain fixed over time, even if the delivery information is changed.
tAlarmDeclared - time when ExecuteAlarms() declared this alarms by writing a message into the Alarms Ring.
tAlarmExecuted - time when the alarm delivery program picked up the message and executed the appropriate alarm sender.

CubeVersionNumber - keeps the next version number for CUBE formatted messages.

idVersion - unique database identifier.
idEvent - identifier of the event.
iVersionNumber - next CUBE version number.

AuditEmailDelivery, AuditPagerDelivery, AuditPhoneDelivery, AuditQddsDeliver, AuditCustomDelivery - identical to non-audit delivery tables described above, but with unchangeable information used to provide static audit trail.

Alarms Delivery Overview

Once ExecuteAlarms() writes an alarm message into the ring, it is up to another Earthworm module to pick that message up and delivery the alarm accordingly. Therefore, we need to define a series of new message types to carry information about alarms, and then to write delivery modules to process those messages.

Message Types

Alarm message types will be defined as needed.

TYPE_EMAIL_MSG

TYPE_QDDS_MSG

TYPE_CUSTOM_ALARM_MSG

<label(description) of the custom mechanism >

TYPE_PAGER_MSG

*** Format not yet defined ***

Delivery Modules

email_sender - picks up TYPE_EMAIL_MSG messages from the ring, parses them and calls the SendMail() earthworm library routine to delivery the message. Then, it updates the appropriate entry in the AlarmsAudit table to indicate that the message has been delivered.

qdds_sender - picks up TYPE_QDDS_MSG messages from the ring, parses them and writes the text of the message to the specified QDDS target directory. For now, this program only deals with placing the files in CUBE format into the correct directory, leaving the bulk of the delivery work to an outside QDDS-capable process.

Custom_delivery - template of a program that picks up TYPE_CUSTOM_ALARM_MSG message from the ring and processes them according to customized local requirement. This delivery mechanism is intended to provide the unlimited flexibility for sites whose alarm requirements do not corresponds to the features of the Earthworm system.

Questions? Issues? Subscribe to the Earthworm List (earthw).