Open Watchdog Framework (OWF)¶

The Open Watchdog Framework (OWF) defines the interfaces and base implementations for watchdog action services - specialized connectors that monitor the open metadata ecosystem for a specific situation or event and, when it occurs, notify the people, teams or processes that need to know.

OWF replaces the watchdog governance action service that used to be one of the five types of governance action service defined by the Open Governance Framework (OGF). The move is about setting clear expectations, not just tidying up the framework list. OGF's remaining four types of governance action service - verification, triage, remediation and provisioning - are all designed to run as short, atomic actions: they start, do their work, and complete. Watching for something to happen is fundamentally different in shape - a watchdog action service is expected to keep running for as long as the situation needs watching, which may be indefinitely. Mixing a long-running listener in among services that are otherwise expected to finish quickly made it hard to tell, just from a governance service's type, what kind of lifecycle to expect from it. Giving watchdog action services their own framework - and their own engine service - removes that ambiguity: a governance action service (OGF) is short-running, a watchdog action service (OWF) is long-running.

Having its own framework also lets watchdog action services do things that were awkward to fit into a short-running connector:

• Be described formally as a notification type, rather than only as configuration properties buried in a connector.
• Manage a durable, changing list of subscribers, rather than a fixed destination process.
• Run independently for as long as the situation needs watching, rather than as one step in a larger governance action process (although a watchdog action service can still trigger a governance action process when it detects something).

Two ways to watch for something¶

A watchdog action service can be set up in either of two ways, matching the two action target types it recognizes.

Watching an instance or type directly¶

The simplest form of monitoring points a watchdog action service at either a single metadata instance (element or relationship), or every instance of a particular open metadata type, and asks it to run a named governance action process whenever a particular kind of event occurs - a new element appearing, a property changing, a classification being added or removed, an element being deleted, and so on. This is the pattern that the watchdog governance action service used before that capability moved into OWF:

The watchdog governance action service monitors changes in the metadata and initiates one of the following as a result:

• engine action
• governance action process
• incident report

One example of a watchdog governance action service is to monitor for new assets. Another example is to monitor the addition of open discovery reports and take action on their content.

The Generic Element Watchdog and Generic Folder Watchdog governance action services are concrete, ready-to-configure examples of this pattern - the second is simply the first, pre-configured to watch a folder's DataFile assets. Both accept a governance action process qualified name for each kind of event they should react to (see the WatchdogRequestParameter values below), and can either watch for a single occurrence and stop, or keep watching indefinitely. They still run as OGF governance action services today, making them exactly the kind of long-running-in-a-short-running-framework case the OWF split is intended to resolve; new instance/type watching should be built directly against OWF's WatchdogActionServiceConnector, in the same way the Baudot Subscription Manager already does for notification-type watching.

Watching through a notification type¶

The richer form of monitoring is built around a NotificationType, an open metadata governance definition that formally describes:

• The resource, or resources, to monitor - linked via the MonitoredResource relationship.
• The subscribers that should be told when the situation occurs - linked via the NotificationSubscriber relationship. A subscriber can hold more than one subscription to the same notification type at once, each independently tracked through its own activity status.
• How often notifications should be sent - a single occurrence, a repeating interval (notificationInterval), or throttled so subscribers aren't overwhelmed (minimumNotificationInterval).

When the watchdog action service detects the situation the notification type describes, it works through the current subscribers and, for each one, creates a Notification - an action assigned to that subscriber describing what has happened. A subscriber's relationship to the notification type is updated as this happens, so it is always possible to see who has been told, and who hasn't.

This pattern is deliberately more stateful than instance/type watching: a notification type persists as metadata in its own right, subscribers can be added and removed from it at any time (by people or by other governance processes), and the watchdog action service that services it is really managing an ongoing subscription list rather than reacting to a one-off event.

The Baudot Subscription Manager is a production example of this pattern - it manages subscriptions to the digital products in the Open Metadata Digital Product Catalog. It is passed the notification types to service as action targets, is designed to run indefinitely rather than complete, and uses each notification type's multipleNotificationsPermitted and notificationInterval properties to decide, per notification type, whether to send a single one-time notification, notify subscribers whenever a monitored product changes, or notify them on a fixed schedule. It also watches the NotificationSubscriber relationship itself, so subscribers are welcomed and dismissed automatically as they subscribe and unsubscribe, and the ActionTarget relationship, so a new notification type can be added to a running instance without needing to restart it.

Watchdog Action Service¶

A watchdog action service is implemented as a specialized connector that is passed a watchdog context as part of its initialization. Its job, once started, is to register interest in the events it cares about and then process each one as it arrives - continuing to do so until it declares itself complete, or is stopped by an administrator.

The types of events a watchdog action service receives are controlled by up to three filters, combined together: the types of event (new element, updated element, deleted relationship, and so on), the open metadata types of the elements/relationships involved, and, optionally, a single specific instance to watch. Only one set of filters can be active at a time, but a watchdog action service can change them while it is running - for example, switching what it is watching for part way through, or after it finishes with one notification type and moves on to the next.

Watchdog Context¶

The watchdog context gives the watchdog action service access to information about the request, along with the open metadata repository interfaces it needs to do its work.

Like the request parameters and action targets used by other kinds of governance service, the context provides the request type, request parameters, action target elements and requesting user for this run of the service.

For notification-type-based monitoring, the context provides:

For instance/type monitoring, the context provides a lower-level registerListener()/disconnectListener() pair that lets the watchdog action service supply its own listener along with the combination of event types, metadata types and (optionally) a specific instance it wants to hear about.

In both cases, the watchdog action service finishes by calling recordCompletionStatus(), in the same pattern used across Egeria's governance services - supplying a completion status, any output guards, and optionally new request parameters or action targets to pass on to whatever runs next.

Reference data¶

Every watchdog action service automatically supports the following action targets, request parameters and guards, defined by the OWF so that watchdog action services stay consistent with one another. A specific watchdog action service may support additional ones of its own.

Action targets¶

Request parameters¶

These are used with instance/type monitoring, to describe what to watch for and what to do about it.

A process name that is not set means that kind of event is simply ignored.

Guards¶

Where watchdog action services run¶

Watchdog action services run in a watchdog action engine, hosted by the Watchdog Action OMES running in an Engine Host - the same overall pattern used by survey action services and governance action services. The Watchdog Action OMES provides a REST API to request that a watchdog action engine starts a watchdog action service and to check on its results; the engine itself calls back into a Metadata Access Server to read about the resources being monitored and to record the notifications it sends.

Implementation in Egeria¶

OWF is implemented in the open-watchdog-framework module. The base connector class is WatchdogActionServiceConnector; it is passed a WatchdogContext and implements a single abstract method, processEvent(), that is called once for each matching event.

The open metadata types that support notification-based monitoring - NotificationType, MonitoredResource and NotificationSubscriber - are described in 0451 Notifications. The Notification entity that is created for each subscriber is described in 0135 Actions for People.

Raise an issue or comment below