Writing a Governance Action Service¶

A governance action service is a specialized Open Connector Framework (OCF) connector that performs a short, atomic piece of governance work: it starts, does its job, records what happened, and completes. Like all connectors it has two parts - a connector provider that describes and creates it, and the connector implementation that does the work.

It is passed a governance action context as it is started. This provides access to the request type and associated request parameters (name-value pairs) used to invoke the governance action service, along with its action targets. The context also gives access to open metadata through the same clients used by every other Egeria connector.

The four kinds of governance action service¶

These describe the shape of the work, and are worth knowing because they determine which guards and action targets other people will expect your service to produce.

Verification¶

A Verification governance action service tests the properties of specific open metadata elements to ensure they are set up correctly and do not indicate a situation where governance activity is required. For example, it may check that a new asset has an owner assigned, is set up with governance zones and includes a connection and a schema.

The verification governance action services publishes guards to report on any errors it finds. It may also create incident reports to coordinate actions to correct any errors. The guards returned from the verification governance action service can be used to trigger other governance services as part of a governance action process.

Triage¶

Triage governance action services run triage rules to determine how to manage a situation that needs human intervention. This could be to initiate an external workflow, wait for manual decision or initiate a remediation request through either an external workflow or by creating a ToDo for a specific person.

Remediation¶

The remediation governance action service performs updates to metadata. Examples of the tasks performed by a remediation governance action service are assigning classifications, duplicate linking and consolidation.

Provisioning¶

A provisioning governance action service invokes a provisioning service whenever a provisioning request is made. Typically, the provisioning service is an external service or pipeline. The provisioning governance action service may also create lineage metadata to describe the provisioning if this is not performed by another service.

Dependencies¶

• Open Connector Framework (OCF) - basic connector interfaces.
• Open Governance Framework (OGF) - the governance action service base class, the governance action context, guards and action targets.
• Open Metadata Framework (OMF) - the open metadata clients, properties, types and completion status.
• Audit Log Framework (ALF) - for audit logging and exceptions.
• Any digital resource connectors the service needs to call the technology it is governing.

dependencies {
    compileOnly 'org.odpi.egeria:audit-log-framework'
    compileOnly 'org.odpi.egeria:open-connector-framework'
    compileOnly 'org.odpi.egeria:open-metadata-framework'
    compileOnly 'org.odpi.egeria:open-governance-framework'
}

    <dependencies>

        <dependency>
            <groupId>org.odpi.egeria</groupId>
            <artifactId>audit-log-framework</artifactId>
            <scope>provided</scope>
        </dependency>

        <dependency>
            <groupId>org.odpi.egeria</groupId>
            <artifactId>open-connector-framework</artifactId>
            <scope>provided</scope>
        </dependency>

        <dependency>
            <groupId>org.odpi.egeria</groupId>
            <artifactId>open-metadata-framework</artifactId>
            <scope>provided</scope>
        </dependency>

        <dependency>
            <groupId>org.odpi.egeria</groupId>
            <artifactId>open-governance-framework</artifactId>
            <scope>provided</scope>
        </dependency>

    </dependencies>

Writing the connector provider¶

The connector provider for your governance action service is the factory method that creates an instance of it. It also describes the service's specification: the governance request types and request parameters it understands, the action targets it expects and produces, the guards it returns, and the configuration properties that adapt it. This is what lets a tool configuring engine actions or governance action processes offer your service correctly and validate how it is being called.

Each connector provider extends GovernanceActionServiceProviderBase .

The descriptive information - connector type GUID, qualified name, display name, description, wiki page, audit log component identifier and development status - is supplied through an implementation of the OpenConnectorDefinition interface. Defining these as an enum keeps the descriptions of all the connectors in your library together; Egeria's own connectors use the EgeriaOpenConnectorDefinition enum.

public class ProvisionUnityCatalogGovernanceActionProvider extends GovernanceActionServiceProviderBase
{
    private static final String connectorClassName = ProvisionUnityCatalogGovernanceActionConnector.class.getName();

    public ProvisionUnityCatalogGovernanceActionProvider()
    {
        super(EgeriaOpenConnectorDefinition.PROVISION_UNITY_CATALOG_GOVERNANCE_ACTION_SERVICE,
              connectorClassName,
              null);

        super.supportedRequestParameters  = ProvisionUnityCatalogRequestParameter.getRequestParameterTypes();
        super.producedGuards              = ProvisionUnityCatalogGuard.getGuardTypes();
        super.producedActionTargetTypes   = Collections.singletonList(ActionTarget.NEW_ASSET.getActionTargetType());
        super.supportedTechnologyTypes    = SupportedTechnologyType.getSupportedTechnologyTypes(
                new DeployedImplementationTypeDefinition[]{
                        UnityCatalogDeployedImplementationType.OSS_UC_CATALOG,
                        UnityCatalogDeployedImplementationType.OSS_UC_SCHEMA,
                        UnityCatalogDeployedImplementationType.OSS_UC_VOLUME,
                        UnityCatalogDeployedImplementationType.OSS_UC_TABLE,
                        UnityCatalogDeployedImplementationType.OSS_UC_FUNCTION});
    }
}

Defining request parameters, action targets and guards¶

Define the names your service uses as enums, next to the connector, and have them return the typed specification objects. This keeps the name, its description and its data type in one place, and means the connector and its provider cannot disagree about them.

The OGF supplies enums of common definitions so that services that do similar things use the same names - RequestType, RequestParameter, ActionTarget and Guard . Reuse them where they fit; a process that expects newAsset as an action target works with any service that produces it.

Each Guard value carries the completion status that normally accompanies it, so the two cannot drift apart:

    SERVICE_COMPLETED("service-completed", CompletionStatus.ACTIONED,  "The service completed successfully."),
    SERVICE_FAILED("service-failed", CompletionStatus.FAILED, "An unexpected error occurred while the governance service was running."),
    NO_TARGETS_DETECTED("no-targets-detected", CompletionStatus.INVALID, "There is no supplied action target and so the governance service does not know which asset to work on."),

Writing the governance action service connector¶

A governance action service extends GeneralGovernanceActionService . The base class sets up:

• governanceContext - the governance action context.
• governanceServiceName - the qualified name of the service, for audit log messages and exceptions.
• auditLog - the audit logging destination.
• connectionBean - the connection used to create the service, including its configuration properties.
• propertyHelper - utility for reading values out of open metadata elements.

It also needs to implement the standard Open Connector Framework (OCF) methods:

• initialize - an optional method where the connection object can be processed to, say, extract the configuration properties.
• start - this is where the processing logic of your governance action service belongs. Call super.start() first; it validates that the context has been set up.
• disconnect - an optional method to free up any runtime resources that the governance action service is using. It is called after the governance action service records a completion status, or if it throws an exception from the start method.

Here is the complete implementation of the Create Asset Governance Action Service , which creates an asset from a template and passes its GUID on as an action target:

public class CreateAssetGovernanceActionConnector extends GeneralGovernanceActionService
{
    @Override
    public void start() throws ConnectorCheckedException, UserNotAuthorizedException
    {
        final String methodName = "start";

        super.start();

        try
        {
            List<String>              outputGuards        = new ArrayList<>();
            List<NewActionTarget>     outputActionTargets = new ArrayList<>();
            CompletionStatus          completionStatus;
            AuditLogMessageDefinition messageDefinition;

            String templateGUID = getProperty(ManageAssetRequestParameter.TEMPLATE_GUID.getName(), null);

            if (templateGUID == null)
            {
                messageDefinition = GovernanceActionConnectorsAuditCode.NO_TEMPLATE_GUID.getMessageDefinition(governanceServiceName);
                outputGuards.add(ManageAssetGuard.MISSING_TEMPLATE.getName());
                completionStatus = ManageAssetGuard.MISSING_TEMPLATE.getCompletionStatus();
            }
            else
            {
                String assetGUID = governanceContext.getOpenMetadataStore().getMetadataElementFromTemplate(...);

                NewActionTarget newActionTarget = new NewActionTarget();

                newActionTarget.setActionTargetGUID(assetGUID);
                newActionTarget.setActionTargetName(ActionTarget.NEW_ASSET.name);

                outputActionTargets.add(newActionTarget);

                messageDefinition = GovernanceActionConnectorsAuditCode.NEW_ASSET_CREATED.getMessageDefinition(...);
                completionStatus  = ManageAssetGuard.SET_UP_COMPLETE.getCompletionStatus();
                outputGuards.add(ManageAssetGuard.SET_UP_COMPLETE.getName());
            }

            auditLog.logMessage(methodName, messageDefinition);

            governanceContext.recordCompletionStatus(completionStatus, outputGuards, null, outputActionTargets, messageDefinition);
        }
        catch (Exception error)
        {
            throw new ConnectorCheckedException(GovernanceActionConnectorsErrorCode.UNEXPECTED_EXCEPTION.getMessageDefinition(governanceServiceName,
                                                                                                                             error.getClass().getName(),
                                                                                                                             error.getMessage()),
                                                error.getClass().getName(),
                                                methodName,
                                                error);
        }
    }
}

Notice the shape: work out what to do, do it, choose a guard and its matching completion status, log a message, and record the outcome. Both the success and failure paths end at the same recordCompletionStatus() call.

Helper methods on the base class¶

• getActionTarget(name) and getAllActionTargets(name) pick the action target(s) with a given name out of the list, rather than iterating it yourself.
• getProperty(name, defaultValue) looks for a value first in the request parameters, then in the connection's configuration properties, then falls back to the default. This is the standard precedence: what the caller asked for beats how the service was configured.
• getStringRequestParameter(), getBooleanRequestParameter(), getIntRequestParameter(), getLongRequestParameter() and getArrayRequestParameter() read typed values out of the request parameters.
• handleUnexpectedException(methodName, error) wraps an unexpected exception in a standard ConnectorCheckedException.

The governance action context¶

GovernanceActionContext extends ConnectorContextBase - the same base class used by the integration context and the survey context. A governance action service therefore has the full set of open metadata clients available to it, not a restricted subset chosen by its base class.

Understanding the request¶

Performing the action¶

• getOpenMetadataStore() returns the generic store that retrieves, creates, updates, classifies, links and deletes metadata elements of any type. The element, classification and relationship maintenance methods that used to be reached only through a remediation base class all live here.
• The typed clients - getAssetClient(), getGovernanceDefinitionClient(), getSchemaTypeClient(), getGlossaryTermClient() and the rest - work with a specific category of element and are usually easier to read.
• getConnectorForAsset(assetGUID) creates a digital resource connector from an asset's connection, so the service can call the technology it is governing without knowing its credentials.
• createIncidentReport() creates an incident report that can provide a focal point for collaboration to resolve a particular issue. Incident reports are often managed by an incident management tool.
• openToDo() creates a To Do assigned to a person or role - the triage pattern.
• createNoteLogEntry() and registerContextEvent() record what happened for the people who come along later.
• getNotificationManagerClient() manages notification subscriptions.

For provisioning services that need to record lineage: createAsset(), createAssetFromTemplate(), createProcess(), createProcessFromTemplate(), createChildProcess(), createPort() and createLineageRelationship().

Initiating more governance¶

• initiateEngineAction() runs a new engine action.
• initiateGovernanceActionType() runs a single governance action type.
• initiateGovernanceActionProcess() runs a whole governance action process.

The context also still carries registerListener()/disconnectListener() for watching open metadata events. For anything that needs to keep watching, build a watchdog action service instead - a governance action service is expected to complete, and an engine host will treat one that never finishes as stuck.

Recording the outcome¶

• updateActionTargetStatus() records that the service has finished with a specific action target. This is optional, but useful to show progress when a governance action process is working through a long list of action targets.

• recordCompletionStatus(status, outputGuards, requestParameters, newActionTargets, completionMessage) declares that the service has completed processing:

  • status - one of ACTIONED (successfully completed), INVALID (the requested action was not appropriate - for example, a false positive), FAILED (the service failed to execute the requested action) or OTHER.
  • outputGuards - the guards used to determine which governance action runs next.
  • requestParameters - properties to pass to whatever runs next.
  • newActionTargets - the elements the follow-on services should process.
  • completionMessage - a message describing the result, or the reason for the failure.

• getCompletionStatus() returns the status passed on recordCompletionStatus(), or null if none has been recorded. It is used to coordinate the shutdown of a service that operates more than one thread.

Deploying and running your governance action service¶

Package your connector provider and implementation in a jar file, and add it (with any third party client libraries it needs) to the OMAG Server Platform class path - the easiest way is to copy the jar files into the extra directory of the platform's assembly.

A collection of related governance action services are then grouped into governance action engines for deployment. The governance action engine maps a governance request type to the governance action service that should be invoked, along with the request parameters to use.

These definitions are created as part of a governance engine pack or through the Asset Maker API and are stored in the open metadata repositories.

Governance action engines are hosted by the Governance Action OMES running in an engine host. You can find instructions for configuring the engine services in the engine host in the administration guide.

Example implementations¶

The governance-action-connectors module holds Egeria's own governance action services, grouped by the kind of work they do:

• verification - Verify Asset checks that an asset is set up correctly.
• remediation - Origin Seeker, Zone Publisher, Retention Classifier and the duplicate linkers update open metadata.
• provisioning - Move, Copy or Delete File provisions a resource and records the lineage.
• stewardship - Create Asset, Delete Asset, Catalog Target Asset, Wait for Steward and Days of Week are small building blocks for governance action processes.

The Coco Pharmaceuticals clinical trial samples show a set of governance action services working together in a realistic multi-step process.

Raise an issue or comment below