Writing a Governance Action Service¶

A governance action service is a specialized connector that performs monitoring of metadata changes, validation of metadata, triage of issues, assessment and/or remediation activities on request. Some governance action services invoke functions in external engines that are working with data and related assets.

A governance action service runs in the Governance Action Open Metadata Engine Service (OMES) hosted by the Engine Host OMAG Server.

Governance action services implement interfaces defined by the Governance Action Framework (GAF). The GAF offers embeddable functions and APIs to simplify the implementation of governance action services, and their integration into the broader digital landscape, whilst being resilient and with good performance.

It is possible to implement complex governance actions in a single governance action service. Alternatively there are five specialized types of governance action services that help you to break down your governance function into reusable components that can be choreographed by governance action processes to maximise the flexibility of your governance automation. When a governance action service completes, it produces guards that define what needs to be done next along with a list of action targets.

• Watchdog Governance Action Service listens for changes to metadata and initiates new [engine actions](/concepts/engine-action, governance action processes or an incident report.

• Verification Governance Action Service validates that a rule or policy is being followed. This is often a test that the metadata elements, relationships and classification are set up as they should be. For example, it may check that a new asset has an owner, is set up with governance zones and includes a connection and a schema there possible. Verification governance action services

• Triage Governance Action Service runs triage rules to determine how to manage a situation or request, such as a request for action from an open discovery service. Often this involves a human decision maker. It may initiate an external workflow, wait for manual decision or create a ToDo for a specific person.

• Remediation Governance Action Service makes updates to metadata elements, relationships between them and classifications. Examples of remediation governance action services include:

  • Classification and linking of metadata elements such as adding owners, governance zones and origin classifications to assets.
  • Duplicate detection, linking and consolidating.

• Provisioning Governance Action Service invokes a provisioning service whenever a provisioning request is made. Typically, the provisioning service is an external service. It may also create lineage metadata to describe the work of the provisioning service if the provisioning service is not able to create lineage itself.

The interfaces for governance action services is defined in the governance-action-framework module.

Governance action context¶

A governance action service is passed a context as it is started. This provides access to the request type and associated request parameters (name-value pairs) used to invoke the governance action service, along with its action targets. There is also a client to access open metadata through the Governance Server OMAS.

The methods of the context are colour-coded. The methods shown in white are available for all types of governance action services. These are divided into three groups:

• Understanding the request

  • getRequestType - returns the governance request type used to invoke the governance action service.
  • getRequestParameters - returns the name-value pairs of properties passed with the request type.
  • getRequestSourceElements - returns details of the process that invoked the governance action.
  • getActionTargetElements - returns the list of action target elements that this governance action service is to work on.

• Performing the action

  • getOpenMetadataStore - returns a client to the Governance Server OMAS that supports the retrieval of metadata elements, their classifications and relationships.
  • createIncidentReport - creates an incident report that can provide a focal point for collaboration to resolve a particular issue. Incident reports are often managed by an incident management tool.

• Recording the outcome of the processing

  • updateActionTargetStatus - records that the governance action service has completed processing a specific action target. This is optional, but useful to show progress when a governance action process is performing action on a long list of action targets.

  • recordCompletionStatus - declares that the governance action service has completed processing. It has three parameters:

    • completionStatus - with values:

      • Actioned - the governance action service has successfully completed processing.
      • Invalid - the governance action service has not performed the requested action because it is not appropriate (for example, a false positive).
      • Failed - The governance action service failed to execute the requested action.
      • Other - Undefined or unknown completion status.

    • outputGuards - the list of guards that are used to determine which governance action to run next.

    • newActionTargets - the list of action targets that the follow-on governance services should process.

  • getCompletionStatus - returns the status passed on recordCompletionStatus or null if no status has been posted. It is used to coordinate the shutdown of multiple threads operated by the governance action service.

The methods in blue are addition services offered to governance action services implementing a watchdog capability:

• registerListener - requests that it is called whenever certain open metadata events occur. It is able to specify a range of conditions from receiving all event, events for particular actions (such as creation of a new metadata element), events for particular metadata types or events on particular instances.
• initiateGovernanceAction - requests that a new engine action is run.
• initiateGovernanceActionProcess - requests that a new governance action process is run.

The methods in purple are additional services offered to governance action services implementing a provisioning capability to record lineage:

• createAsset - create an asset - typically the destination of the provisioning.
• createProcess - create a representation of the provisioning process.
• createPort - create a port for the process
• createLineageMapping - create a lineage relationship.

There are no additional services for governance action services implementing a verification capability which is why the green box is empty.

The methods in yellow are additional services offered to governance action services implementing a triage capability to initiate a stewardship action:

• createToDo - create a ToDo that is assigned to an individual.

The methods in red are additional services offered to governance action services implementing a remediation capability to make updates to open metadata. They are divided into four groups:

• Working with metadata elements

  • createMetadataElement
  • updateMetadataElement
  • updateMetadataElementStatus
  • deleteMetadataElement

• Working with classification on a metadata element

  • classifyMetadataElement
  • reclassifyMetadataElement
  • updateClassificationStatus
  • declassifyMetadataElement

• Working with relationships (links) between metadata elements

  • createRelatedElements
  • updateRelatedElements
  • updateRelatedElementsStatus
  • deleteRelatedElements

• Performing specialist governance actions

  • linkElementsAsPeerDuplicates
  • LinkConsolidatedDuplicate

A governance action service has a choice of base classes to extend that controls the methods provided in the governance action context:

• GeneralGovernanceActionService - for the complete context
• WatchdogGovernanceActionService - for the white and blue methods
• ProvisioningGovernanceActionService - for the white and purple methods
• VerificationGovernanceActionService - for the white and green methods
• TriageGovernanceActionService - for the white and yellow methods
• RemediationGovernanceActionService - for the white and red methods

The Writing the governance action service connector section provides more information on how to use the methods.

Writing the connector provider¶

The connector provider for your governance action service provides the factory method to create an instance of the governance action service. It also supports methods to describe the options that the governance action service provides, such as its supported governance request types, request parameters, action target types, configuration properties and output guards.

Below is the implementation of the connector provider for the Move Copy File Provisioning Governance Action Service. This is a highly configurable governance action service that can be instructed to move, copy or delete a file, and has different styles of lineage it can create. The action it performs is supplied in the governance request type. The source file and destination folder can be supplied either through the request parameters or as action targets. There are two guards: "provisioning-complete" for success and "provisioning-failed" if something went wrong.

The methods of the connector provider enables a tool that is configuring engine actions or governance action processes to query the capabilities of the corresponding governance action service.

public class MoveCopyFileGovernanceActionProvider extends GovernanceActionServiceProviderBase
{
    private static final String  connectorTypeGUID = "e2a14ca8-57b1-48d7-9cc4-d0b44983ca79";
    private static final String  connectorTypeQualifiedName = "Egeria:GovernanceActionService:Provisioning:MoveCopyFile";
    private static final String  connectorTypeDisplayName = "Move or Copy File Governance Action Service";
    private static final String  connectorTypeDescription = "Provisioning Governance Action Service that moves or copies files on request.";

    static final String TARGET_FILE_NAME_PATTERN_PROPERTY           = "targetFileNamePattern";

    static final String NO_LINEAGE_PROPERTY                         = "noLineage";
    static final String TOP_LEVEL_PROCESS_NAME_PROPERTY             = "topLevelProcessQualifiedName";
    static final String TOP_LEVEL_PROCESS_TEMPLATE_NAME_PROPERTY    = "topLevelProcessTemplateQualifiedName";
    static final String DESTINATION_TEMPLATE_NAME_PROPERTY          = "destinationFileTemplateQualifiedName";
    static final String TOP_LEVEL_PROCESS_ONLY_LINEAGE_PROPERTY     = "topLevelProcessLineageOnly";
    static final String LINEAGE_TO_DESTINATION_FOLDER_ONLY_PROPERTY = "lineageToDestinationFolderOnly";
    static final String LINEAGE_FROM_SOURCE_FOLDER_ONLY_PROPERTY    = "lineageFromSourceFolderOnly";

    static final String COPY_REQUEST_TYPE   = "copy-file";
    static final String MOVE_REQUEST_TYPE   = "move-file";
    static final String DELETE_REQUEST_TYPE = "delete-file";

    static final String SOURCE_FILE_PROPERTY        = "sourceFile";
    static final String DESTINATION_FOLDER_PROPERTY = "destinationFolder";

    static final String PROVISIONING_COMPLETE_GUARD = "provisioning-complete";
    static final String PROVISIONING_FAILED_GUARD   = "provisioning-failed";

    private static final String connectorClassName = MoveCopyFileGovernanceActionConnector.class.getName();


    /**
     * Constructor used to initialize the ConnectorProviderBase with the Java class name of the specific
     * store implementation.
     */
    public MoveCopyFileGovernanceActionProvider()
    {
        super();
        super.setConnectorClassName(connectorClassName);

        supportedRequestTypes = new ArrayList<>();
        supportedRequestTypes.add(COPY_REQUEST_TYPE);
        supportedRequestTypes.add(MOVE_REQUEST_TYPE);
        supportedRequestTypes.add(DELETE_REQUEST_TYPE);

        supportedRequestParameters = new ArrayList<>();
        supportedRequestParameters.add(SOURCE_FILE_PROPERTY);
        supportedRequestParameters.add(DESTINATION_FOLDER_PROPERTY);

        supportedTargetActionNames = new ArrayList<>();
        supportedTargetActionNames.add(SOURCE_FILE_PROPERTY);
        supportedTargetActionNames.add(DESTINATION_FOLDER_PROPERTY);

        supportedGuards = new ArrayList<>();
        supportedGuards.add(PROVISIONING_COMPLETE_GUARD);
        supportedGuards.add(PROVISIONING_FAILED_GUARD);

        super.setConnectorClassName(connectorClassName);

        ConnectorType connectorType = new ConnectorType();
        connectorType.setType(ConnectorType.getConnectorTypeType());
        connectorType.setGUID(connectorTypeGUID);
        connectorType.setQualifiedName(connectorTypeQualifiedName);
        connectorType.setDisplayName(connectorTypeDisplayName);
        connectorType.setDescription(connectorTypeDescription);
        connectorType.setConnectorProviderClassName(this.getClass().getName());

        List<String> recognizedConfigurationProperties = new ArrayList<>();
        recognizedConfigurationProperties.add(TARGET_FILE_NAME_PATTERN_PROPERTY);
        recognizedConfigurationProperties.add(NO_LINEAGE_PROPERTY);
        recognizedConfigurationProperties.add(TOP_LEVEL_PROCESS_NAME_PROPERTY);
        recognizedConfigurationProperties.add(TOP_LEVEL_PROCESS_TEMPLATE_NAME_PROPERTY);
        recognizedConfigurationProperties.add(DESTINATION_TEMPLATE_NAME_PROPERTY);
        recognizedConfigurationProperties.add(DESTINATION_FOLDER_PROPERTY);
        recognizedConfigurationProperties.add(TOP_LEVEL_PROCESS_ONLY_LINEAGE_PROPERTY);
        recognizedConfigurationProperties.add(LINEAGE_FROM_SOURCE_FOLDER_ONLY_PROPERTY);
        recognizedConfigurationProperties.add(LINEAGE_TO_DESTINATION_FOLDER_ONLY_PROPERTY);
        connectorType.setRecognizedConfigurationProperties(recognizedConfigurationProperties);

        super.connectorTypeBean = connectorType;
    }
}

Writing the governance action service connector¶

A governance action service must extend one of the governance action service base classes. Your choice determine the methods available to your governance action service through the governance action context.

• GovernanceActionService - for the complete context
• WatchdogGovernanceActionService - for the shared (white) and watchdog (blue) methods
• ProvisioningGovernanceActionService - for the shared (white) and provisioning (purple) methods
• VerificationGovernanceActionService - for the shared (white) and verification (green) methods
• TriageGovernanceActionService - for the shared (white) and triage (yellow) methods
• RemediationGovernanceActionService - for the shared (white) and remediation (red) methods

In addition, it needs to implement the standard Open Connector Framework (OCF) methods:

• initialize - an optional method where the connection object can be processed to, say, extract the configuration properties.
• start - this is where the processing logic of your governance action service belongs.
• disconnect - an optional method to free up any runtime resources that the governance action service is using. It is called after the governance action process records a completion status, or if it throws an exception from the start method.

Watchdog context methods¶

The watchdog governance action service monitors changes in the metadata and initiates one of the following as a result:

• [engine action](/concepts/engine-action
• governance action process
• incident report

One example of a watchdog governance action service is to monitor for new assets. Another example is to monitor the addition of open discovery reports and take action on their content.

Verification context methods¶

A Verification governance action service tests the properties of specific open metadata elements to ensure they are set up correctly and do not indicate a situation where governance activity is required. For example, it may check that a new asset has an owner assigned, is set up with governance zones and includes a connection and a schema.

The verification governance action services publishes guards to report on any errors it finds. It may also create incident reports to coordinate actions to correct any errors. The guards returned from the verification governance action service can be used to trigger other governance services as part of a governance action process.

Triage context methods¶

Triage governance action services run triage rules to determine how to manage a situation that needs human intervention. This could be to initiate an external workflow, wait for manual decision or initiate a remediation request through either an external workflow or by creating a ToDo for a specific person.

Remediation context methods¶

The remediation governance action service performs updates to metadata. Examples of the tasks performed by a remediation governance action service are assigning classifications, duplicate linking and consolidation.

Provisioning context methods¶

A provisioning governance action service invokes a provisioning service whenever a provisioning request is made. Typically, the provisioning service is an external service or pipeline. The provisioning governance action service may also create lineage metadata to describe the provisioning if this is not performed by another service.

Raise an issue or comment below