Skip to content

Writing a Survey Action Service

An survey-action service is a component that performs analysis of the contents of a digital resource on request. The aim of the survey action service is to enable a detailed picture of the properties of a resource to be built up.

Each time a survey action service runs, it creates a new survey report linked off of the digital resource's Asset metadata element that records the results of the analysis.

Asset with survey reports

Each time an survey action service runs to analyse a digital resource, a new survey report is created and attached to the resource's asset. If the survey action service is run regularly, it is possible to track how the contents are changing over time.

The survey report contains one or more sets of related properties that the survey action service has discovered about the resource, its metadata, structure and/or content. These are stored in a set of annotations linked off of the survey report.

An survey action service is designed to run at regular intervals to gather a detailed perspective on the contents of the digital resource and how they are changing over time. Each time it runs, it is given access to the results of previously run survey-action services, along with a review of these findings made by individuals responsible for the digital resource (such as stewards, owners, custodians).

Operation of an survey action service

Operation of a survey action service

  1. Each time a survey action service runs, Egeria creates a survey report to describe the status and results of the survey action service's execution. The survey action service is passed a survey context that provides access to metadata.
  2. The survey context is able to supply metadata about the asset and create a connector to the digital resource using the connection information linked to the asset. The survey action service uses the connector to access the digital resource's contents in order to perform the analysis.
  3. The survey action service creates annotations to record the results of its analysis. It adds them to the survey context which stores them in open metadata attached to the survey report.
  4. The annotations can be reviewed and commented on through an external stewardship process. This means choices from, for example, a list of potential options proposed by the survey action services, can be verified and the best one selected by an individual expert. The resulting choices are added to annotation reviews attached to the appropriate annotations.
  5. The next time the survey action service runs, a new survey report is created to link new attachments.
  6. The survey context provides access to the existing attachments for that asset along with any annotation reviews. The survey action services is able to link its new annotations to the existing annotations as an annotation extension. This means that the stewards can see the history associated with the new information.
Runtime for an survey action service

Survey action services are packaged into Survey Action Engines that run in the Survey Action OMES hosted in an Engine Host.

The metadata repository interface for metadata discovery tools is implemented by the Stewardship Actions OMAS that runs in a Metadata Access Server.

An survey action service may be triggered via an Engine Action or as part of a governance action process.

Survey Action Service

There is a lot of common functions that are used repeatedly during the discovery process.

An open discovery pipeline is a specialized implementation of an open discovery service that runs a set of open discovery services against a single digital resource. The implementation of the open discovery pipeline determines the order that these open discovery services are run.

Open discovery pipeline example

Each open discovery service in the pipeline is able to access the results of the open discovery services that have run before it through the discovery context. The combined results of the open discovery pipeline are grouped into a single discovery analysis report linked off of the asset.

The aim of the open discovery pipeline is to enable reusable open discovery service implementations to be choreographed together for different types of digital resource.

Implementing survey action services

A survey action service is implemented as a specialized Open Connector Framework (OCF) connector. This means it has two parts to it: a connector provider that is responsible for creating a new instance of the survey action service and the survey action service implementation (that is the connector implementation in the OCF terminology.)

Dependencies

The interfaces and base classes of the connector provider and survey action service implementation are provided by the Survey Action Framework (ODF). This means you need to include the following Egeria modules in your build script:

  • Open Connector Framework
  • Survey Action Framework
  • Governance Action Framework for access to open metadata definitions.
  • Audit Log Framework for audit logging and exceptions.
  • The digital resource connector(s) used to gain access to the digital resource being analysed.
    <dependencies>

        <dependency>
            <groupId>org.odpi.egeria</groupId>
            <artifactId>audit-log-framework</artifactId>
        </dependency>

        <dependency>
            <groupId>org.odpi.egeria</groupId>
            <artifactId>open-connector-framework</artifactId>
        </dependency>

        <dependency>
            <groupId>org.odpi.egeria</groupId>
            <artifactId>survey-action-framework</artifactId>
        </dependency>

        <dependency>
            <groupId>org.odpi.egeria</groupId>
            <artifactId>governance-action-framework</artifactId>
        </dependency>

        <dependency>
              :
           Digital resource connectors need to be included
              :
        </dependency>

    </dependencies>
dependencies {
    implementation project(':open-metadata-implementation:frameworks:audit-log-framework')
    implementation project(':open-metadata-implementation:frameworks:open-connector-framework')
    implementation project(':open-metadata-implementation:frameworks:survey-action-framework')
    implementation project(':open-metadata-implementation:frameworks:governance-action-framework')
    implementation project('...data store connector...')  
}

Connector provider for a survey action service

Each connector provider for a survey action service extends the following base class:

org.odpi.openmetadata.frameworks.surveyaction.SurveyActionServiceProvider
This assumes:

  • The survey action service implementation class is instantiated with the default constructor. This means all of its configuration information is contained in the Connection object supplied on the initialize() method.

If your survey action service implementation matches these requirements, its connector provider implementation need only implement a constructor to configure the base class's function with details of itself and the Java class of the connector it needs using:

  • a GUID for the connector type
  • a name for the connector type.
  • a description of what the connector is for and how to configure it.
  • the connector class it instantiates.
  • a list of the additional properties, configuration properties and secured properties needed to configure instances of the connector.
  • a description of the connector for its audit log (if the connector implements AuditLoggingComponent).
/**
 * XXXSurveyActionProvider is the OCF connector provider for the XXX survey action service.
 */
public class XXXSurveyActionProvider extends SurveyActionProviderBase
{
    /*
     * Unique identifier of the connector for the audit log.
     */
    private static final int    connectorComponentId   = 10001; /* Add unique number here - Egeria uses numbers under 1000 */

    /*
     * Unique identifier for the connector type.
     */
    private static final String connectorTypeGUID      = "Add unique GUID here";

    /*
     * Descriptive information about the connector for the connector type and audit log.
     */
    private static final String connectorQualifiedName = "MyOrg:XXXSurveyActionService";
    private static final String connectorDisplayName   = "XXX Survey Action Service";
    private static final String connectorDescription   = "A survey action service that supports ... add details here.";
    private static final String connectorWikiPage      = "Add url to documentation here";


    /*
     * Define the name of the connector implementation.
     */
    private static final String connectorClassName = "packagename.XXXSurveyActionService";

    /*
     * Define the name of configuration properties (optional).
     */
    public static final String REQUIRED_ANNOTATION_TYPES_CONFIGURATION_PROPERTY = "requiredAnnotationTypes";

    /**
     * Constructor used to initialize the ConnectorProviderBase class.
     */
    public XXXSurveyActionProvider()
    {
        super();

        /*
         * Set up the class name of the survey action service that this provider creates.
         */
        super.setConnectorClassName(connectorClassName);

        /*
         * Set up the connector type that should be included in a connection used to configure this survey action service.
         */
        ConnectorType connectorType = new ConnectorType();
        connectorType.setType(ConnectorType.getConnectorTypeType());
        connectorType.setGUID(connectorTypeGUID);
        connectorType.setQualifiedName(connectorQualifiedName);
        connectorType.setDisplayName(connectorDisplayName);
        connectorType.setDescription(connectorDescription);
        connectorType.setConnectorProviderClassName(this.getClass().getName());

        List<String> recognizedConfigurationProperties = new ArrayList<>();
        recognizedConfigurationProperties.add(REQUIRED_ANNOTATION_TYPES_CONFIGURATION_PROPERTY);
        connectorType.setRecognizedConfigurationProperties(recognizedConfigurationProperties);

        super.connectorTypeBean = connectorType;

        /*
         * Set up the component description used in the survey action service's audit log messages.
         */
        AuditLogReportingComponent componentDescription = new AuditLogReportingComponent();

        componentDescription.setComponentId(connectorComponentId);
        componentDescription.setComponentName(connectorQualifiedName);
        componentDescription.setComponentDescription(connectorDescription);
        componentDescription.setComponentWikiURL(connectorWikiPage);

        super.setConnectorComponentDescription(componentDescription);
    }
}

Survey action service implementation

The survey action service implementation inherits from:

org.odpi.openmetadata.frameworks.surveyaction.SurveyActionServiceConnector
This base class handles the survey context, any nested connectors and the audit log. The following variables are set up:

  • surveyServiceName - the qualified name of the survey action service from the open metadata definition.

  • surveyContext - the survey context that provides methods for querying and storing survey metadata.

  • auditLog - audit logging destination.

  • embeddedConnectors - connectors created from connections nested in the survey action service connection. These connectors are typically survey action services that are embedded because this survey action service is an survey action pipeline.

There are two methods that you need to implement:

  • start() - this method is where all the work takes place. It retrieves any configuration properties from the connection, uses the survey context to access the asset and its connector to the digital resource. As it analyses the content of the digital resource, it adds its results to the annotation store retrieved from the survey context. The annotations are immediately stored in one of the open metadata repositories.

  • disconnect() - this method is called either because the survey action service has completed running, or because its server is shutting down. The survey action service implementation should release any resources it is using - for example by calling the disconnect method on the digital resource connector it is using.

Survey Context

The survey context provides the survey action service with access to information about the survey request along with the open metadata repository interfaces.

Structure of the survey context

Survey Asset Catalog Store

The survey asset catalog store provides a search interface that enables a survey action service to locate assets that are described in the open metadata repository.

Annotation Store

The survey annotation store provides a survey action service with access to the annotations created about a specific asset both from past invocations of survey action services and those created during the current execution.

Open Metadata Store

Provides access to all types of open metadata through the Governance Action Framework (GAF).

Example implementations

The definition of the connector interfaces for survey action services is defined in the survey-action-services module.

There are implementations of survey action services in the survey-service-connectors.


Raise an issue or comment below