Skip to content
Stable

This component is complete and can be used. The interfaces will be supported until the function is removed from the project via the deprecation process. There will be ongoing extensions to this function, but it will be done to ensure backward compatibility as far as possible. If there is a need to break backward compatibility, this will be discussed and reviewed in the community, with a documented timeline.

Data Manager Open Metadata Access Service (OMAS)

The Data Manager OMAS provides APIs for technologies wishing to register new data assets, connections and related schema from data resources located in database servers, file systems, event brokers, API gateways and file managers and content managers.

The caller of this interface may be the data manager itself, or an integration daemon if the data manager does not support open metadata directly. The integration daemon calls the Data Manager OMAS client through the following integration services.

There are specific APIs for different types of data managers and assets. These reflect the terminology typically associated with the specific type of data manager to make it easier for people to map the Data Manager OMAS APIs and events to the actual technology. However, the specific implementation objects supported by these APIs all inherit from common open metadata types so it is possible to work with the resulting metadata in a technology agnostic manner using services such as the Asset Consumer OMAS.

Basic metadata model

Figure 1 shows the types of metadata captured by the Data Manager OMAS.

Figure 1

Figure 1: Basic metadata capture by the Data Manager OMAS

These are:

  • Asset - Asset describes the digital resource such as the data set, database schema, topic, API etc.
  • Connection, Connector Type and Endpoint are used to create a connector to access the data in the data asset.
  • Schema Element(s) describe the structure of the data managed by the data asset.

To make it possible to search for particular types of asset, there are many specialized asset types defined in Egeria. The full list is shown here, however Data Manager OMAS supports the following subtypes of Asset (and any additional subtypes of these types that you wish to define yourself).

  • DeployedAPI for API descriptions.
  • Topic for topics supported by an event manager.
  • DataFile for a file with sub types of:
  • CSVFile for CSV files.
  • AvroFile for files using the Avro format.
  • JSONFile for files using the JSON format.
  • Database for databases.
  • DeployedDatabaseSchema for schemas within a database.
  • DeployedReport for reports.
  • Form for interactive forms.
  • InformationView for queries.

The Data Manager OMAS APIs needs to accommodate slight variations between different vendor implementations of data managers, along with information relating to local deployment standards. As such there is provision in these interfaces to support:

  • VendorProperties for properties unique to a specific vendor implementation, and
  • AdditionalProperties for properties that the metadata team wish to add to the metadata.

Data Managers

The Data Manager OMAS Supports the following types of data managers:

Icon Name Provenance Description
File System File System Local Cohort Create metadata elements for files and folders along with their data connections and any known schema information. Catalogued files and folders are members of the local cohort because many different types of processes may work with them.
File Manager File Manager External Create metadata elements for files and folders along with their data connections and any known schema information. Catalogued files and folders are members of the data manager's metadata collection because it is responsible for their maintenance.
Database System Database Manager External Create metadata elements for databases, database schemas, tables, views, columns, primary keys and foreign keys. Catalogued elements are members of the data manager's metadata collection because it is responsible for their maintenance.
Event Manager Event Broker Local Cohort or External Create metadata elements for topics and the event payloads they support. Catalogued elements are members of the data manager's metadata collection (ie External provenance) if it is responsible for their maintenance.
API Gateways API Manager Local Cohort or External Create metadata elements for APIs and their supported headers and payloads. Catalogued elements are members of the data manager's metadata collection (ie External provenance) if it is responsible for their maintenance.
Display Tools Display Tools Local Cohort or External Create metadata elements for reporting tools, forms and associated queries. Catalogued elements are members of the data manager's metadata collection (ie External provenance) if it is responsible for their maintenance.

More information

Raise an issue or comment below