A connector is a client library for accessing a resource and, optionally, its associated asset metadata.
The concept of a connector comes from the Open Connector Framework (OCF). The OCF provides a common framework for components (ie connectors) that enable one technology to call another, arbitrary technology through a common interface. The implementation of the connector is dynamically loaded based on the connector's configuration.
Through the OCF, we can:
- Plug-in different technologies to Egeria.
- Plug-in support for open metadata into the client libraries used by applications to access data resources and services.
Many subsystems in Egeria's OMAG Server Platform and servers support the first approach. They define a specialized interface for the type of connector they support.
- One or more connector implementations supporting that interface are then written either by the Egeria community or other organizations.
- When an Egeria OMAG Server is configured, details of which connector implementation to use is specified in the server's configuration document using the connection structure.
- At start up, the OMAG Server passes the connector configuration to the OCF to instantiate the required connector instance.
Connectors enable Egeria to operate in many environments and with many types of third party technologies, just by managing the configuration of the OMAG Servers.
The second approach is used by organizations that want to make use of metadata directly in applications and tools - or to externalize the security and driver properties needed to call the data source or service. In this case the OCF connector typically has the same interface as the data source's client library (unless you can do better ). This minimizes the learning curve for application developers. The configuration for the connector is stored in an open metadata server and the application uses the Asset Consumer OMAS client to request a new instance of the connector. The application uses the returned connector instance to access the data source or server along with the metadata stored about it.
The connector provides two main APIs:
- An standard API called
getConnectedAssetPropertiesfor retrieving metadata. This API is defined by the OCF.
- An API for working with the resource itself. This interface is specialized for the type of resource and the use it is being put to.
Specialist connectors that plug into the OMAG Server Platform¶
Various Egeria services provide specific interfaces for connectors to support their operation. For example, the Open Metadata Integration Services (OMIS) define the interfaces for integration-connectors used to exchange metadata between Egeria and third party technologies.
The table below lists the different types of connectors supported by Egeria's services. Connectors that are implemented by the Egeria community or trusted third parties are listed in the connector catalog. You may write your own connectors to integrate additional types of technology or extend the capabilities of Egeria - and if you think your connector is more generally useful, you could consider contributing it to the Egeria project.
|Type of Connector||Description||Documentation||Connector Catalog link|
|Integration Connector||Implements metadata exchange with third party tools.||Building Integration Connectors||Integration Connector Catalog|
|Open Discovery Service||Implements automated metadata discovery.||Building Open Discovery Services||Open Discovery Service Catalog|
|Governance Action Service||Implements automated governance.||Building Governance Action Services||Governance Action Services Catalog|
|Archive Service||Maintains open metadata archives based in the content of the open metadata repositories.||Building Archive Services||Archive Service Catalog|
|Configuration Document Store||Persists the configuration document for an OMAG Server.||Building Configuration Document Store Connectors||Configuration Document Store Catalog|
|Platform Security Connector||Manages service authorization for the OMAG Server Platform.||Building Platform Metadata Security Connectors||Platform Metadata Security Connector Catalog|
|Server Security Connector||Manages service and metadata instance authorization for an OMAG Server.||Building Server Metadata Security Connectors||Server Metadata Security Connector Catalog|
|Open Metadata Archive Store||Reads an open metadata archive from a particular type of store.||OMRS Open Metadata Archive Store Connector||open-metadata-archive-connectors|
|Audit Log Store||Audit logging destination||OMRS Audit Log Store Connector||audit-log-connectors|
|Cohort Registry Store||Local store of membership of an open metadata repository cohort.||Building Cohort Registry Store Connectors||cohort-registry-store-connectors|
|Open Metadata Topic Connector||Sends and received events on a topic hosted by an external event bus such as Apache Kafka.||Building Open Metadata Topic Connectors||open-metadata- topic-connectors|
|Metadata Collection (repository) Store||Interfaces with a metadata repository API for retrieving and storing metadata.||OMRS Repository Connectors||open-metadata-collection-store-connectors|
|Metadata Collection (repository) Event Mapper||Maps events from a third party metadata repository to open metadata events.||OMRS Event Mappers||none|
OCF connectors are not limited to representing Assets as they are physically implemented. An OCF connector can represent a simplified logical (virtual) asset, such as a data set, that is designed for the needs of a specific application or tool. This type of connector delegates the requests it receives to one or more physical data resources. It is called a virtual connector and us created using a virtual connection.
The connector catalog describes the connectors that are pre-built for the Egeria ecosystem.
The Egeria Developer Guide describes how to write your own connectors.
Raise an issue or comment below