Skip to content

Roadmap

Egeria is a large project with many activities adding content to the project. This page provides an overview of the aims of the project and a reflection of where we are today.

Capability layers

Egeria aims to deliver against 6 capability layers:

Governance Solutions Education UIs Integration Platform Developer Platform Deployment Runtimes Deployment Resources

Governance solutions

Support the leadership team for a governance program providing the ability to create governance definitions, subject areas and governance roles and to monitor the success of the governance efforts across the enterprise.

Governance Solutions

The implementation of a governance solution is focused mainly on the extension of the Egeria UI to support additional roles and functions. They make use of the services provided by the developer platform and may exploit additional content, utilities and connector implementations from the integration platform.

Education

Provides educational resources for different personas and starting points.

Education

Egeria's education aims to broaden the knowledge of people who need to work with digital resources about metadata, governance practices and the use of Egeria. They are It is based around the Coco Pharmaceuticals scenario and are organized by persona, so you can target your learning to your interests.

  • The content packs are included in the omag-server-platform distribution created by the build for egeria.git. They are formatted as Open Metadata Archives and can be configured to load at server startup using the Administration Services or while the server is running using the Server Operations.
  • There are also many sample clients, server configurations and sample data included in the omag-server-platform distribution created by the build for egeria.git. They are accompanied by README.md files to explain how to use them.
  • Reference data in the form of metadata valid values is supplied in omag-server-platform distribution created by the build for egeria.git.
  • The Hands-on labs provide practical experiences in running the Egeria code and using the different services.
  • The Governance practices provides governance best practices. They aim to guide a team that is setting up or revising their governance program through common governance tasks.

User Interfaces

Most users will experience the open metadata ecosystem via their own tools. However Egeria does have some simple user interfaces to cover its unique capabilities"

UIs

  • Rich Widgets and Textual UIs provide visualization of key element. They are designed for technical people operating the platform.
  • Brain Explorers are cloud based websites allowing you to interact with a graph interface to visualized open metadata content.
  • Documentation site contains comprehensive documentation on Egeria's features.

Integration platform

Supports integration of popular technologies by installing and configuring Egeria. Minimal coding is still required around unusual and home-grown tools and technologies.

Integration Platform

  • Utilities and converters: support for different standard formats to load industry standard definitions, models, glossaries, and other content packs built on industry standard definitions, models, glossaries and other content packs. Examples include JSON-LD, OWL/RDF, XML, ...
  • There are clients written in both Java and Python to aid programmers calling the Egeria services.
  • Pre-canned connectors to third party technologies: popular metadata repositories, databases, data formats and platforms; data movement engines, data virtualization engines, dev ops tools, analytics/AI tools, data catalogs, MDM and user directories, CMDBs, SDLC tools, ...
  • Conformance test suite: Supports the testing of third party connectors. Each type of connector or service is supported by its own test workbench.

Developer platform

Provides frameworks, APIs, and hosting platforms for building an integrated metadata and distributed governance solutions.

Developer Platform

The developer platform contains the core Egeria implementation and provides support for integrating third party technology into the open metadata ecosystem and extending Egeria to run in different environments or to use different infrastructure services.

Its use is described in the developer's guide.

  • Open Metadata and Governance (OMAG) registered services are dynamically loaded in the OMAG Server Platform. This means they can be added and removed as needed to create a customized platform. This may include registered services written by the Egeria community and supplied by third parties.

    • Access services provide specialist APIs / events for different types of tools. They work with the pre-defined open metadata types and use the repository services to access metadata.
    • Engine services provide the services that host a specific type of governance engine. The governance engines collectively provide active governance to the assets and their associated metadata.
    • Integration services each provide a specialized API to integration connectors. These are hosted in an integration daemon. The purpose of the integration services is to simplify the implementation and management of connectors that integrate metadata exchange with third party technologies.
    • View services provide the services used by UIs. They are typically fine-grained services and they run in the view server. The use of the separate server (and server platform) enables an extra firewall to be erected between the view servers and the metadata servers and governance servers, hiding the internal systems from end users.

  • The open metadata types provide definitions for the different types of metadata needed by an organization. The open metadata type system is extendable; however, by providing a comprehensive starter set, and encouraging tools to use them, Egeria ensures metadata can be seamlessly shared amongst them.

  • The framework services provide Egeria clients to support metadata retrieval for connectors defined by the frameworks.

  • The OMAG Server Platform provides a multi-tenant runtime platform for OMAG Servers. Each OMAG Server hosts the connectors along with the Egeria services to integrate third party technology.

  • The open metadata frameworks define the interfaces implemented by components that "plug-in" to Egeria, either to integrate calls to third party technology or extend the function of Egeria. The frameworks are as follows:

Deployment runtimes

The runtimes package up the services to simplify the process of deploying Egeria's capability.

Deployment Runtimes

There are three runtimes:

  • OMAG Server Platform provides support for running multiple OMAG Servers.
  • OMAG Server runtime provides support for running a single OMAG Servers.
  • Egeria UI Application is a deprecated runtime for the backend services for the Egeria General User UI.

Deployment resources

Aim to simplify the process of deploying the OMAG Server Platform and its connectors into an operational environment.

Deployment Resources

  • The Egeria docker image is built daily and pushed to Quay.io. It contains an OMAG Server Platform. You can download it and use it in your own container environments. It is used by the helm charts.
  • The Kubernetes Helm charts make use of the docker image to create a rich Egeria deployment used in the open metadata labs.
  • The Kubernetes operators are in development. They will provide an easy way to control an Egeria deployment running on Kubernetes.

Support for docker compose was removed in release 3.5.

Understanding the roadmap

Current status

Following is an overview of the content status of the functions in Egeria's latest release.

Status of functions found in each capability layer

As you can see, some progress has been made on all layers. However, since they do build on one another, most of the early work has been focused on establishing the frameworks, connector APIs and other services to provide the developer platform. The developer platform provides the libraries and interfaces to build connectors to integrate third party tools along with the runtime to host these connectors and manage the metadata exchange.

Today we have a robust OMAG Server Platform and the ability to configure OMAG Servers that host specific types of connectors to third party tools. The continual development of the registered services is broadening the types of metadata, and tools, that can integrate with Egeria through the federated query mechanism that

History

The initial implementation of Egeria focused on the Open Metadata Repository Services (OMRS) to support different types of metadata repositories exchanging metadata via an open metadata repository cohort. This also involved the build out of the OMAG Server Platform to host this code. The aim was to demonstrate how third party metadata servers could exchange metadata. This capability was delivered along with two repository connectors for the following third party connectors along with the Conformance Test Suite.

Through 2020, our focus shifted to the integration platform as we added connector implementations for popular third party technologies (see the connector catalog) and built out the ecosystem user interface (UI) that enables an organization to:

  • configure OMAG Servers on OMAG Server Platforms
  • visualize the open metadata types through the type explorer (TEX)
  • visualize open metadata instances in a single repository or across the open metadata repository cohorts that a server is connected to.
  • visualize to cohort and query the operational status of the OMAG Servers and services operating in the open metadata ecosystem
  • configure OMAG Servers and deploy them to OMAG Server Platforms

The ecosystem UI makes calls to specialized REST services supported by a type of OMAG Server called the view server. The view server is new for 2020 and enables the REST APIs to the UIs to be deployed in a DMZ and the metadata servers to be behind an additional firewall. It also takes much of the load for supporting end users off of the metadata servers.

In 2020 support for a new type of OMAG Server called the integration daemon was also added. This server supports integration services that can host integration connectors dedicated to exchanging metadata with specific third party technologies.

2021 had a focus on governing metadata. There is a new OMAG Server called the engine host that runs governance engines. These are supported by new access services for governance.

This new website was added to the project in 2021, and it has resulted in more interest in consuming Egeria.

2022 continued the focus on metadata governance. The following OMASs were refactored to call the generic handlers rather than direct calls to the repository handler.

There was investment in both the function and performance of the generic handlers, which provide many of the metadata governance functions supported by all OMASs, such as metadata security, provenance validation, anchor management, LatestChange classifications, effectivity dating, memento management and de-duplicating query results.

Integration with third party technologies made good progress with the addition of OpenLineage support, the new JDBC, Hive Metastore, schema registry, OpenAPI Specification and Apache Kafka connectors.

2023 and 2024 saw a dramatic change in the project's direction as it shifted its primary focus from providing libraries for tools vendors to supporting organizations that are looking to create an open metadata ecosystem. This resulted in more samples, view services content packs and a new range of UIs. The OMAG Server Platform has had some internal restructure and clean up of deprecated code. The result is a simpler coding experience for new community members.

Future Plans

After the release of version 5.0, the team plans to focus on publicity around the new platform and its advanced capability. There will also be more connectors, increasing Egeria's reach across the IT landscape.

Raise an issue or comment below