Skip to content

Planning Runtime Deployment Guide

Egeria is highly flexible and configurable and so the first piece of advice is to start small and simple and then expand as your experience and understanding of your workloads grows.

Platforms and servers

Egeria exchanges metadata between many types of tools distributed across different data centers and cloud platforms.

Egeria integrating different tools distributed over many locations

The green clouds represent each of the deployment locations. The different technologies are shown as grey boxes and Egeria itself is shown in blue and orange.

The blue rounded boxes are Egeria's Open Metadata and Governance (OMAG) Server Platform. This platform is the heart of Egeria's implementation. Typically you would expect to have at least one OMAG Server Platform deployed in each location. However, when you are experimenting with Egeria, it is often sufficient to start with one OMAG Server Platform and expand the number of platforms as you expand the technologies being integrated.

The OMAG Server Platform is capable of hosting one or more Open Metadata and Governance (OMAG) Servers. The OMAG Servers are the orange circles in the illustration above. They manage the connectors to third party technology as well as the frameworks and intelligence that Egeria brings to distributed metadata management.

It is a simple command to move an OMAG Server from one platform instance to another. This means you can start experimenting with a single platform and then add more as your deployment grows. The platform can also run as a node in container technologies such as Docker and Kubernetes.

OMAG Server deployment choices

Container platform deployment

Alternatively, for cloud or hybrid deployment scenarios, you may want to offload some of the platform responsibilities to an orchestrated container hosting environment such as Kubernetes. In such a scenario, OMAG servers run as individual containers. To visualise this, look at the topology image at the beginning: The blue rounded boxes are Kubernetes clusters and the orange circles are OMAG servers running as containers. Read more about this approach in the Container Platform Deployment section below.

Different types of technology need different types of integration and Egeria has OMAG Servers to match. Each type of OMAG Server is focused on the integration of a specific type of tool, engine or platform:

Types of OMAG Servers

The way to understand the diagram is that the arrows should be read as is a. For example, the repository proxy is a cohort member and the cohort member is a OMAG Server. This means that everything documented about a particular type of server is also true for all server types that point to it through the is a arrow, all the way down the hierarchy.

Object-oriented software engineers would know of this type of relationship as behavior inheritance.

OMAG Server interactions

How the OMAG Servers interact

Suggested incremental deployment approach

The servers' integration can be viewed as a series of nested spheres. The inner sphere involves the cohort members and provides metadata exchange between metadata repositories (and conformance test the integrations). The next level out adds the governance servers to automate the exchange of metadata between the metadata repositories and third party tools, engines and platforms. Finally, adding the view server and user interfaces delivers a governance solution to an organization.

Spheres of interaction between OMAG Servers

This architecture means that you can incrementally add function to your deployment. Here is a suggested approach:

  1. Start with creating an Egeria graph metadata access store. This will provide a metadata repository that can store any type of open metadata.
  2. Decide on a name for an open metadata repository cohort and configure your graph metadata repository to join it.
  3. If you want to have other third party metadata repositories that you want to share metadata with, configure repository proxies for each including registering them to the same cohort as the metadata server.
  4. If you then want to add in metadata synchronization with other types of technology beyond metadata repositories, work out which integration daemons you need and configure them to connect to the metadata access server. Make sure the appropriate access services for these integration daemons are enabled in the metadata server.
  5. If you want to use the governance services then these run in an engine host server and connect to the metadata server via the Governance Server OMAS and Governance Engine OMAS.

Working through this exercise gives you an understanding of the Egeria technology that you need for your deployment and how it connects together. The Solutions Guide describes different solutions that you can build with Egeria, how they work and the configuration that you will need.

Building connectors

If you discover that there is a third party technology that is not currently supported by Egeria then you need to build a connector to translate between the APIs and events of that technology and the open metadata APIs and events.

Identify the scope of the metadata integration

Another useful planning exercise is to identify the community of users and the tools that they use that need to share metadata. This gives you a view of the technology that needs to be integrated. If each community is fairly self-contained with their own tools deployment then you may want to consider deploying an OMAG Server platform for this community and deploying the servers they need onto it. This means they can control their access to open metadata along with the delivery of open metadata to the rest of the organization.

More importantly, it helps with the definition of the organization's governance zones.

Deployment checklist

This is a checklist of planning tasks for the deployment of your OMAG Server Platforms and OMAG Servers:

Container platform deployment

Egeria software architecture adopts well the microservice architecture principles. OMAG server can be configured to provide specific capabilities and context bound metadata services. Container platform deployment style complements this with the runtime aspects allowing you to run OMAG servers independently as isolated, stateless and immutable containers.

Container platform

Kubernetes (or k8s) is de-facto standard platform for deploying, scaling and managing containerized applications. To support this deployment model better, Egeria offers implementation that is designed to start server in a container more efficiently complying with the principles mentioned above.

It is very important to note that wether you choose to run server on a native OMAG platform or in a stand alone container runtime, the same functional principles apply - all servers are interoperable and comply to the open metadata exchange protocols.

More detail to follow

The text above is a very high level overview of the planning process. More detail will be added to this guide as time permits.

Raise an issue or comment below