Skip to content

Latest Release

Release 4.3 (September 2023)

Release 4.3 continues the process of improving Egeria's usability. These changes have some effect on backwards compatibility but will lead to a smoother experience in the long term.

Change to the original Open Metadata Assembly

When the gradle build runs in the core egeria git repository (egeria.git), it creates an assembly of the Egeria runtimes. This is located in the build directory under open-metadata-distributions/open-metadata-assemblies. In release 4.2, this assembly was updated to create a new directory called platform which held the runtime and libraries for the OMAG Server Platform. The docker image for Egeria (and hence the helm charts) was updated so that it runs the platform from this new directory. The platform runtime was originally located in a directory called server which caused confusion. The content of the server directory was not changed at that time. However, in 4.3, the server directory now contains a new cloud native runtime for a single OMAG Server. This is the structure you will see in 4.3.

New assembly structure

New assemblies for the OMAG Server Platform and Egeria UI Platform

The single assembly described above suffered from conflicts of runtime settings between the different runtimes. The result was a complex and less than obvious organization of the files. There are new assemblies created during the core egeria build that are focused on a single runtime. Their structure is simpler, making it more obvious which files/properties are controlling each runtime. These assemblies are located under open-metadata-distribution. Each assembly is created by its own module. The egeria-ui-application module builds the an assembly for the Egeria UI Application. This includes the certificates and property files used by this application. The OMAG Server Platform assembly is created by a module called omag-server-platform. It includes three main directories:

  • platform - The platform runtime.
  • etc - Utilities and reports.
  • opt - Optional content including sample metadata (loadable as an Open Metadata Archive), sample data, sample applications that call the Egeria APIs and postman collections

The top level directory also includes a docker build configuration that packages files under the platform directory into a docker container and publishes it to your docker daemon. Once egeria's grable build is complete, you can add and remove libraries and other content from the platform directory and then build your own customized docker container.

Each directory in the assembly includes a README.md file to explain its contents.

New OMAG server standalone runtime

We are releasing tech preview for the standalone OMAG server spring boot launcher application. This is the first milestone achieved as result of the work planned in the Cloud Native workgroup. It is step forward to enable additional deployment/operation model more suitable for orchestrated container runtime environment such as Kubernetes. Functionally, it runs the same set of OMAG server capabilities/services as if you are running 'virtual' server instance on an OMAG platform. Irrespective of the operating style the OMAG servers are always interoperable.

We are working to update the guides on the website to address this new feature.

If you are curious, you can already try it out by following technical how-to instructions on github or check-out the new egeria-server helm chart showcasing simple OMAG server deployment.

Feedback is always appreciated.

Database connectors are now in egeria.git

The JDBC database connectors are now located in the core Egeria git repository. The JDBC Resource Connector now includes helper functions when issuing simple SQL commands.

Developer utilities are now in egeria.git

The utilities located in egeria-dev-projects.git are now located in egeria.git under the open-metadata-resources module. They are now included automatically in the omag-server-platform assembly. These utilities are used in the Developer Dojo. This will be updated to use the new assembly once release 4.3 is shipped.

Extended integration connector function

Integration connectors can now check to see if they are in the middle of refresh() processing. This is valuable in multi-threaded operation, for example when processing events. Typically, many of the events that are generated while the connector is being called to refresh() are caused by the connector's activity. There is a new method on the integration context called isRefreshInProgress(). It returns true if the connector is currently running its refresh() method. An example of how to use this method is found in the developer guide.

New configuration command for when a cohort member moves platform

Servers that are members of a cohort regularly broadcast their network address to other members to allow them to dynamically configure their federated query support. This network address is configured in the OMAG Server's configuration document as the localRepositoryRemoteConnection's endpoint address. When an OMAG Server is relocated to a new platform, this address must change. There is a new administration command called resetRemoteCohortURL with URL POST {platformURLRoot}/open-metadata/admin-services/users/{userId}/servers/{serverName}/local-repository/configuration/remote-repository-connector-url. The new URL is passed in the request body:

{
    "class": "URLRequestBody",
    "urlRoot": "{platformURLRoot}/servers/{serverName}"
}
Open Metadata Integration Services (OMISs) updates
Apache Atlas Integration Connector

The Apache Atlas Integration Connector has been enhanced to catalog Assets and Lineage from Apache Atlas into the open metadata ecosystem.

A future version will maintain semantic assignment relationships between the glossary terms, exchanged in the first release of the connector, with the Apache Atlas assets it finds.

Release 4.3 also extracted the java client from this connector and created a new resource connector for Apache Atlas. The resulting Apache Atlas REST Connector provides a Java API that calls the Apache Atlas REST API with no dependencies on Apache Atlas nor the Hadoop libraries it depends on. This resource connector is used in the Apache Atlas Integration Connector and is intended for new connectors for Apache Atlas planned to be written for a future release.

Conformance Test Suite

There is a new option on the repository workbench to allow you to restrict the number of different types of instances that are tested. By default, all combinations of the types supported by the technology under test, are used when testing the instance methods. For native repository connectors, that can add up to a lot of tests, taking many hours to run. In 4.3, we have introduced a new option called testEntityTypes that is added to the workbench configuration. It is set to a list of entity types such as ["Asset", "InformalTag"] and the tests are restricted to these types, their super types and any relationships and classifications that can connector to these entity types.

Metadata Observability

There is a new Audit log severity called ACTIVITY that is used to report activity in the open metadata ecosystem. This severity is used when logging messages about:

  • The REST API calls made by each user.
  • The requests to retrieve, create and maintain assets.
Open Metadata Types
  • The HostClusterMember relationship has new properties memberRole and additionalProperties to improve ability to describe a host cluster operation.
  • The DataSet, DataStore, DeployedAPI, DeployedSoftwareComponent and DataFeed entity types each have a new attribute called deployedImplementationType to describe the technology supporting the associated component. This addition is to enable integration connectors to be more precise in describing the technology they are cataloguing. This attribute is also used in SoftwareServerPlatform, SoftwareServer and SoftwareCapability entity types; along with the ServerPurpose and CloudPlatform classification. There are subtypes of DataStore (LogFile, GraphStore, MetadataRepository) have been modified because they already include the property and duplicate properties in the type hierarchy cause indexing problems in some repositories. Similarly, there were duplicated deployedImplementationType properties in the ServerPurpose classification affecting RepositoryProxy and MetadataServer classifications.
  • A new entity type called RelationalDatabase has been added to distinguish databases that follow the relational schema and can be accessed through JDBC.
  • The Person entity type has a new attribute called residentCountry to describe the primary country of residence. In addition, there is a new attribute called timeZone to describe the time zone of the location where this person is located.
New Helm chart and dependency updates

New helm chart egeria-server to demonstrate stand-alone omag server deployment in kubernetes.

Dependency updates include:

  • Egeria docker image 4.3
  • Strimzi 0.37.0 and Kafka 3.5.1
  • Jupyter image based on minimal-notebook:lab-4.0.5 and egeria notebook fixes V4.3

Raise an issue or comment below