Skip to content

Tools for code contributors

Git and GitHub

Git is an open source version control system. The Egeria project uses git to:

  • Store the source code, documentation and other file-based resources.
  • Track changes to the underlying Egeria code as the project evolves
  • Track issues and enhancements, and link these back to the code changes that resolve them
  • Collaborate on and review the issues, enhancements and code changes

As a result, it gives us a definitive source for the latest and greatest source code for Egeria itself, its history, and the rationale behind various decisions that are made over time.

Egeria's git repositories on GitHub

The Egeria project's git repositories are located on GitHub . GitHub is a free, public git service for sharing code and related files. It has a web interface to make it easier for the Egeria community to monitor the activity in the project and process new content.

Egeria has the following repositories:

Repository Purpose
egeria Egeria core
egeria-docs Documentation repository for the Egeria project.
egeria-charts Helm chart repository
egeria-python Python library for Egeria
egeria-jupyter-notebooks Egeria Jupiter notebooks used in the Open Metadata Labs
egeria-template-newrepo Template for new Egeria repository
egeria-connector-hivemetastore Egeria repository connector for Hive metastore. Initially targeted at spark metadata
egeria-connector-ibm-information-server IBM Information Server connectors for Egeria: repository proxy connector for IGC, data engine proxy connector for DataStage.
egeria-connector-integration-event-schema Provides an integration connector that extract event schemata from a schema registry (including Confluent schema registry). The connector will be a polling connector and will look in Egeria for new topics that if present in the confluent registry, the associate schema elements will be brought into Egeria.
egeria-connector-integration-lineage-event-driven-sample Sample showing how to bring bespoke lineage into Egeria
egeria-connector-integration-topic-strimzi Strimzi Egeria integration connector for Kafka Topics
egeria-connector-omrs-caching Provides an OMRS repository proxy connector that has an embedded repository that can be used to cache entities and relationships.
egeria-connector-repository-file-sample Provides a repository proxy sample that uses polling. The target is a file folder
egeria-connector-sas-viya Egeria connector for SAS Information Catalog
egeria-samples-api A collection of samples illustrating the different APIs of Egeria.
egeria-test-cts Automated CTS (conformance test) execution for Egeria

All of these repositories are publicly visible. However, if you want to contribute new content then you need to create a GitHub account. This can be done from the top of the GitHub home page .

Further information

Interested to learn more?

IntelliJ IDEA

IntelliJ IDEA by JetBrains is the Interactive Development Environment (IDE) used by most of the Egeria developers. The community edition is free to use and covers all the function needed by an Egeria developer.

We provide our own tutorial for IntelliJ.

Build tools

The build process takes the source files from the git repository and creates executable libraries needed to run Egeria. When you download (clone) the contents of a git repository from GitHub, a new directory is created that is named after the repository that you cloned. For example, the directory created when the main egeria.git repository is cloned is called egeria. This directory contains all the source and the build scripts.

The project uses three main build technologies:

  • Gradle is the primary build tool for the Egeria repositories.
  • Apache Maven is an alternative build tool to Gradle and is being phased out.
  • npm is used for Javascript repositories associated with the User Interfaces.

The build scripts that use these technologies ensure the software is built in the correct order.

Building with Gradle

The Gradle processing works through the project modules. Each module has a build.gradle file that defines the artifact, its dependencies and any special processing that the module builds. The top-level build.gradle file at the root of the repository's source code directory structure controls the overall process. It runs the build in parallel threads to speed up the process of the build, but may take many cycles of your machine's capacity. When it is running, it may be a good time for a break!

Maven repositories

This processing includes locating and downloading external libraries and dependencies, typically from an online open source repository called Maven Central and our snapshot repository on https://oss.sonatype.org, so make sure you are online when you run the build.

No gradle installation is required, as we use the 'gradle wrapper' which will automatically install gradle if needed. This reduces the setup steps, and ensure everyone runs the same version of gradle.

This is a regular incremental build, but will also run all tests and generate javadoc. 

./gradlew build

The quick build skips generation of javadoc, and tests 

./gradlew build -x test -x javadoc

We avoid any use of cache, and ensure a full clean build. This may be needed when you want to recheck something that has no changed sources, but needs a rebuild -- for example to review compiler warning messages (not errors) 

./gradlew clean build --no-build-cache

This build option creates an OMAG Server Platform where the registered services are optional. The OMAG Server Platform loads the registered services it finds on the loader path specified with the -Dloader.path={directoryName} option of its startup command. Use this option if you want to remove the registered services that you are not using, or you would like introduce your own registered services.

./gradlew -PadminChassisOnly build

The build will typically take from seconds to 10 minutes depending on the speed of your machine and the number of projects that need to be built.

BUILD SUCCESSFUL in 4m 51s
3290 actionable tasks: 3172 executed, 118 up-to-date
Gradle development

For egeria Gradle is a replacement build tool to Maven and offers:

  • better support for parallel builds
  • more flexibility for build tasks
  • breaking the link between directory structure and maven artifacts
  • extremely fast incremental builds

As of version 4, Egeria can only be built using gradle.

Building with Maven

If building a version of Egeria prior to version 4, the maven instructions can be found below:

Prior to V4.0 Maven is used to build the following repositories:

  • egeria.git - main Egeria libraries.
  • egeria-samples.git - coded samples of using Egeria.
  • egeria-dev-projects.git - utilities and connectors for developers to use and develop further.

The Maven processing organizes the modules into a hierarchy. Each module has a pom.xml file (called the pom file) that defines the artifact, its parent / children, dependencies and any special processing that the module builds. The top-level pom file is the pom.xml file at the root of the repository's source code directory structure.

When the Maven command is run, it passes through the hierarchy of modules multiple times. Each pass processes a particular lifecycle phase of the build (to ensure, for example, Java source files are compiled before the resulting object files are packaged into a jar file).

Maven repositories

This processing includes locating and downloading external libraries and dependencies, typically from an online open source repository called Maven Central. The directory where these external dependencies is stored locally is called .m2.

Installing Maven

Installing Maven

Apache Maven is a build tool at is being phased out in the Egeria project, but is still required by some repositories and the Egeria developer dojo. It is capable of code compilation, running unit tests, validating dependencies and Javadoc as well as build our distribution archive.

Where it is used, Egeria requires Maven 3.5 or higher. 3.6.x or above is recommended.

Check if Maven is installed

mvn --version

Maven can be installed by downloading the software from the Apache maven website and unpacking it into a directory that is included in your PATH. Alternatively these methods are available:

Install Maven through HomeBrew 

brew install maven

Install through yum

yum install maven

Install through apt-get

apt-get install maven

On Windows, you should use Windows Subsystem for Linux Version 2 or above, install an appropriate Linux distribution, and follow the instructions for that Linux distribution.

!! cli "Rebuild a module with Maven" From the module's directory issue command: 

mvn clean install

The egeria.git repository has a top-level pom file so all of the modules can be built using one mvn clean install command from the top-level egeria directory. There is also a quick build option for people just wishing to use Egeria rather than make changes - enter mvn clean install -P quick -D skipFVT

The egeria-samples.git repository does not have a top-level pom file. Each sample is built separately. When you want to build a sample, change to the sample's directory where the pom.xml file is located and issue mvn clean install.

The egeria-dev-projects.git repository has a top-level pom fileo all of the modules can be built using one mvn clean install command from the top-level egeria-dev-projects directory.

The build can take 15 minutes to over an hour depending on the repository and on the speed/load on your machine. However eventually you will see the message:

[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 54:54 min
[INFO] Finished at: 2020-01-29T09:33:17Z
[INFO] Final Memory: 171M/3510M
[INFO] ------------------------------------------------------------------------

Process finished with exit code 0

Raise an issue or comment below