Configuring the OMAG Server Platform¶

The OMAG Server Platform is a JVM process that includes a Tomcat web server and uses Spring Boot to support its REST APIs. The Egeria code is in the centre of this. Through configuration, Egeria starts up plug-in components called connectors and coordinates calls to them to manage and exchange metadata.

High-level structure of the OMAG Server Platform

The behaviour of the OMAG Server Platform components is influenced by:

• The command line properties passed when the platform is started.
• The contents of the application.properties file (or their environment variable equivalents).
• The contents of the classpath. This includes the Java libraries that provide the function of the OMAG Server Platform as well as a special file called logback.xml used to control the logging output of the platform.

Configuration choices for the OMAG Server Platform

Command line options¶

The command to start the OMAG Server Platform follows this form:

$ java <command line options> -jar omag-server-platform-{release}.jar

The working directory is displayed in one of the messages emitted by the OMAG Server Platform when it starts up. You can also see that it is listening on port 9443.

%  Project Egeria - Open Metadata and Governance
    ____   __  ___ ___    ______   _____                                 ____   _         _     ___
   / __ \ /  |/  //   |  / ____/  / ___/ ___   ____ _   __ ___   ____   / _  \ / / __    / /  / _ /__   ____ _  _
  / / / // /|_/ // /| | / / __    \__ \ / _ \ / __/| | / // _ \ / __/  / /_/ // //   |  / _\ / /_ /  | /  _// || |
 / /_/ // /  / // ___ |/ /_/ /   ___/ //  __// /   | |/ //  __// /    /  __ // // /  \ / /_ /  _// / // /  / / / /
 \____//_/  /_//_/  |_|\____/   /____/ \___//_/    |___/ \___//_/    /_/    /_/ \__/\//___//_/   \__//_/  /_/ /_/

 :: Powered by Spring Boot (v3.0.6) ::

2023-06-13T11:53:54.571+01:00  INFO 55711 --- [           main] o.o.o.s.springboot.OMAGServerPlatform    : Starting OMAGServerPlatform using Java 17.0.6 with PID 55711 (/egeria/deployments/platform/omag-server-platform-4.2.jar started by adminUser in /egeria/deployments)
2023-06-13T11:53:54.573+01:00  INFO 55711 --- [           main] o.o.o.s.springboot.OMAGServerPlatform    : No active profile set, falling back to 1 default profile: "default"
2023-06-13T11:53:58.709+01:00  INFO 55711 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat initialized with port(s): 9443 (https)
2023-06-13T11:54:00.187+01:00  INFO 55711 --- [           main] o.o.o.s.springboot.OMAGServerPlatform    : Working directory is: /egeria/deployments

• -Dloader.path is set to a list of directories whether the java libraries for the class path are located. These can either be absolute path names or paths relative to the working directory. Each name is separated by your operating system's file path separator (; on Windows and : on *nix).
• -Dserver.port is set to the port that the platform should listen on.
• -Dspring.config.location is set to the location of the application.properties file. Again this location can either be an absolute path name or a path name relative to the working directory.

For example, the command below starts the 4.2 version of the platform listening on port 9444 with the application.properties and lib directory in a subdirectory called platform that is under the working directory.

$ java -Dloader.path=platform/lib -Dserver.port=9444 -Dspring.config.location=platform/application.properties -jar omag-server-platform-4.2.jar

The application.properties file¶

The application.properties file may include properties that configure Tomcat, Spring or Egeria. The sections that follow are not an exhaustive list but cover the commonly used values.

Egeria's platform connectors¶

The implementation of the two platform connectors can be specified in the application.properties file:

• The configuration document store connector manages the storage and retrieval of configuration documents - one for each OMAG server.
• The platform metadata security connector authorizes calls to the Administration Services that maintain the configuration documents as well as the Platform Services that control the platform when it is running.

If these connectors are specified in the application.properties file, they are activated as part of the platform start up - assuming the JAR file(s) containing the implementation of the specified connectors is in the classpath. If these connectors are not specified in the application.properties file then each configuration document is stored in its own encrypted file and there is no platform security authorization. This default may be sufficient for a development environment. For production the platform should be configured with a platform metadata security connector because this ensures the management of OMAG Server is performed by authorized users.

The platform connectors running in the OMAG Server Platform

Configuration document store¶

The configuration document is the place where the configuration for a single OMAG Server is stored. This may include security certificates and passwords. By default, the configuration document is stored in its own encrypted file under the working directory of the OMAG Server Platform, named:

./data/servers/{serverName}/config/{serverName}.config

• Move the location of the configuration documents.
• Change the storage of the configuration documents to clear text.
• Write you own alternative store for the configuration documents.

All of these options are possible because the configuration document store is implemented in a configuration document store connector. It is therefore possible to change the implementation or behavior of this connector with a simple configuration change to the OMAG Server Platform.

From V4.2, the configuration document store connector is typically configured in the application.properties. Alternatively, the configuration document store connector can be dynamically configured using the following Administration Services.

A configuration document store connector typically needs two pieces of information:

• The class name of the Connector Provider.
• The location where the configuration documents are to be stored and retrieved from.

These two pieces of information can be provided through these two properties:

• platform.configstore.provider - class name of the configuration document store provider.
• platform.configstore.endpoint - name/pattern of resource, eg file name, needed by the connector.

The default configuration document store connector is the encrypted file store connector. If you wanted your configuration documents to be stored in clear text, set the following properties in the application.properties file:

platform.configstore.provider=org.odpi.openmetadata.adapters.adminservices.configurationstore.file.FileBasedServerConfigStoreProvider
platform.configstore.endpoint=./data/servers/{0}/config/{0}.config

The connector will substitute the name of the server for {0}.

Configuring the default configuration document¶

Configuration Documents describe the properties of an OMAG Server. They can be built up through a series of REST API calls to the platform's Administration Services. It is possible to set up a default set of properties that will be added to any new configuration documents that are created on the platform using the platform.default.config.document property in the application.properties file. The default properties are specified as a JSON document. Here is an example that set the default values for the organizationName, maxPageSize and eventBusConfig:

###############################################
### Set up the default configuration document for any new OMAG Server configurations
###############################################
platform.default.config.document=\
  {\
    "class": "OMAGServerConfig",\
    "organizationName": "myOrg",\
    "maxPageSize": 600,\
    "eventBusConfig": \
        {\
            "class": "EventBusConfig",\
            "topicURLRoot": "egeria.omag",\
            "configurationProperties":\
             {\
                "producer": {"bootstrap.servers": "localhost:9092"},\
                "consumer": {"bootstrap.servers": "localhost:9092"}\
             }\
        }\
  }

###############################################
### Set up the default configuration document for any new OMAG Server configurations
###############################################
platform.default.config.document=${DEFAULT_SERVER_CONFIG}

Configuring the platform metadata security connector¶

The OMAG Server Platform provides configuration, operational and diagnostic services for OMAG Servers which in themselves provide access to a wide variety of information and control points. Therefore, it is necessary to provide authorization services relating to the dynamic management of the platform.

Egeria provides a platform security authorization capability. It is implemented in a platform metadata security connector that is called whenever requests are made to the server platform services.

A platform metadata security connector typically needs the following pieces of information:

• platform.security.provider - class name of the platform metadata security provider
• platform.security.name - name (or platform URL root) for platform to use in security error messages.

For example, to configure tha Coco Pharmaceuticals platform metadata security connector set up the following properties:

platform.security.provider=org.odpi.openmetadata.metadatasecurity.samples.CocoPharmaPlatformSecurityProvider
platform.security.name=Coco Pharmaceuticals Platform

Auto-starting servers¶

The Platform Services are used to dynamically start and stop OMAG Servers on the platform. They also provide query APIs that return details of the OMAG Servers that are running - or have ever run - on the platform.

When the OMAG Servers start up, their associated configuration document defines the open metadata and governance services to run and the connectors that these services are to use. The classpath needs to include all the Java libraries for both the Egeria services and the connectors that are configured for each OMAG Server.

OMAG Servers running open metadata and governance services along with configured connectors

OMAG Servers can be automatically activated when an OMAG Server Platform starts up by setting the startup.server.list property in the application.properties file. The server names are listed without quotes. For example:

startup.server.list=cocoMDS1, cocoMDS2

Each server is started with the administration user id set in the property startup.user.

For example:

startup.user=garygeeke

By default, this user id is set to system.

When the platform shuts down, if any of the servers that were in the startup list are still running, they will be shutdown before the platform exits.

Transport Layer Security (TLS)¶

Transport layer security describes the security applied to API calls made between servers. The most commonly known transport layer security is SSL.

Egeria uses TLS with certificates to secure the communication to and from the OMAG Server Platforms.

Brief background on TLS¶

Transport Layer Security (TLS) protects communications over network connections through encryption, authentication and integrity. It is just one layer of security of many.

One-way SSL exchange¶

One-way SSL exchange is typically in use when browsing the web - since as a user you are most concerned that the server you are connecting to is authentic.

With this approach, the server is not able to guarantee your authenticity at the transport level. This means you can be assured of the identity of the server, but it cannot be sure of who you are.

Two-way (mutual) SSL exchange¶

With two-way (mutual) SSL exchange, trust is established both ways. This mutual authentication is more typical when interconnecting different systems and applications which are known in advance.

Certificates for the OMAG Server Platform¶

An instance of the OMAG Server Platform services requests over a REST based API from other OMAG Server Platforms, UIs, tools and engines. In this regard its role in SSL network requests described above is that of a network server, with its callers performing the role of a network client.

However, since the OMAG Server Platform also makes requests to other systems (including other OMAG Server Platforms and Apache Kafka) it is also fulfilling a network client role.

As a Spring application, the OMAG Server Platform's configuration for its network server role allows the following Spring properties to be set:

For further details on these and other less common configuration options, refer to the Spring Docs.

Since the OMAG Server Platform is also a network client the settings in the next section for clients are also required.

Egeria Java clients¶

Standard Java properties need to be set within the JVM running the Egeria client code (this includes the OMAG Server Platform):

Note that in the case of Java clients, these are system properties, and do not use Spring conventions.

Other clients¶

Similar principles to those documented for Java should apply. If you need further assistance, please contact the team on Slack . A pull request (or issue) with contributed documentation is also very welcome!

Example to launch Egeria¶

Examples certificates are provided in the codebase under open-metadata-resources/open-metadata-deployment/certificates

As an example of running the Egeria OMAG Server Platform with the certificates generated above, add the following options when launching the omag-server-platform.jar jar file:

-Dserver.ssl.key-store=${KS}
-Dserver.ssl.key-alias=EgeriaServerChassis
-Dserver.ssl.key-store-password=egeria
-Dserver.ssl.trust-store=truststore.p12
-Dserver.ssl.trust-store-password=egeria
-Djavax.net.ssl.keyStore=EgeriaServerChassis
-Djavax.net.ssl.keyStorePassword=egeria
-Djavax.net.ssl.trustStore=truststore.p12
-Djavax.net.ssl.trustStorePassword=egeria

Creating your own certificates¶

The example certificates are fine for development; however, it is important to have your own certificates for a production environment.

An example script (MacOS/Linux) to create certificates is provided in gensamplecerts.sh . It is intended only as an example. It requires the openssl tool and keytool. Deployment frameworks in cloud services may also offer support to generate certificates, and it is likely an enterprise process will be in place in larger organizations.

The script creates a Certificate Authority and then specific certificates for different Egeria components. It could be extended to create certificates for other clients especially if using 2 way SSL.

When the script is run it also makes use of the configuration template openssl.cnf .

Together, both set some important characteristics that are needed to allow the certificate to work properly, especially with current browsers:

• ensuring basicConstraints are specified
• ensuring the certificate expiry time is not too far in the future
• ensuring subjectAltName is specified.

Logging¶

The application.properties file controls which types of developer/debug logging should be produced by the OMAG Server Platform. The Egeria code (and most third party technology integrated into it) uses SLF4J (or its predecessor Log4J which can be routed to SLF4J) to write log entries that explain the code pathways that are running and any errors encountered. This can be extremely useful when debugging a set up or code issue. However, it is very expensive in terms of performance and should be used when needed.

The logging.level.xxx properties in the application.properties file define which Java classes should produce log entries and the level of severity that should be included (TRACE, DEBUG, INFO, WARN and ERROR).

These are logging values included in the default application.properties file. The logging.level.root provides the default logging level.

logging.level.root=OFF
logging.level.org.springframework=ERROR
logging.level.org.springframework.boot.web.embedded.tomcat=INFO
logging.level.org.odpi.openmetadata.serverchassis.springboot=INFO

"org.odpi.openmetadata.frameworks.auditlog.{serverName}.{componentName}

This means, for example, if you wanted all Exception or Error audit log records from any server to be produced as SLF4J log entries add the following in your application.properties file.

logging.level.org.odpi.openmetadata.frameworks.auditlog=ERROR

In addition, if you wanted all audit log messages from the cocoMDS1 server to be produced, then add the following property.

logging.level.org.odpi.openmetadata.frameworks.auditlog.cocoMDS1=INFO

Note that the severities of audit log records processed by the SLF4J audit log destination connector can be filtered by changing the configuration for this connector.

The logback.xml file is used to control where this logging is routed to.

(1) Audit log record are produced by all components supporting an OMAG Server. (2) These components also produce debug logging using SLF4J (3) Audit log record are routed to the audit log destinations described in the OMAG Server's configuration document (4) The SLF4J audit log destination routes audit log records to SLF4J (5) The logging level properties in application.properties define which log entries are produced by SLF4J. The logback.xml file defined where these log entries are stored.

Swagger¶

Swagger provides an interactive webpage that describes the REST APIs supported by the OMAG Server Platform. This is very useful in a development environment, but may not be desired in a production environment. The properties in the application.properties file control whether the swagger endpoint and web page is enabled.

These are the default values for the OMAG Server Platform.

springdoc.version='@springdoc.version@'
springdoc.api-docs.enabled=true
springdoc.api-docs.path=/v3/api-docs
springdoc.swagger-ui.path=/swagger-ui.html
springdoc.swagger-ui.displayRequestDuration=true
springdoc.swagger-ui.tagsSorter=alpha
springdoc.swagger-ui.operationsSorter=alpha
springdoc.swagger-ui.docExpansion=none

The classpath¶

As mentioned above, the classpath is the list of folders where the Java Virtual Machine (JVM) is to look for JAR files. The JAR files contain the java classes (implementation) of Egeria's modules and the connectors it is running. By default, the OMAG Server Platform is built as a single JAR file that contains all the Egeria modules. The lib directory contains all the connector implementations located in the egeria.git repository. You need to add the JAR files of any additional connectors that you wish to run. You can either add them to the lib directory or create a new directory called extralibs and add your additional JARs there. This separation means that it is clear which Jar files have come from Egeria's build and which are sourced from elsewhere.

If you choose the extralibs approach, then remember to update your -Dloader.path command line option to include the extralibs directory.

For example:

-Dloader.path=platform/lib:platform/extralibs

There is also a build option called -PadminChassisOnly to create a skinny OMAG Server Platform. This does not include any of the Registered Services nor connector implementations.

The JAR files for the registered services and connectors that you want to use should be copied from the Egeria build into a directory on the classpath (eg libor extralibs). These registered services will be dynamically loaded when the OMAG Server Platform starts. The connectors will be loaded when their configuration is activated either by the OMAG Server Platform start up or the actions of an OMAG Server.

The logback.xml file¶

The logback.xml is also located in one of the directories on the classpath. This is used to control the output of developer/debug logging produced by the software running in the OMAG Server Platform through the SLF4J/logback service.

The sample logback.xml file in the egeria.git repository shows how to redirect the logging output of the OMAG Server Platform to a directory where the recent log entries are in a clear text file called debug.log. After a time (or if the debug.log file gets to big) the older log entries are archived and TARed to new files identifying their time window.

It also redirects audit log messages for a server called cocoMDS1 that are being routed to SLF4J through the SLF4J audit log destination connector. These log messages are initially stored in clear text, and rolled up into TARed archive files just like debug.log.

The sample logback.xml is not activated in the OMAG Server Platform by default. It is provided as a sample for you to customise.

Inside the logback.xml file¶

When you look in the logback.xml file, you will see three types of entries:

• Appenders - define where log entries are to be rooted to
• Loggers - define which log entries are to be routed to each appender
• Root - defines the default routing to the appenders

There are separate appenders for standard out (console), the audit log entries from cocoMDS1 and the debug log fro many component running in the OMAG Server Platform.

What next?¶

Raise an issue or comment below