Conformance Test Suite Chart (egeria-cts)¶
This is a deployment of Egeria that will automatically run the Conformance Test Suite's repository workbench against a deployed metadata repository.
Prerequisites
In order to use the chart, you'll first need to have the following installed:
- A Kubernetes cluster at 1.15 or above
- the
kubectltool in your path - Helm 3.0 or above
No configuration of the chart is required to use defaults, but information is provided below.
Installation¶
Install (deploy) the CTS chart
helm repo add egeria https://odpi.github.io/egeria-charts
helm repo update
helm install [-f overrides.yaml] <name> egeria/egeria-cts
The -f overrides.yaml is optional, and only necessary if you are overriding any of the configuration (see options below), while the <name> is the name you want to give your deployment.
The installation may take a minute or so
This is because it is not only creating the required objects in Kubernetes to run the platforms, but also is configuring Egeria itself - which involves waiting for everything to startup before configuring Egeria via REST API calls.
Configuration¶
In a helm chart the configuration that has been externalized by the chart writer is specified in the values.yaml file which you can find in this directory. However, rather than edit this file directly, it's recommended you create an additional file with the required overrides (for example, called overrides.yaml).
The primary values you will likely want to override in this chart are as follows:
Technology under test¶
The technology under test ("tut") defines the repository that you want to run the CTS against. By default, the chart will run the CTS against the built-in graph repository. To configure it to test some other repository, you will need to override one or more of the following:
| Parameter | Description |
|---|---|
tut.serverType |
Defines the type of the repository to be tested: native for a built-in repository of Egeria core (like the in-memory repository), plugin for a pluggable native repository (like XTDB), or proxy for a third party repository technology (like Apache Atlas). |
tut.connectorProvider |
Should be the canonical class name of the connector provider class for the repository connector to use. |
tut.connectorConfig |
An optional structure defining any additional configuration for the connector. This should be provided as YAML, and will be automatically translated into JSON to pass through to the configurationProperties parameter of the connector when configuring it. |
tut.serverEndpoint.host |
When using a third party technology that relies on integrating to a system outside Egeria (i.e. when using a proxy), this is the hostname of the third party technology. |
tut.serverEndpoint.port |
When using a third party technology that relies on integrating to a system outside Egeria (i.e. when using a proxy), this is the port number of the third party technology. |
tut.serverEndpoint.protocol |
When using a third party technology that relies on integrating to a system outside Egeria (i.e. when using a proxy), this is the protocol to use when accessing the third party technology (e.g. https). |
tut.serverEndpoint.username |
When using a third party technology that relies on integrating to a system outside Egeria (i.e. when using a proxy), this is the username of the system user to use when accessing the third party technology. |
tut.serverEndpoint.password |
When using a third party technology that relies on integrating to a system outside Egeria (i.e. when using a proxy), this is the password for the system user to use when accessing the third party technology. |
Example tut override for XTDB
tut:
serverType: "plugin"
connectorProvider: "org.odpi.egeria.connectors.juxt.xtdb.repositoryconnector.XtdbOMRSRepositoryConnectorProvider"
connectorConfig:
xtdbConfig:
xtdb/index-store:
kv-store:
xtdb/module: xtdb.lmdb/->kv-store
db-dir: data/servers/xtdb/lmdb-index
xtdb/document-store:
kv-store:
xtdb/module: xtdb.rocksdb/->kv-store
db-dir: data/servers/xtdb/rdb-docs
xtdb/tx-log:
kv-store:
xtdb/module: xtdb.rocksdb/->kv-store
db-dir: data/servers/xtdb/rdb-tx
xtdb.lucene/lucene-store:
db-dir: data/servers/xtdb/lucene
Connector downloads¶
In addition to the general technology under test configuration outlined above, when configuring the chart to test a non-core Egeria connector you also need to specify any dependencies that need to be downloaded to run that connector (i.e. at a minimum to make that connector's connector provider and connector itself available to the pod running the technology under test).
You do this by overriding the downloads value with a list of filename and url pairs.
Example downloads override for XTDB - for latest RELEASE of connector
downloads:
- filename: egeria-connector-xtdb-LATEST_RELEASE-jar-with-dependencies.jar
url: "http://repository.sonatype.org/service/local/artifact/maven/redirect?r=central-proxy&g=org.odpi.egeria&a=egeria-connector-xtdb&v=RELEASE&c=jar-with-dependencies"
Example downloads override for XTDB - for latest SNAPSHOT of connector
Only use this to use development level code for the connector. You will need to explicitly specify the version, in this case 3.9-SNAPSHOT.
```yaml downloads: - filename: egeria-connector-xtdb-LATEST_SNAPSHOT-jar-with-dependencies.jar url: "http://oss.sonatype.org/service/local/artifact/maven/redirect?r=snapshots&g=org.odpi.egeria&a=egeria-connector-xtdb&v=3.9-SNAPSHOT&c=jar-with-dependencies"
Why both filename and URL?
As illustrated in the examples above, some URLs may dynamically redirect and resolve to a given filename, for example to always download the latest release of a given file. Because we have minimal utilities installed in the pod to actually dynamically do these downloads, there are cases (like the example above) where the utilities are not able to automatically determine the filename to produce from such dynamic URLs.
Therefore, by specifying both the URL and the filename we can ensure that the file is downloaded and stored as an expected output (like a .jar file) to ensure it can be automatically resolved by e.g. Java's class loader.
Scale of test¶
There is a single option to configure the scale of the CTS test:
records: defines the "scale factor" for the CTS, where the number of instances that will be created (per type) is roughly 2x this number, and the maximum number of search results for the paged search tests will be limited to this number
Monitoring progress¶
You can monitor the progress of the CTS execution by looking at the log output from the initialization and reporting pods:
Get the init and report pod names
shell
kubectl get pods -l 'app.kubernetes.io/component in (init,report)'
Example output for retrieving the init pod name
NAME READY STATUS RESTARTS AGE
p320-10-init-5845c9bb79-5ddwh 1/1 Running 0 45h
t12-init-585b47f74-85r8m 1/1 Running 0 39m
If you had multiple CTS deployments running in parallel in your cluster, this may return more than one result (as in the example above). The one that starts with the name you used as the name of your deployment is the one that represents your particular deployment.
You will also see reporting pods once the tests start - and after finishing the init pod will go away.
Once you have the pod name, you can then view the log:
Review the init pod logs
kubectl logs -f <podname>
Where the <podname> is the name of the pod discovered in the command above (e.g. t12-init-585b47f74-85r8m).
Example output from the init log
This opening section simply displays environment variables that have been configured, primarily useful for debugging or other diagnostics purposes:
-- Environment variables --
...
CONNECTOR_PROVIDER=org.odpi.egeria.connectors.juxt.xtdb.repositoryconnector.XtdbOMRSRepositoryConnectorProvider
...
-- End of Environment variables --
The configuration then occurs. Of primary importance here is that all the results are 200 to indicate each operation was successful:
-- Configuring platform with required servers...
> Configuring conformance test suite driver:
{"class":"VoidResponse","relatedHTTPCode":200}
(200 - https://t12-platform:9443/open-metadata/admin-services/users/admin/servers/cts/server-url-root?url=https://t12-platform:9443)
{"class":"VoidResponse","relatedHTTPCode":200}
(200 - https://t12-platform:9443/open-metadata/admin-services/users/admin/servers/cts/server-type?typeName=Conformance)
{"class":"VoidResponse","relatedHTTPCode":200}
(200 - https://t12-platform:9443/open-metadata/admin-services/users/admin/servers/cts/event-bus?topicURLRoot=egeria)
{"class":"VoidResponse","relatedHTTPCode":200}
(200 - https://t12-platform:9443/open-metadata/admin-services/users/admin/servers/cts/cohorts/cts)
{"class":"VoidResponse","relatedHTTPCode":200}
(200 - https://t12-platform:9443/open-metadata/admin-services/users/admin/servers/cts/conformance-suite-workbenches/repository-workbench/repositories)
> Configuring technology under test:
{"class":"VoidResponse","relatedHTTPCode":200}
(200 - https://t12-platform:9443/open-metadata/admin-services/users/admin/servers/tut/server-url-root?url=https://t12-platform:9443)
{"class":"VoidResponse","relatedHTTPCode":200}
(200 - https://t12-platform:9443/open-metadata/admin-services/users/admin/servers/tut/server-type?typeName=TUT)
{"class":"VoidResponse","relatedHTTPCode":200}
(200 - https://t12-platform:9443/open-metadata/admin-services/users/admin/servers/tut/organization-name?name=Egeria)
{"class":"VoidResponse","relatedHTTPCode":200}
(200 - https://t12-platform:9443/open-metadata/admin-services/users/admin/servers/tut/event-bus?topicURLRoot=egeria)
{"class":"VoidResponse","relatedHTTPCode":200}
(200 - https://t12-platform:9443/open-metadata/admin-services/users/admin/servers/tut/local-repository/mode/plugin-repository/connection)
{"class":"VoidResponse","relatedHTTPCode":200}
(200 - https://t12-platform:9443/open-metadata/admin-services/users/admin/servers/tut/cohorts/cts)
-- End of configuration
The CTS itself is then started up, followed by the technology under test:
-- Running the conformance test suite...
> Starting conformance test suite:
{"class":"SuccessMessageResponse","relatedHTTPCode":200,"successMessage":"Fri Oct 22 14:43:54 GMT 2021 cts is running the following services: [Open Metadata Repository Services (OMRS), Connected Asset Services, Conformance Suite Services]"}
(200 - https://t12-platform:9443/open-metadata/admin-services/users/admin/servers/cts/instance)
> Starting the technology under test:
{"class":"SuccessMessageResponse","relatedHTTPCode":200,"successMessage":"Fri Oct 22 14:44:12 GMT 2021 tut is running the following services: [Open Metadata Repository Services (OMRS)]"}
(200 - https://t12-platform:9443/open-metadata/admin-services/users/admin/servers/tut/instance)
-- End of conformance test suite startup
Once the CTS is running, a busy loop is then run to wait until the CTS has completed. The status of the CTS execution will be re-checked every 30 seconds and an update printed out to the log accordingly:
-- Collecting results of the conformance test suite...
> Collecting basic configuration information...
> Waiting for the conformance test suite to complete...
... still waiting (0d:00h:01m:00s)
... still waiting (0d:00h:01m:30s)
...
Once completed, the pod will retrieve the detailed results from the CTS itself:
...
... still waiting (0d:02h:13m:30s)
> Retrieving detailed profile results...
... retrieving profile details for: Metadata sharing
... retrieving profile details for: Reference copies
... retrieving profile details for: Metadata maintenance
... retrieving profile details for: Dynamic types
... retrieving profile details for: Graph queries
... retrieving profile details for: Historical search
... retrieving profile details for: Entity proxies
... retrieving profile details for: Soft-delete and restore
... retrieving profile details for: Undo an update
... retrieving profile details for: Reidentify instance
... retrieving profile details for: Retype instance
... retrieving profile details for: Rehome instance
... retrieving profile details for: Entity search
... retrieving profile details for: Relationship search
... retrieving profile details for: Entity advanced search
... retrieving profile details for: Relationship advanced search
> Retrieving detailed test case results...
... retrieving test case details for: repository-attribute-typedef-ActivityType
...
Once all the results are collected, the pod will then bundle these all up into an archive and then print out the location of that archive within the pod:
...
... retrieving test case details for: repository-typedef-ZoneHierarchy-null
> Bundling all results into an archive...
-- End of conformance test suite results collection, download from: /tmp/t12.tar.gz
The report pod will continue running after the CTS has completed
Note that the report pod itself will continue running after the CTS has completed: this is to provide adequate time to copy the bundled archive file of the results out of the pod. (If the pod were allowed to stop its execution any files within it would be lost.)
Retrieving results¶
Once the CTS has completed running and the bundled archive file is available, it can be copied from the report pod:
Copy results archive from the pod
kubectl exec cts-report--1-d8p6k -- sh -c 'cat /export/pipe' | tar -xvf -
Where the <podname> is the name of the pod as discovered above (e.g. t12-report-585b47f74-85r8m) and the <name> is the name of the Helm deployment. (Alternatively, you could just copy the location from the log output as shown above in the monitoring section.)
This will create a local file ie: ```shell $ kubectl exec cts-report--1-d8p6k -- sh -c 'cat /export/pipe' | tar -xvf - Defaulted container "wait-for-retrieval" out of: wait-for-retrieval, wait-for-platform (init), wait-for-kafka (init), wait-for-init (init), report (init) x export/cts.tar.gztar.gz
Uninstallation¶
Once you have retrieved the results, or if you want to otherwise cancel or stop the running of the CTS:
Delete the deployment
helm delete <name>
Where <name> is the name of your deployment. (If you are unsure what name you used, helm list lists all the deployments.)
Raise an issue or comment below