Definitions are statically-defined nodes in the JSON schema. They are suited to
for constraining string values and enumerations and for defining
They are implemented in
Add a definition¶
Adding a new definition requires just a few steps:
- Decide on a schema id. It must be unique within the SD2 namespace, between 6 and 32 characters long, contain only
[a-z0-9_], and be descriptive of the schema contents.
- Create a file in
datacatalog/definitions/jsondocsnamed after the schema id you have chosed (e.g. <id>.json)
- Write your schema in JSON schema Draft 7 (with 7 being preferable).
- Optional: Include examples of valid values for the schema in the
examplesarray in the schema file
- Validate your schema using the NewtonSoft JSON Schema Validator or equivalent
- Ensure your schema builds and does not disrupt building of other schemas. Use Makefile target
make schemas-testfor this.
- Build a persistent instance of your new schema. Use``make schemas`` for this. Check that a schema with the appropriate name was generated and is not empty.
- Validate all built schemas. This is especially important if you have referenced other schemas from within the one you are building. Use
make schemas-validatefor this.
- When all is working according to plan, open a pull request to the
python-datacatalogrepository indicating what you have added.
Below is an example of using that Makefile target. If you see both
SCHEMA PACKAGE and
SCHEMA for definitions, it was likely successful.
$ make schemas-test LOCALONLY=1 MAKETESTS=1 python -m scripts.build_schemas ... SCHEMA PACKAGE definitions SCHEMA: definitions ...
Update a definition¶
The workflow for updating a definition is nearly identical, save that you do not need to create a new JSON file. Be very careful in updating existing definitions (especially their validations and enumerations) as this can break pipeline and data management components that depend on the project schema if the PR including your change makes it into production.
Measure twice. Confirm with someone else what you’re going to cut. Then, cut once