datacatalog.linkedstores.pipeline package¶
Submodules¶
datacatalog.linkedstores.pipeline.exceptions module¶
-
exception
datacatalog.linkedstores.pipeline.exceptions.
DuplicatePipelineError
[source]¶ Bases:
datacatalog.linkedstores.basestore.exceptions.CatalogError
-
exception
datacatalog.linkedstores.pipeline.exceptions.
PipelineCreateFailure
[source]¶ Bases:
datacatalog.linkedstores.basestore.exceptions.CatalogError
-
exception
datacatalog.linkedstores.pipeline.exceptions.
PipelineUpdateFailure
[source]¶ Bases:
datacatalog.linkedstores.basestore.exceptions.CatalogError
datacatalog.linkedstores.pipeline.schema module¶
-
class
datacatalog.linkedstores.pipeline.schema.
PipelineDocument
(inheritance=True, **kwargs)[source]¶ Bases:
datacatalog.linkedstores.basestore.heritableschema.HeritableDocumentSchema
Defines a compute or ETL pipeline
datacatalog.linkedstores.pipeline.schemas module¶
datacatalog.linkedstores.pipeline.serializer module¶
-
datacatalog.linkedstores.pipeline.serializer.
INDENT
= 0¶ Default indentation for JSON rendering
-
datacatalog.linkedstores.pipeline.serializer.
SEPARATORS
= (',', ':')¶ Default tuple of JSON separators
-
datacatalog.linkedstores.pipeline.serializer.
SORT_KEYS
= True¶ Default behavior for sorting keys when rendering to JSON
-
class
datacatalog.linkedstores.pipeline.serializer.
SerializedPipeline
(component_list)[source]¶ Bases:
object
Helper class for serializing a pipeline definition
Parameters: list – pipeline components expressed as dict objects -
COMPONENT_KEYS
= {'abaco_actor': (('id', True), ('image', True), ('options', False), ('uuid', False)), 'agave_app': (('id', True), ('modules', False), ('inputs', True), ('outputs', False), ('parameters', True), ('uuid', False)), 'agave_job': (('appId', True), ('id', True), ('uuid', False)), 'deployed_container': (('image', True), ('hostname', True), ('hash', False), ('options', False), ('uuid', False)), 'web_service': (('uri', True), ('identifier', False), ('options', False), ('uuid', False))}¶ Fields to include in component definitions. Key is named after jsonschema. Tuple is (field, required)
-
datacatalog.linkedstores.pipeline.store module¶
-
class
datacatalog.linkedstores.pipeline.store.
PipelineStore
(mongodb, config={}, session=None, **kwargs)[source]¶ Bases:
datacatalog.linkedstores.basestore.softdelete.SoftDelete
,datacatalog.linkedstores.basestore.store.LinkedStore
Manage storage and retrieval of PipelineDocuments
-
LINK_FIELDS
= ['child_of', 'derived_from']¶
-
-
class
datacatalog.linkedstores.pipeline.store.
StoreInterface
(mongodb, config={}, session=None, **kwargs)[source]¶ Bases:
datacatalog.linkedstores.pipeline.store.PipelineStore