datacatalog.stores package¶

Submodules¶

datacatalog.stores.agave module¶

class datacatalog.stores.agave.StorageSystem(name, local=True, agave=None)[source]¶

Bases: str

Abstract representation of an Agave API storage system

abaco_dir¶

agave_dir(path)[source]¶

classmethod agave_path_uri(path)[source]¶

description¶

get_system_record(permissive=False)[source]¶

home_dir¶

hpc_dir¶

jupyter_dir¶

classmethod jupyter_path_uri(path)[source]¶

name¶

owner¶

page_size¶

root_dir¶

set_type_and_name(permissive=False)[source]¶: Uses regular expressions to find type and short name for system

short_name¶

ssh_host¶

system_id¶

type¶

work_dir¶

datacatalog.stores.agave.abspath(self, filepath, storage_system=None, agave=None, validate=False)[source]¶: Resolve absolute path on host filesystem for an Agave path

datacatalog.stores.exceptions module¶

exception datacatalog.stores.exceptions.ManagedStoreError[source]¶: Bases: Exception

datacatalog.stores.level module¶

class datacatalog.stores.level.Level[source]¶

Bases: str

Representation of a data processing level

MEMBERS = [('0', 'Level 0 data'), ('1', 'Level 1 data'), ('2', 'Level 2 data'), ('3', 'Level 3 data'), ('Reference', 'Reference data'), ('User', 'Managed user data')]¶

STORAGE_SYSTEM = 'data-sd2e-community'¶

datacatalog.stores.managed module¶

class datacatalog.stores.managed.ManagedStore(prefix=None, level='User', storage_system='data-sd2e-community', agave=None, **kwargs)[source]¶

Bases: datacatalog.extensible.ExtensibleAttrDict

Representation of a Data Catalog managed store

BLACKLIST = ['/usr', '/bin', '/var', '/tmp', '/root', '/lib']¶

class datacatalog.stores.managed.ManagedStores[source]¶

Bases: object

Representation of all Data Catalog managed stores

STORES = [{'level': '0', 'prefix': '/uploads', 'storage_system': 'data-sd2e-community', 'manager': 'sd2eadm', 'database': 'catalog'}, {'level': '1', 'prefix': '/products', 'storage_system': 'data-sd2e-community', 'manager': 'sd2eadm', 'database': 'catalog'}, {'level': '2', 'prefix': '/products', 'storage_system': 'data-sd2e-community', 'manager': 'sd2eadm', 'database': 'catalog'}, {'level': '3', 'prefix': '/products', 'storage_system': 'data-sd2e-community', 'manager': 'sd2eadm', 'database': 'catalog'}, {'level': 'Reference', 'prefix': '/reference', 'storage_system': 'data-sd2e-community', 'manager': 'sd2eadm', 'database': 'catalog'}, {'level': 'User', 'prefix': '/sample', 'storage_system': 'data-sd2e-community', 'manager': 'sd2eadm', 'database': 'catalog'}]¶

classmethod levels_for_filepath(filepath)[source]¶

classmethod levels_for_prefix(prefix)[source]¶

classmethod prefixes_for_level(level)[source]¶

classmethod store_for_level(level)[source]¶

classmethod store_for_prefix(prefix)[source]¶

classmethod stores()[source]¶

classmethod stores_for_filepath(filepath)[source]¶

datacatalog.stores.managed.abspath(filepath, validate=False)[source]¶: Absolute path on host filesystem

datacatalog.stores.pathmapping module¶

class datacatalog.stores.pathmapping.PathMapping(system_id, path_mapping, agave=None)[source]¶

Bases: datacatalog.extensible.ExtensibleAttrDict

Representation of a managed store with levels

agave_path¶: Agave absolute path

agave_path_uri¶: Agave files URI

hpc_path¶: Path on TACC CLI systems

hpc_path_uri(username='<username>')[source]¶

SFTP URI

Implements RFC-3986 (expired) encoding of an SFTP URI

jupyter_path¶: Path within Jupyter notebooks

jupyterhub_path¶: User-agnostic path in JupyterHub application

jupyterhub_path_uri¶: JupyterHub application URI

class datacatalog.stores.pathmapping.PathMappings[source]¶