Provided Methods

Method documentation

All the examples expect that there is already a OneDataApi object initialized and assigned to a variable called "onedata_api".

### Creation of the ONE DATA Api ###
onedata_api = OneDataApi(base_url=base_url, username=user,
                         password=pw, verify=False, timeout=999)

createVirtual

Description

Creates a virtual data table with the given name from a data table with the given ID.

Optional request options (request_transformers, response_transformers, timeout, verify, sleep_after_response_millis, deserializer) can be specified.

Returns: DataTable

Parameters

PropertyTypeRequiredDefaultDescription
vdt_source_idstrtrueID of the source data table
vdt_namestrtrueName of the newly created virtual data table
target_projectstrtrueThe target project for the newly created virtual data table
descriptionstrfalseNoneDescription for the new virtual data table
notesstrfalseNoneNotes for the new virtual data table
tags[str]falseNoneTags for the new virtual data table

Usage

### Create new virtual data table ###
vdt: DataTable = onedata_api.datatables.create_virtual(vdt_source_id="87ee58aa-1d56-4ba9-b940-0a87762b4a27",
                                                       vdt_name="New Data Table",
                                                       target_project="94cc58bb-1d78-4ba9-s244-0a76562b4b78")

updateVirtual

Description

Updates the virtual data table with the given id XOR name and project id. The vdt can either be updated by specifying the data source configuration or the sql and the id of the underlying data table.

Optional request options (request_transformers, response_transformers, timeout, verify, sleep_after_response_millis, deserializer) can be specified.

Returns: DataTable

Parameters

PropertyTypeRequiredDefaultDescription
vdt_source_idstrtrueID of the source data table
vdt_namestrtrueName of the newly created virtual data table
target_projectstrtrueThe target project for the newly created virtual data table
descriptionstrfalseNoneDescription for the new virtual data table
notesstrfalseNoneNotes for the new virtual data table
tags[str]falseNoneTags for the new virtual data table

Usage

### Update virtual data table ###

update_config: dict = {
    "origin": "datatable",
    "config": {
        "schema": "table",
        "dataId": "b06349ee-3169-4eee-901f-fcdaa12b9b35",
        "dataOptions": {
            'sql': 'Select * FROM inputTable'
        }
    }
}

vdt: DataTable = onedata_api.datatables.update_virtual(vdt_id="cd21e90f-d4bd-4462-9197-67e7526f6bdf",
                                                       datasource_configuration=update_config,
                                                       execution_target_project_id="ee548bc5-5853-4b32-90ba-5bf4f34e149f")

createVirtualWithConfiguration

Description

Creates a virtual data table with the specified name from the specified source id. The vdt can either be created by specifying the data source configuration or the sql and the id of the underlying data table.

Optional request options (request_transformers, response_transformers, timeout, verify, sleep_after_response_millis, deserializer) can be specified.

Returns: DataTable

Parameters

PropertyTypeRequiredDefaultDescription
vdt_source_idUnion[str, uuid.UUID]trueID of the source data table
vdt_namestrtrueName of the newly created virtual data table
target_projectUnion[str, uuid.UUID]trueThe ID of the project the virtual data table should be created in
datasource_configurationdictfalseNoneThe new datasource configuration of the virtual data table
sqlstrfalseNoneThe sql that should be applied to the source table
descriptionstrfalseNoneDescription for the new virtual data table
notesstrfalseNoneNotes for the new virtual data table
tags[str]falseNoneTags for the new virtual data table
execution_target_project_idUnion[str, uuid.UUID]falseNoneThe ID of the target project which affects the selection of the Spark Execution Context.

Usage

### Create virtual data table with configuration ###

update_config: dict = {
    "origin": "datatable",
    "config": {
        "schema": "table",
        "dataId": "b06349ee-3169-4eee-901f-fcdaa12b9b35",
        "dataOptions": {
            'sql': 'Select * FROM inputTable'
        }
    }
}

vdt: DataTable = onedata_api.datatables.create_virtual_with_configuration(vdt_source_id="b06349ee-3169-4eee-901f-fcdaa12b9b35",
                                                                          vdt_name=f"Test-{uuid4()}",
                                                                          description="VDT by SDK",
                                                                          tags=["Guten", "Tag"], notes="Gute Noten,
                                                                          target_project_id=project_id,
                                                                          datasource_configuration=update_config)