Authorization

Authorization using username and password

Authorization can be accomplished by providing a username and a password to the OneDataApi constructor.

Example:

from onedata.api import OneDataApi

od_base_url = "https://internal.onedata.de"
od_username = "test@onelogic.de"
od_password = "testtesttest"

onedata_api = OneDataApi(base_url=od_base_url, username=od_username, password=od_password)

Examples with additional parameters:

from onedata.api import OneDataApi

onedata_api = OneDataApi(base_url="https://internal.onedata.de",
                         username="test@onelogic.de",
                         password="testtesttest",
                         timeout=10,  # requests will timeout after 10 seconds
                         verify=False,  # disables ssl verification, invalid/self-signed certificates will be accepted
                         sleep_after_response_millis=1000,  # performs an artificial delay after
                         request_transformers=[],  # custom request transformers, see documentation for more details
                         response_transformers=[])  # custom response transformers, see documentation for more details

Authorization using authentication token

Authorization can be accomplished by providing an authentication token to the OneDataApiconstructor.

Example:

from onedata.api import OneDataApi

od_base_url = "https://internal.onedata.de"
auth_token = "Bearer SomeRandomToken"

onedata_api = OneDataApi(base_url=od_base_url, token=auth_token)

Note: Providing the username, password and the token to the OneDataApi constructor will result in an error. This is due to the fact that it would be impossible to know which of the two authorization methods should be used.

Authorization using global variables (used in OD Python Processors)

Authorization can be accomplished by providing the request to OneDataApi.from_globals.

Example:

from onedata.api import OneDataApi

onedata_api = OneDataApi.from_globals(globals())

Authorization using request (used in OD Functions)

Authorization can be accomplished by providing the request to OneDataApi.from_request.

Example:

from onedata.api import OneDataApi

def handle(req):
    onedata_api = OneDataApi.from_request(req)

Connections

This page shows how to get information about datatables of a database connection and how to create a datatable from a database connection.


Note: All the examples expect that there is already a OneDataApi object initialized and assigned to a variable called "onedata_api"

Create datatable from database connection

from onedata.datatables.types import DataTable

connection_id: str = "5ff899b4-0915-450c-a1c3-3a7f8794dccb"
name: str = "Datatable from Connection via SDK"

dt: DataTable = onedata_api.connections.create_data_table(connection_id=connection_id,
                                                          connection_table_name="tms_test",
                                                          dt_name=name,
                                                          dt_description="desc")

Get all datatables of a database connection

from onedata.common.types import SortOrder, Paginated
from onedata.connections.types import TablesListingMode

connection_id: str = "5ff899b4-0915-450c-a1c3-3a7f8794dccb"

connections_list: Paginated = onedata_api.connections.paginated_database_table_information(connection_id=connection_id,
                                                                                           ordering=SortOrder.DESC,
                                                                                           tables_listing_mode=TablesListingMode.SCHEMA_CONFIGURED_IN_THE_CONNECTION)

Credentials

This page shows the different ways to retrieve credentials and how to use them to get authorization when communicating with ONE DATA API.

Note: All the examples expect that there is already a OneDataApi object initialized and assigned to a variable called "onedata_api".

Get credential by ID

Credentials can be retrieved by providing its ID.

from onedata.credentials.types import Credentials

technical_user_credentials_id: uuid = "30b2d7d9-4027-41ad-b5d1-1c4ba98390f9"
technical_user_credentials: Credentials = onedata_api.credentials.get(technical_user_credentials_id)

Authorization using credentials of a technical user

Authorization as a technical user can be accomplished by providing username and password from an exposed credential to the OneDataApi constructor. 

Example:

from onedata.api import OneDataApi
from onedata.credentials.types import Credentials

onedata_base_url = "https://internal.onedata.de"
technical_user_credentials_id: uuid = "30b2d7d9-4027-41ad-b5d1-1c4ba98390f9"
technical_user_credentials: Credentials = onedata_api.credentials.get(technical_user_credentials_id)

onedata_api_technical_user = OneDataApi(base_url=onedata_base_url,
                                        username=technical_user_credentials.key_information.username,
                                        password=technical_user_credentials.key_information.password)

Datatables

This page describes how to interact with ONE DATA datatables.

Note: All the examples expect that there is already a OneDataApi object initialized and assigned to a variable called "onedata_api".

Create datatable

from onedata.datatables.types import DatasetLock, DataTableStorageFormat, UploadedDataResponse
import pandas

### You can create a datatable by specifying the headers and the content

project_id = "00000000-0000-0000-0000-000000000002"

response: UploadedDataResponse = onedata_api.datatables.create(name="Datatable created from headers and content via SDK",
                                                               headers=["a", "b"],
                                                               content=[{"a": 1, "b": "asd"}] * 200, # to create 200 records in the datatable
                                                               project_id=project_id,
                                                               locked=DatasetLock.OPEN, # optional, default OPEN
                                                               storage_format=DataTableStorageFormat.PARQUET, # optional, default PARQUET
                                                               create_fallback=True, # optional, default True
                                                               description="some desc", # optional, default None
                                                               notes="notes", # optional, default None
                                                               tags=["tag1", "tag2"]) # optional, default None


### You can create a datatable by specifying a pandas DataFrame

project_id = "00000000-0000-0000-0000-000000000002"
pandas_df: pandas.DataFrame = pandas.DataFrame(data={'col1': [1, 2], 'col2': [3, 4]})

response: UploadedDataResponse = onedata_api.datatables.create_from_pandas(name="Datatable created from pandas DataFrame via SDK",
                                                                           data=pandas_df,
                                                                           project_id=project_id,
                                                                           locked=DatasetLock.OPEN, # optional, default OPEN
                                                                           storage_format=DataTableStorageFormat.PARQUET, # optional, default PARQUET
                                                                           create_fallback=True, # optional, default True
                                                                           description="some desc", # optional, default None
                                                                           notes="notes", # optional, default None
                                                                           tags=["tag1", "tag2"]) # optional, default None

Get datatable

from onedata.datatables.types import DataTable

### You can get a datatable by datatable ID

datatable_id = "00000000-0000-0000-0000-000000000004"

response: DataTable = onedata_api.datatables.get(id=datatable_id)


### You can get a datatable by name and project ID

datatable_name = "Some DT name"
project_id = "00000000-0000-0000-0000-000000000002"

response: DataTable = onedata_api.datatables.get(name=datatable_name,
                                                 project_id=project_id,
                                                 force_exact_match=True, # optional, default True
                                                 force_distinct=True) # optional, default True

Update datatable

from onedata.datatables.types import DataTable

### You can update a datatable by datatable ID

datatable_id = "00000000-0000-0000-0000-000000000004"

response: DataTable = onedata_api.datatables.update(data_id=datatable_id,
                                                    new_name="Some random new name", # optional, default None
                                                    new_tags=["new_tag"], # optional, default None
                                                    new_description="new_desc", # optional, default None
                                                    new_notes="new_notes", # optional, default None
                                                    new_attributes=None, # optional, default None
                                                    new_attributes_operator=None, # optional, default None
                                                    new_locked=None, # optional, default None
                                                    new_home_project_id=None) # optional, default None


### You can update a datatable by name and project ID

datatable_name = "Some DT name"
project_id = "00000000-0000-0000-0000-000000000002"

response: DataTable = onedata_api.datatables.update(name=datatable_name,
                                                    project_id=project_id,
                                                    force_exact_match=True, # optional, default True
                                                    force_distinct=True, # optional, default True
                                                    new_name="Some random new name", # optional, default None
                                                    new_tags=["new_tag"], # optional, default None
                                                    new_description="new_desc", # optional, default None
                                                    new_notes="new_notes", # optional, default None
                                                    new_attributes=None, # optional, default None
                                                    new_attributes_operator=None, # optional, default None
                                                    new_locked=None, # optional, default None
                                                    new_home_project_id=None) # optional, default None


### You can update a datatable by modifying a DataTable object and saving it

datatable_id = "00000000-0000-0000-0000-000000000004"
datatable: DataTable = onedata_api.datatables.get(id=datatable_id)
datatable.description = "Hey, I am the new fancy description. Set by Python SDK. COOL, hum? Hum?"

response: DataTable = onedata_api.datatables.update_by_reference(data_table=datatable)

Get datatable content

from onedata.datatables.types import DataTableContent

### You can get a datatable's content by datatable ID

datatable_id = "00000000-0000-0000-0000-000000000004"

datatable: DataTableContent = onedata_api.datatables.get_content(data_id=datatable_id,
                                                                 page=1, # optional, default 1
                                                                 pagination_size=100) # optional, default 10000


### You can get a datatable's content by FRT ID

frt_id = "00000000-0000-0000-0000-000000000005"

datatable: DataTableContent = onedata_api.datatables.get_content(frt_id=frt_id,
                                                                 page=1, # optional, default 1
                                                                 pagination_size=100) # optional, default 10000


### You can get a datatable's content by name and project ID

datatable_name = "Some DT name"
project_id = "00000000-0000-0000-0000-000000000002"

datatable: DataTableContent = onedata_api.datatables.get_content(name=datatable_name,
                                                                 project_id=project_id,
                                                                 page=1, # optional, default 1
                                                                 pagination_size=100, # optional, default 10000
                                                                 force_exact_match=True, # optional, default True
                                                                 force_distinct=True) # optional, default True

Get datatable samples

from onedata.datatables.types import DataTableContent

### You can get a datatable's samples by datatable ID

datatable_id = "00000000-0000-0000-0000-000000000004"

datatable: DataTableContent = onedata_api.datatables.get_content_samples(data_id=datatable_id,
                                                                         sample_count=50) # optional, default 20


### You can get a datatable's samples by FRT ID

frt_id = "00000000-0000-0000-0000-000000000005"

datatable: DataTableContent = onedata_api.datatables.get_content_samples(frt_id=frt_id,
                                                                         sample_count=50) # optional, default 20


### You can get a datatable's samples by name and project ID

datatable_name = "Some DT name"
project_id = "00000000-0000-0000-0000-000000000002"

datatable: DataTableContent = onedata_api.datatables.get_content_samples(name=datatable_name,
                                                                         project_id=project_id,
                                                                         sample_count=50, # optional, default 20
                                                                         force_exact_match=True, # optional, default True
                                                                         force_distinct=True) # optional, default True

Replace datatable content

### You can replace a datatable's content by datatable ID

datatable_id = "00000000-0000-0000-0000-000000000004"
new_content = [{"a": 10, "b": "replaced"}]

onedata_api.datatables.replace_content(data_id=datatable_id,
                                       new_data=new_content) # optional, default None resulting in clearing the datatable content


### You can replace a datatable's content by FRT ID

frt_id = "00000000-0000-0000-0000-000000000005"
new_content = [{"a": 10, "b": "replaced"}]

onedata_api.datatables.replace_content(frt_id=frt_id,
                                       new_data=new_content) # optional, default None resulting in clearing the datatable content


### You can replace a datatable's content by name and project ID

datatable_name = "Some DT name"
project_id = "00000000-0000-0000-0000-000000000002"

onedata_api.datatables.replace_content(name=datatable_name,
                                       project_id=project_id,
                                       new_data=new_content, # optional, default None resulting in clearing the datatable content
                                       force_exact_match=True, # optional, default True
                                       force_distinct=True) # optional, default True

Patch datatable content

### You can patch a datatable's content by datatable ID

datatable_id = "00000000-0000-0000-0000-000000000004"
deleted = ["2c790106-ea5e-4ff2-9f07-9016691242b3"]
updated = [{"a": 2, "b": "some text", "__ROW_ID__": "2c790106-ea5e-4ff2-9f07-9016691242b2"}]
added = [{"a": 1, "b": "new"}]

onedata_api.datatables.patch_content(data_id=datatable_id,
                                     deleted=deleted, # optional, default None
                                     updated=updated, # optional, default None
                                     added=added) # optional, default None


### You can patch a datatable's content by FRT ID

frt_id = "00000000-0000-0000-0000-000000000005"
deleted = ["2c790106-ea5e-4ff2-9f07-9016691242b3"]
updated = [{"a": 2, "b": "some text", "__ROW_ID__": "2c790106-ea5e-4ff2-9f07-9016691242b2"}]
added = [{"a": 1, "b": "new"}]

onedata_api.datatables.patch_content(frt_id=frt_id,
                                     deleted=deleted, # optional, default None
                                     updated=updated, # optional, default None
                                     added=added) # optional, default None


### You can patch a datatable's content by name and project ID

datatable_name = "Some DT name"
project_id = "00000000-0000-0000-0000-000000000002"
deleted = ["2c790106-ea5e-4ff2-9f07-9016691242b3"]
updated = [{"a": 2, "b": "some text", "__ROW_ID__": "2c790106-ea5e-4ff2-9f07-9016691242b2"}]
added = [{"a": 1, "b": "new"}]

onedata_api.datatables.patch_content(name=datatable_name,
                                     project_id=project_id,
                                     deleted=deleted, # optional, default None
                                     updated=updated, # optional, default None
                                     added=added, # optional, default None
                                     force_exact_match=True, # optional, default True
                                     force_distinct=True) # optional, default True

Update datatable content

### You can update a datatable's content by datatable ID

datatable_id = "00000000-0000-0000-0000-000000000004"
updated = [{"a": 2, "b": "some text", "__ROW_ID__": "2c790106-ea5e-4ff2-9f07-9016691242b2"}]

onedata_api.datatables.update_content(data_id=datatable_id,
                                      updated=updated) # optional, default None


### You can update a datatable's content by FRT ID

frt_id = "00000000-0000-0000-0000-000000000005"
updated = [{"a": 2, "b": "some text", "__ROW_ID__": "2c790106-ea5e-4ff2-9f07-9016691242b2"}]

onedata_api.datatables.update_content(frt_id=frt_id,
                                      updated=updated) # optional, default None


### You can update a datatable's content by name and project ID

datatable_name = "Some DT name"
project_id = "00000000-0000-0000-0000-000000000002"
updated = [{"a": 2, "b": "some text", "__ROW_ID__": "2c790106-ea5e-4ff2-9f07-9016691242b2"}]

onedata_api.datatables.update_content(name=datatable_name,
                                      project_id=project_id,
                                      updated=updated, # optional, default None
                                      force_exact_match=True, # optional, default True
                                      force_distinct=True) # optional, default True

Delete datatable content

### You can delete a datatable's content by datatable ID

datatable_id = "00000000-0000-0000-0000-000000000004"
deleted = ["2c790106-ea5e-4ff2-9f07-9016691242b3"]

onedata_api.datatables.delete_content(data_id=datatable_id,
                                      deleted=deleted) # optional, default None


### You can delete a datatable's content by FRT ID

frt_id = "00000000-0000-0000-0000-000000000005"
deleted = ["2c790106-ea5e-4ff2-9f07-9016691242b3"]

onedata_api.datatables.delete_content(frt_id=frt_id,
                                      deleted=deleted) # optional, default None


### You can delete a datatable's content by name and project ID

datatable_name = "Some DT name"
project_id = "00000000-0000-0000-0000-000000000002"
deleted = ["2c790106-ea5e-4ff2-9f07-9016691242b3"]

onedata_api.datatables.delete_content(name=datatable_name,
                                      project_id=project_id,
                                      deleted=deleted, # optional, default None
                                      force_exact_match=True, # optional, default True
                                      force_distinct=True) # optional, default True

Add datatable content

### You can add to a datatable's content by datatable ID

datatable_id = "00000000-0000-0000-0000-000000000004"
added = [{"a": 1, "b": "new"}]

onedata_api.datatables.add_content(data_id=datatable_id,
                                   added=added) # optional, default None


### You can add to a datatable's content by FRT ID

frt_id = "00000000-0000-0000-0000-000000000005"
added = [{"a": 1, "b": "new"}]

onedata_api.datatables.add_content(frt_id=frt_id,
                                   added=added) # optional, default None


### You can add to a datatable's content by name and project ID

datatable_name = "Some DT name"
project_id = "00000000-0000-0000-0000-000000000002"
added = [{"a": 1, "b": "new"}]

onedata_api.datatables.add_content(name=datatable_name,
                                   project_id=project_id,
                                   added=added, # optional, default None
                                   force_exact_match=True, # optional, default True
                                   force_distinct=True) # optional, default True

Functions

This page describes how to interact with ONE DATA Functions.

Note: All the examples expect that there is already a OneDataApi object initialized and assigned to a variable called "onedata_api"

Executing a Function

from onedata.functions.types import FunctionExecutionResult

### You can execute a function by its ID

function_id = "00000000-0000-0000-0000-000000000001"

result: FunctionExecutionResult = onedata_api.functions.execute(id=function_id)


### You can also execute a function by name and project ID

function_name = "Some Function name"
project_id = "00000000-0000-0000-0000-000000000002"

result: FunctionExecutionResult = onedata_api.functions.execute(name=function_name, project_id=project_id, force_exact_match=False)


### You can also execute a function with arguments

function_id = "00000000-0000-0000-0000-000000000001"

result: FunctionExecutionResult = onedata_api.functions.execute(id=function_id, arguments={"text": "I love Python"})

Production Line Jobs

This page shows how to get jobs of a production line.

Note: All the examples expect that there is already a OneDataApi object initialized and assigned to a variable called "onedata_api"

Get production line job by ID

from onedata.productionlines.jobs.types import ProductionLineJob

pl_job_id = "00000000-0000-0000-0000-000000000003"

pl_job: ProductionLineJob = onedata_api.production_lines.jobs.get(id=pl_job_id)

Get latest job of a production line

from onedata.productionlines.jobs.types import ProductionLineJob

pl_id = "00000000-0000-0000-0000-000000000003"

pl_job: ProductionLineJob = onedata_api.production_lines.jobs.get_latest(pl_id=pl_id)

Production Lines

This page shows the different ways to retrieve a single production line or a paginated list of production lines.

Note: All the examples expect that there is already a OneDataApi object initialized and assigned to a variable called "onedata_api".

Listing production lines

from onedata.productionlines.types import ProductionLine
from onedata.common.types import Paginated

paginated_production_lines: Paginated = onedata_api.production_lines.paginated()
production_line_list: [ProductionLine] = paginated_production_lines.items

Listing production lines with parameters

from onedata.productionlines.types import ProductionLine
from onedata.common.types import Paginated, ResourceSortProperty, SortOrder

page = 3
limit = 5
sort_by = ResourceSortProperty.name
ordering = SortOrder.ASC

paginated_production_lines: Paginated = onedata_api.production_lines.paginated(page=page, limit=limit, sort_by=sort_by, ordering=ordering)
production_line_list: [ProductionLine] = paginated_production_lines.items

Getting a production line by id

from uuid import UUID
from onedata.productionlines.types import ProductionLine

pl_id = "00000000-0000-0000-0000-000000000001"

pl: ProductionLine = onedata_api.production_lines.get(id=UUID(pl_id))
print(pl.name)

Getting a production line by name and project id

from uuid import UUID
from onedata.productionlines.types import ProductionLine

pl_name = "New Production Line"
project_id = "00000000-0000-0000-0000-000000000001"

pl: ProductionLine = onedata_api.production_lines.get(name=pl_name, project_id=UUID(project_id))
print(pl.id)

Executing a production line asynchronously

from onedata.productionlines.jobs.types import ProductionLineJob

### You can execute a production line by its ID

pl_id = "00000000-0000-0000-0000-000000000001"

pl_job: ProductionLineJob = onedata_api.production_lines.execute_async(id=pl_id)


### You can also specify a specific version of the production line to execute

pl_version = 20

pl_job: ProductionLineJob = onedata_api.production_lines.execute_async(id=pl_id, version=pl_version)


### You can also execute a production line by name and project ID

pl_name = "Some PL name"
project_id = "00000000-0000-0000-0000-000000000002"

pl_job: ProductionLineJob = onedata_api.production_lines.execute_async(name=pl_name, project_id=project_id, force_exact_match=True, force_distinct=True)


### You can also execute a production line with variables

pl_id = "00000000-0000-0000-0000-000000000001"

pl_job: ProductionLineJob = onedata_api.production_lines.execute_async(id=pl_id, variable_assignments=
    [{"variableType": "integer", "variableName": "myVar", "variableValue": 5}])

Executing a production line synchronously

from onedata.productionlines.jobs.types import ProductionLineJob

### You can execute a production line by its ID

pl_id = "00000000-0000-0000-0000-000000000001"

pl_job: ProductionLineJob = onedata_api.production_lines.execute_sync(id=pl_id)


### You can also specify a specific version of the production line to execute

pl_version = 20

pl_job: ProductionLineJob = onedata_api.production_lines.execute_sync(id=pl_id, version=pl_version)


### You can also execute a production line by name and project ID

pl_name = "Some PL name"
project_id = "00000000-0000-0000-0000-000000000002"

pl_job: ProductionLineJob = onedata_api.production_lines.execute_sync(name=pl_name, project_id=project_id, force_exact_match=True, force_distinct=True)


### You can also execute a production line with variables

pl_id = "00000000-0000-0000-0000-000000000001"

pl_job: ProductionLineJob = onedata_api.production_lines.execute_sync(id=pl_id, variable_assignments=
    [{"variableType": "integer", "variableName": "myVar", "variableValue": 5}])

Projects

This page describes how to interact with ONE DATA Projects.

Note: All the examples expect that there is already a OneDataApi object initialized and assigned to a variable called "onedata_api".

Getting a list of paginated projects

from onedata.common.types import Paginated, ResourceSortProperty

domain_id = "00000000-0000-0000-0000-000000000001"

projects: Paginated = onedata_api.projects.paginated(domain_id=domain_id, page=0, limit=5, sort_by=ResourceSortProperty.NAME, ordering=SortOrder.ASC, search="Test")

print(projects)
print(projects.items())

Users

This page shows how to get user information.

Note: All the examples expect that there is already a OneDataApi object initialized and assigned to a variable called "onedata_api".

Listing all users

from onedata.users.types import User
from onedata.common.types import Paginated

paginated_users: Paginated = onedata_api.users.paginated()
user_list: [User] = paginated_users.items
print(user_list)

Listing all users with pagination parameters

from onedata.users.types import User, UserSortProperty
from onedata.common.types import Paginated, SortOrder

page = 3
limit = 5
sort_by = UserSortProperty.email
ordering = SortOrder.ASC

paginated_users: Paginated = onedata_api.users.paginated(page=page, limit=limit, sort_by=sort_by, ordering=ordering)
user_list: [User] = paginated_users.items
print(user_list)

Listing all users of a domain/group

from onedata.users.types import User
from onedata.common.types import Paginated

group_id = "00000000-0000-0000-0000-000000000001"

paginated_users: Paginated = onedata_api.users.paginated(group_id=group_id)
user_list: [User] = paginated_users.items
print(user_list)

Listing all users with specific properties

from onedata.users.types import User
from onedata.common.types import Paginated

viewer = True # default is False
activated = False # default is True

paginated_users: Paginated = onedata_api.users.paginated(viewer=viewer, activated=activated)
user_list: [User] = paginated_users.items
print(user_list)

Get user by ID

from onedata.users.types import User

user_id = "00000000-0000-0000-0000-000000000001"

user: User = onedata_api.users.get(id=user_id)
print(user)

Get user by search (first name, last name or email must contain the string)

from onedata.users.types import User

search_string = "Test"

user: User = onedata_api.users.get(search_in_column=search_string)
print(user)

Get own user information

from onedata.users.types import User

# Retrieve the current authorized user of the OneDataApi object.

me: User = onedata_api.users.me()
print(me)

Virtual Data Tables

This page describes how to interact with ONE DATA virtual datatables.

Note: All the examples expect that there is already a OneDataApi object initialized and assigned to a variable called "onedata_api".

Create virtual datatable

from onedata.datatables.types import DataTable

### You can create a virtual datatable by specifying the name, the source datatable and the target project

target_project_id = "00000000-0000-0000-0000-000000000001"
vdt_source_id = "00000000-0000-0000-0000-000000000002"

vdt: DataTable = onedata_api.datatables.create_virtual(vdt_source_id=vdt_source_id,
                                                       vdt_name="New VDT via SDK",
                                                       target_project_id=target_project_id)

Update virtual datatable

from onedata.datatables.types import DataTable

vdt_id = "00000000-0000-0000-0000-000000000001"
vdt_source_id = "00000000-0000-0000-0000-000000000003"

update_config = {
    "origin": "datatable",
    "config": {
        "schema": "table",
        "dataId": vdt_source_id,
        "dataOptions": {
            'sql': 'Select * FROM inputTable'
        }
    }
}

### You can update a virtual datatable by ID with a specified datasource configuration

vdt: DataTable = onedata_api.datatables.update_virtual(vdt_id=vdt_id,
                                                       datasource_configuration=update_config)

### You can also update a virtual datatable by ID and specifying a sql and source_id

sql = "Select * FROM inputTable"

vdt: DataTable = onedata_api.datatables.update_virtual(vdt_id=vdt_id,
                                                       sql=sql,
                                                       vdt_source_id=vdt_source_id)


### You can also update a virtual datatable by name and project id and

vdt_project_id = "00000000-0000-0000-0000-000000000004"
vdt_name = "New Virtual Data Table"

vdt: DataTable = onedata_api.datatables.update_virtual(vdt_name=vdt_name,
                                                       vdt_project_id=vdt_project_id,
                                                       datasource_configuration=update_config)

Create virtual data table with specified configuration

from onedata.datatables.types import DataTable

vdt_source_id = "00000000-0000-0000-0000-000000000001"
vdt_name = "New Virtual Data Table"
target_project_id = "00000000-0000-0000-0000-000000000002"

config = {
    "origin": "datatable",
    "config": {
        "schema": "table",
        "dataId": vdt_source_id,
        "dataOptions": {
            'sql': 'Select * FROM inputTable'
        }
    }
}

### You can create a virtual data table with a specified  datasource configuration

vdt: DataTable = onedata_api.datatables.create_virtual_with_configuration(vdt_source_id=vdt_source_id,
                                                                          vdt_name=vdt_name,
                                                                          target_project_id=target_project_id,
                                                                          datasource_configuration=config)


### You can create a virtual data table with a specified sql

sql = "Select * FROM inputTable"

vdt: DataTable = onedata_api.datatables.create_virtual_with_configuration(vdt_source_id=vdt_source_id,
                                                                          vdt_name=vdt_name,
                                                                          target_project_id=target_project_id,
                                                                          sql=sql)

Workflow Jobs

This page shows how to get the shallow and deep jobs of a workflow.

Note: All the examples expect that there is already a OneDataApi object initialized and assigned to a variable called "onedata_api".

Get deep workflow job by ID

from onedata.workflows.jobs.types import WorkflowJob,

wf_job_id = "00000000-0000-0000-0000-000000000003"

wf_job: WorkflowJob = od.workflows.jobs.get(id=wf_job_id)

Get latest shallow job of a workflow

from onedata.workflows.jobs.types import WorkflowJobShallow, JobExecutionState

wf_id = "00000000-0000-0000-0000-000000000001"

latest_job_shallow: WorkflowJobShallow = onedata_api.workflows.jobs.get_latest_job_shallow(wf_id=wf_id, job_execution_state=JobExecutionState.SUCCESS)
print(f"latest shallow job: {latest_job_shallow}")

Get latest deep job of a workflow

from onedata.workflows.jobs.types import WorkflowJob, JobExecutionState

wf_id = "00000000-0000-0000-0000-000000000001"

latest_job_deep: WorkflowJob = onedata_api.workflows.jobs.get_latest_job_deep(wf_id=wf_id, job_execution_state=JobExecutionState.SUCCESS)
print(f"latest deep job: {latest_job_deep}")

Workflows

This page describes how to list, get & execute workflows.

Note: All the examples expect that there is already a OneDataApi object initialized and assigned to a variable called "onedata_api".

Listing workflows

from onedata.workflows.types import WorkflowInformation
from onedata.common.types import Paginated

# Warning: The list command is deprecated due to missleading naming.
paginated_workflows: Paginated = onedata_api.workflows.list()

# Please use paginated()
paginated_workflows: Paginated = onedata_api.workflows.paginated()
wf_list: [WorkflowInformation] = paginated_workflows.items

Listing workflows with parameters

from onedata.workflows.types import WorkflowInformation
from onedata.common.types import Paginated, ResourceSortProperty, SortOrder

page = 3
limit = 5
sort_by = ResourceSortProperty.name
ordering = SortOrder.ASC

# Warning: The list command is deprecated due to missleading naming.
paginated_workflows: Paginated = onedata_api.workflows.list(page=page, limit=limit, sort_by=sort_by, ordering=ordering)

# Please use paginated()
paginated_workflows: Paginated = onedata_api.workflows.list(page=page, limit=limit, sort_by=sort_by, ordering=ordering)
wf_list: [WorkflowInformation] = paginated_workflows.items

Getting a workflow

from onedata.workflows.types import Workflow

wf_id = "00000000-0000-0000-0000-000000000001"

wf: Workflow = onedata_api.workflows.get(wf_id)
print(wf.name)

Executing a workflow asynchronously

from onedata.workflows.jobs.types import WorkflowJob

### You can execute a workflow by its ID

wf_id = "00000000-0000-0000-0000-000000000001"

wf_job: WorkflowJob = onedata_api.workflows.execute_async(id=wf_id)


### You can also specify a specific version of the workflow to execute

wf_version = 20

wf_job: WorkflowJob = onedata_api.workflows.execute_async(id=wf_id, version=wf_version)


### You can also execute a workflow by name and project ID

wf_name = "Some WF name"
project_id = "00000000-0000-0000-0000-000000000002"

wf_job: WorkflowJob = onedata_api.workflows.execute_async(name=wf_name, project_id=project_id, force_exact_match=True, force_distinct=True)


### You can also execute a workflow with variables
wf_id = "00000000-0000-0000-0000-000000000001"

wf_job: WorkflowJob = onedata_api.workflows.execute_async(id=wf_id, variable_assignments=
    [{"variableType": "integer", "variableName": "myVar", "variableValue": 5}])

Executing a workflow synchronously

from onedata.workflows.jobs.types import WorkflowJob

### You can execute a workflow by its ID

wf_id = "00000000-0000-0000-0000-000000000001"

wf_job: WorkflowJob = onedata_api.workflows.execute_sync(id=wf_id)


### You can also specify a specific version of the workflow to execute

wf_version = 20

wf_job: WorkflowJob = onedata_api.workflows.execute_sync(id=wf_id, version=wf_version)


### You can also execute a workflow by name and project ID

wf_name = "Some WF name"
project_id = "00000000-0000-0000-0000-000000000002"

wf_job: WorkflowJob = onedata_api.workflows.execute_sync(name=wf_name, project_id=project_id, force_exact_match=True, force_distinct=True)


### You can also execute a workflow with variables
wf_id = "00000000-0000-0000-0000-000000000001"

wf_job: WorkflowJob = onedata_api.workflows.execute_sync(id=wf_id, variable_assignments=
    [{"variableType": "integer", "variableName": "myVar", "variableValue": 5}])