- Authorization
- Connections
- Credentials
- Datatables
- Functions
- Production Line Jobs
- Production Lines
- Projects
- Users
- Virtual Data Tables
- Workflow Jobs
- Workflows
Authorization
Authorization using username and password
Authorization can be accomplished by providing a username and a password to the OneDataApi constructor.
Example:
from onedata.api import OneDataApi
od_base_url = "https://internal.onedata.de"
od_username = "test@onelogic.de"
od_password = "testtesttest"
onedata_api = OneDataApi(base_url=od_base_url, username=od_username, password=od_password)
Examples with additional parameters:
from onedata.api import OneDataApi
onedata_api = OneDataApi(base_url="https://internal.onedata.de",
username="test@onelogic.de",
password="testtesttest",
timeout=10, # requests will timeout after 10 seconds
verify=False, # disables ssl verification, invalid/self-signed certificates will be accepted
sleep_after_response_millis=1000, # performs an artificial delay after
request_transformers=[], # custom request transformers, see documentation for more details
response_transformers=[]) # custom response transformers, see documentation for more details
Authorization using authentication token
Authorization can be accomplished by providing an authentication token to the OneDataApiconstructor.
Example:
from onedata.api import OneDataApi
od_base_url = "https://internal.onedata.de"
auth_token = "Bearer SomeRandomToken"
onedata_api = OneDataApi(base_url=od_base_url, token=auth_token)
Note: Providing the username, password and the token to the OneDataApi constructor will result in an error. This is due to the fact that it would be impossible to know which of the two authorization methods should be used.
Authorization using global variables (used in OD Python Processors)
Authorization can be accomplished by providing the request to OneDataApi.from_globals.
Example:
from onedata.api import OneDataApi
onedata_api = OneDataApi.from_globals(globals())
Authorization using request (used in OD Functions)
Authorization can be accomplished by providing the request to OneDataApi.from_request.
Example:
from onedata.api import OneDataApi
def handle(req):
onedata_api = OneDataApi.from_request(req)
Connections
This page shows how to get information about datatables of a database connection and how to create a datatable from a database connection.
Note: All the examples expect that there is already a OneDataApi object initialized and assigned to a variable called "onedata_api"
Create datatable from database connection
from onedata.datatables.types import DataTable
connection_id: str = "5ff899b4-0915-450c-a1c3-3a7f8794dccb"
name: str = "Datatable from Connection via SDK"
dt: DataTable = onedata_api.connections.create_data_table(connection_id=connection_id,
connection_table_name="tms_test",
dt_name=name,
dt_description="desc")
Get all datatables of a database connection
from onedata.common.types import SortOrder, Paginated
from onedata.connections.types import TablesListingMode
connection_id: str = "5ff899b4-0915-450c-a1c3-3a7f8794dccb"
connections_list: Paginated = onedata_api.connections.paginated_database_table_information(connection_id=connection_id,
ordering=SortOrder.DESC,
tables_listing_mode=TablesListingMode.SCHEMA_CONFIGURED_IN_THE_CONNECTION)
Credentials
This page shows the different ways to retrieve credentials and how to use them to get authorization when communicating with ONE DATA API.
Note: All the examples expect that there is already a OneDataApi object initialized and assigned to a variable called "onedata_api".
Get credential by ID
Credentials can be retrieved by providing its ID.
from onedata.credentials.types import Credentials
technical_user_credentials_id: uuid = "30b2d7d9-4027-41ad-b5d1-1c4ba98390f9"
technical_user_credentials: Credentials = onedata_api.credentials.get(technical_user_credentials_id)
Authorization using credentials of a technical user
Authorization as a technical user can be accomplished by providing username and password from an exposed credential to the OneDataApi constructor.
Example:
from onedata.api import OneDataApi
from onedata.credentials.types import Credentials
onedata_base_url = "https://internal.onedata.de"
technical_user_credentials_id: uuid = "30b2d7d9-4027-41ad-b5d1-1c4ba98390f9"
technical_user_credentials: Credentials = onedata_api.credentials.get(technical_user_credentials_id)
onedata_api_technical_user = OneDataApi(base_url=onedata_base_url,
username=technical_user_credentials.key_information.username,
password=technical_user_credentials.key_information.password)
Datatables
This page describes how to interact with ONE DATA datatables.
Note: All the examples expect that there is already a OneDataApi object initialized and assigned to a variable called "onedata_api".
Create datatable
from onedata.datatables.types import DatasetLock, DataTableStorageFormat, UploadedDataResponse
import pandas
### You can create a datatable by specifying the headers and the content
project_id = "00000000-0000-0000-0000-000000000002"
response: UploadedDataResponse = onedata_api.datatables.create(name="Datatable created from headers and content via SDK",
headers=["a", "b"],
content=[{"a": 1, "b": "asd"}] * 200, # to create 200 records in the datatable
project_id=project_id,
locked=DatasetLock.OPEN, # optional, default OPEN
storage_format=DataTableStorageFormat.PARQUET, # optional, default PARQUET
create_fallback=True, # optional, default True
description="some desc", # optional, default None
notes="notes", # optional, default None
tags=["tag1", "tag2"]) # optional, default None
### You can create a datatable by specifying a pandas DataFrame
project_id = "00000000-0000-0000-0000-000000000002"
pandas_df: pandas.DataFrame = pandas.DataFrame(data={'col1': [1, 2], 'col2': [3, 4]})
response: UploadedDataResponse = onedata_api.datatables.create_from_pandas(name="Datatable created from pandas DataFrame via SDK",
data=pandas_df,
project_id=project_id,
locked=DatasetLock.OPEN, # optional, default OPEN
storage_format=DataTableStorageFormat.PARQUET, # optional, default PARQUET
create_fallback=True, # optional, default True
description="some desc", # optional, default None
notes="notes", # optional, default None
tags=["tag1", "tag2"]) # optional, default None
Get datatable
from onedata.datatables.types import DataTable
### You can get a datatable by datatable ID
datatable_id = "00000000-0000-0000-0000-000000000004"
response: DataTable = onedata_api.datatables.get(id=datatable_id)
### You can get a datatable by name and project ID
datatable_name = "Some DT name"
project_id = "00000000-0000-0000-0000-000000000002"
response: DataTable = onedata_api.datatables.get(name=datatable_name,
project_id=project_id,
force_exact_match=True, # optional, default True
force_distinct=True) # optional, default True
Update datatable
from onedata.datatables.types import DataTable
### You can update a datatable by datatable ID
datatable_id = "00000000-0000-0000-0000-000000000004"
response: DataTable = onedata_api.datatables.update(data_id=datatable_id,
new_name="Some random new name", # optional, default None
new_tags=["new_tag"], # optional, default None
new_description="new_desc", # optional, default None
new_notes="new_notes", # optional, default None
new_attributes=None, # optional, default None
new_attributes_operator=None, # optional, default None
new_locked=None, # optional, default None
new_home_project_id=None) # optional, default None
### You can update a datatable by name and project ID
datatable_name = "Some DT name"
project_id = "00000000-0000-0000-0000-000000000002"
response: DataTable = onedata_api.datatables.update(name=datatable_name,
project_id=project_id,
force_exact_match=True, # optional, default True
force_distinct=True, # optional, default True
new_name="Some random new name", # optional, default None
new_tags=["new_tag"], # optional, default None
new_description="new_desc", # optional, default None
new_notes="new_notes", # optional, default None
new_attributes=None, # optional, default None
new_attributes_operator=None, # optional, default None
new_locked=None, # optional, default None
new_home_project_id=None) # optional, default None
### You can update a datatable by modifying a DataTable object and saving it
datatable_id = "00000000-0000-0000-0000-000000000004"
datatable: DataTable = onedata_api.datatables.get(id=datatable_id)
datatable.description = "Hey, I am the new fancy description. Set by Python SDK. COOL, hum? Hum?"
response: DataTable = onedata_api.datatables.update_by_reference(data_table=datatable)
Get datatable content
from onedata.datatables.types import DataTableContent
### You can get a datatable's content by datatable ID
datatable_id = "00000000-0000-0000-0000-000000000004"
datatable: DataTableContent = onedata_api.datatables.get_content(data_id=datatable_id,
page=1, # optional, default 1
pagination_size=100) # optional, default 10000
### You can get a datatable's content by FRT ID
frt_id = "00000000-0000-0000-0000-000000000005"
datatable: DataTableContent = onedata_api.datatables.get_content(frt_id=frt_id,
page=1, # optional, default 1
pagination_size=100) # optional, default 10000
### You can get a datatable's content by name and project ID
datatable_name = "Some DT name"
project_id = "00000000-0000-0000-0000-000000000002"
datatable: DataTableContent = onedata_api.datatables.get_content(name=datatable_name,
project_id=project_id,
page=1, # optional, default 1
pagination_size=100, # optional, default 10000
force_exact_match=True, # optional, default True
force_distinct=True) # optional, default True
Get datatable samples
from onedata.datatables.types import DataTableContent
### You can get a datatable's samples by datatable ID
datatable_id = "00000000-0000-0000-0000-000000000004"
datatable: DataTableContent = onedata_api.datatables.get_content_samples(data_id=datatable_id,
sample_count=50) # optional, default 20
### You can get a datatable's samples by FRT ID
frt_id = "00000000-0000-0000-0000-000000000005"
datatable: DataTableContent = onedata_api.datatables.get_content_samples(frt_id=frt_id,
sample_count=50) # optional, default 20
### You can get a datatable's samples by name and project ID
datatable_name = "Some DT name"
project_id = "00000000-0000-0000-0000-000000000002"
datatable: DataTableContent = onedata_api.datatables.get_content_samples(name=datatable_name,
project_id=project_id,
sample_count=50, # optional, default 20
force_exact_match=True, # optional, default True
force_distinct=True) # optional, default True
Replace datatable content
### You can replace a datatable's content by datatable ID
datatable_id = "00000000-0000-0000-0000-000000000004"
new_content = [{"a": 10, "b": "replaced"}]
onedata_api.datatables.replace_content(data_id=datatable_id,
new_data=new_content) # optional, default None resulting in clearing the datatable content
### You can replace a datatable's content by FRT ID
frt_id = "00000000-0000-0000-0000-000000000005"
new_content = [{"a": 10, "b": "replaced"}]
onedata_api.datatables.replace_content(frt_id=frt_id,
new_data=new_content) # optional, default None resulting in clearing the datatable content
### You can replace a datatable's content by name and project ID
datatable_name = "Some DT name"
project_id = "00000000-0000-0000-0000-000000000002"
onedata_api.datatables.replace_content(name=datatable_name,
project_id=project_id,
new_data=new_content, # optional, default None resulting in clearing the datatable content
force_exact_match=True, # optional, default True
force_distinct=True) # optional, default True
Patch datatable content
### You can patch a datatable's content by datatable ID
datatable_id = "00000000-0000-0000-0000-000000000004"
deleted = ["2c790106-ea5e-4ff2-9f07-9016691242b3"]
updated = [{"a": 2, "b": "some text", "__ROW_ID__": "2c790106-ea5e-4ff2-9f07-9016691242b2"}]
added = [{"a": 1, "b": "new"}]
onedata_api.datatables.patch_content(data_id=datatable_id,
deleted=deleted, # optional, default None
updated=updated, # optional, default None
added=added) # optional, default None
### You can patch a datatable's content by FRT ID
frt_id = "00000000-0000-0000-0000-000000000005"
deleted = ["2c790106-ea5e-4ff2-9f07-9016691242b3"]
updated = [{"a": 2, "b": "some text", "__ROW_ID__": "2c790106-ea5e-4ff2-9f07-9016691242b2"}]
added = [{"a": 1, "b": "new"}]
onedata_api.datatables.patch_content(frt_id=frt_id,
deleted=deleted, # optional, default None
updated=updated, # optional, default None
added=added) # optional, default None
### You can patch a datatable's content by name and project ID
datatable_name = "Some DT name"
project_id = "00000000-0000-0000-0000-000000000002"
deleted = ["2c790106-ea5e-4ff2-9f07-9016691242b3"]
updated = [{"a": 2, "b": "some text", "__ROW_ID__": "2c790106-ea5e-4ff2-9f07-9016691242b2"}]
added = [{"a": 1, "b": "new"}]
onedata_api.datatables.patch_content(name=datatable_name,
project_id=project_id,
deleted=deleted, # optional, default None
updated=updated, # optional, default None
added=added, # optional, default None
force_exact_match=True, # optional, default True
force_distinct=True) # optional, default True
Update datatable content
### You can update a datatable's content by datatable ID
datatable_id = "00000000-0000-0000-0000-000000000004"
updated = [{"a": 2, "b": "some text", "__ROW_ID__": "2c790106-ea5e-4ff2-9f07-9016691242b2"}]
onedata_api.datatables.update_content(data_id=datatable_id,
updated=updated) # optional, default None
### You can update a datatable's content by FRT ID
frt_id = "00000000-0000-0000-0000-000000000005"
updated = [{"a": 2, "b": "some text", "__ROW_ID__": "2c790106-ea5e-4ff2-9f07-9016691242b2"}]
onedata_api.datatables.update_content(frt_id=frt_id,
updated=updated) # optional, default None
### You can update a datatable's content by name and project ID
datatable_name = "Some DT name"
project_id = "00000000-0000-0000-0000-000000000002"
updated = [{"a": 2, "b": "some text", "__ROW_ID__": "2c790106-ea5e-4ff2-9f07-9016691242b2"}]
onedata_api.datatables.update_content(name=datatable_name,
project_id=project_id,
updated=updated, # optional, default None
force_exact_match=True, # optional, default True
force_distinct=True) # optional, default True
Delete datatable content
### You can delete a datatable's content by datatable ID
datatable_id = "00000000-0000-0000-0000-000000000004"
deleted = ["2c790106-ea5e-4ff2-9f07-9016691242b3"]
onedata_api.datatables.delete_content(data_id=datatable_id,
deleted=deleted) # optional, default None
### You can delete a datatable's content by FRT ID
frt_id = "00000000-0000-0000-0000-000000000005"
deleted = ["2c790106-ea5e-4ff2-9f07-9016691242b3"]
onedata_api.datatables.delete_content(frt_id=frt_id,
deleted=deleted) # optional, default None
### You can delete a datatable's content by name and project ID
datatable_name = "Some DT name"
project_id = "00000000-0000-0000-0000-000000000002"
deleted = ["2c790106-ea5e-4ff2-9f07-9016691242b3"]
onedata_api.datatables.delete_content(name=datatable_name,
project_id=project_id,
deleted=deleted, # optional, default None
force_exact_match=True, # optional, default True
force_distinct=True) # optional, default True
Add datatable content
### You can add to a datatable's content by datatable ID
datatable_id = "00000000-0000-0000-0000-000000000004"
added = [{"a": 1, "b": "new"}]
onedata_api.datatables.add_content(data_id=datatable_id,
added=added) # optional, default None
### You can add to a datatable's content by FRT ID
frt_id = "00000000-0000-0000-0000-000000000005"
added = [{"a": 1, "b": "new"}]
onedata_api.datatables.add_content(frt_id=frt_id,
added=added) # optional, default None
### You can add to a datatable's content by name and project ID
datatable_name = "Some DT name"
project_id = "00000000-0000-0000-0000-000000000002"
added = [{"a": 1, "b": "new"}]
onedata_api.datatables.add_content(name=datatable_name,
project_id=project_id,
added=added, # optional, default None
force_exact_match=True, # optional, default True
force_distinct=True) # optional, default True
Functions
This page describes how to interact with ONE DATA Functions.
Note: All the examples expect that there is already a OneDataApi object initialized and assigned to a variable called "onedata_api"
Executing a Function
from onedata.functions.types import FunctionExecutionResult
### You can execute a function by its ID
function_id = "00000000-0000-0000-0000-000000000001"
result: FunctionExecutionResult = onedata_api.functions.execute(id=function_id)
### You can also execute a function by name and project ID
function_name = "Some Function name"
project_id = "00000000-0000-0000-0000-000000000002"
result: FunctionExecutionResult = onedata_api.functions.execute(name=function_name, project_id=project_id, force_exact_match=False)
### You can also execute a function with arguments
function_id = "00000000-0000-0000-0000-000000000001"
result: FunctionExecutionResult = onedata_api.functions.execute(id=function_id, arguments={"text": "I love Python"})
Production Line Jobs
This page shows how to get jobs of a production line.
Note: All the examples expect that there is already a OneDataApi object initialized and assigned to a variable called "onedata_api"
Get production line job by ID
from onedata.productionlines.jobs.types import ProductionLineJob
pl_job_id = "00000000-0000-0000-0000-000000000003"
pl_job: ProductionLineJob = onedata_api.production_lines.jobs.get(id=pl_job_id)
Get latest job of a production line
from onedata.productionlines.jobs.types import ProductionLineJob
pl_id = "00000000-0000-0000-0000-000000000003"
pl_job: ProductionLineJob = onedata_api.production_lines.jobs.get_latest(pl_id=pl_id)
Production Lines
This page shows the different ways to retrieve a single production line or a paginated list of production lines.
Note: All the examples expect that there is already a OneDataApi object initialized and assigned to a variable called "onedata_api".
Listing production lines
from onedata.productionlines.types import ProductionLine
from onedata.common.types import Paginated
paginated_production_lines: Paginated = onedata_api.production_lines.paginated()
production_line_list: [ProductionLine] = paginated_production_lines.items
Listing production lines with parameters
from onedata.productionlines.types import ProductionLine
from onedata.common.types import Paginated, ResourceSortProperty, SortOrder
page = 3
limit = 5
sort_by = ResourceSortProperty.name
ordering = SortOrder.ASC
paginated_production_lines: Paginated = onedata_api.production_lines.paginated(page=page, limit=limit, sort_by=sort_by, ordering=ordering)
production_line_list: [ProductionLine] = paginated_production_lines.items
Getting a production line by id
from uuid import UUID
from onedata.productionlines.types import ProductionLine
pl_id = "00000000-0000-0000-0000-000000000001"
pl: ProductionLine = onedata_api.production_lines.get(id=UUID(pl_id))
print(pl.name)
Getting a production line by name and project id
from uuid import UUID
from onedata.productionlines.types import ProductionLine
pl_name = "New Production Line"
project_id = "00000000-0000-0000-0000-000000000001"
pl: ProductionLine = onedata_api.production_lines.get(name=pl_name, project_id=UUID(project_id))
print(pl.id)
Executing a production line asynchronously
from onedata.productionlines.jobs.types import ProductionLineJob
### You can execute a production line by its ID
pl_id = "00000000-0000-0000-0000-000000000001"
pl_job: ProductionLineJob = onedata_api.production_lines.execute_async(id=pl_id)
### You can also specify a specific version of the production line to execute
pl_version = 20
pl_job: ProductionLineJob = onedata_api.production_lines.execute_async(id=pl_id, version=pl_version)
### You can also execute a production line by name and project ID
pl_name = "Some PL name"
project_id = "00000000-0000-0000-0000-000000000002"
pl_job: ProductionLineJob = onedata_api.production_lines.execute_async(name=pl_name, project_id=project_id, force_exact_match=True, force_distinct=True)
### You can also execute a production line with variables
pl_id = "00000000-0000-0000-0000-000000000001"
pl_job: ProductionLineJob = onedata_api.production_lines.execute_async(id=pl_id, variable_assignments=
[{"variableType": "integer", "variableName": "myVar", "variableValue": 5}])
Executing a production line synchronously
from onedata.productionlines.jobs.types import ProductionLineJob
### You can execute a production line by its ID
pl_id = "00000000-0000-0000-0000-000000000001"
pl_job: ProductionLineJob = onedata_api.production_lines.execute_sync(id=pl_id)
### You can also specify a specific version of the production line to execute
pl_version = 20
pl_job: ProductionLineJob = onedata_api.production_lines.execute_sync(id=pl_id, version=pl_version)
### You can also execute a production line by name and project ID
pl_name = "Some PL name"
project_id = "00000000-0000-0000-0000-000000000002"
pl_job: ProductionLineJob = onedata_api.production_lines.execute_sync(name=pl_name, project_id=project_id, force_exact_match=True, force_distinct=True)
### You can also execute a production line with variables
pl_id = "00000000-0000-0000-0000-000000000001"
pl_job: ProductionLineJob = onedata_api.production_lines.execute_sync(id=pl_id, variable_assignments=
[{"variableType": "integer", "variableName": "myVar", "variableValue": 5}])
Projects
This page describes how to interact with ONE DATA Projects.
Note: All the examples expect that there is already a OneDataApi object initialized and assigned to a variable called "onedata_api".
Getting a list of paginated projects
from onedata.common.types import Paginated, ResourceSortProperty
domain_id = "00000000-0000-0000-0000-000000000001"
projects: Paginated = onedata_api.projects.paginated(domain_id=domain_id, page=0, limit=5, sort_by=ResourceSortProperty.NAME, ordering=SortOrder.ASC, search="Test")
print(projects)
print(projects.items())
Users
This page shows how to get user information.
Note: All the examples expect that there is already a OneDataApi object initialized and assigned to a variable called "onedata_api".
Listing all users
from onedata.users.types import User
from onedata.common.types import Paginated
paginated_users: Paginated = onedata_api.users.paginated()
user_list: [User] = paginated_users.items
print(user_list)
Listing all users with pagination parameters
from onedata.users.types import User, UserSortProperty
from onedata.common.types import Paginated, SortOrder
page = 3
limit = 5
sort_by = UserSortProperty.email
ordering = SortOrder.ASC
paginated_users: Paginated = onedata_api.users.paginated(page=page, limit=limit, sort_by=sort_by, ordering=ordering)
user_list: [User] = paginated_users.items
print(user_list)
Listing all users of a domain/group
from onedata.users.types import User
from onedata.common.types import Paginated
group_id = "00000000-0000-0000-0000-000000000001"
paginated_users: Paginated = onedata_api.users.paginated(group_id=group_id)
user_list: [User] = paginated_users.items
print(user_list)
Listing all users with specific properties
from onedata.users.types import User
from onedata.common.types import Paginated
viewer = True # default is False
activated = False # default is True
paginated_users: Paginated = onedata_api.users.paginated(viewer=viewer, activated=activated)
user_list: [User] = paginated_users.items
print(user_list)
Get user by ID
from onedata.users.types import User
user_id = "00000000-0000-0000-0000-000000000001"
user: User = onedata_api.users.get(id=user_id)
print(user)
Get user by search (first name, last name or email must contain the string)
from onedata.users.types import User
search_string = "Test"
user: User = onedata_api.users.get(search_in_column=search_string)
print(user)
Get own user information
from onedata.users.types import User
# Retrieve the current authorized user of the OneDataApi object.
me: User = onedata_api.users.me()
print(me)
Virtual Data Tables
This page describes how to interact with ONE DATA virtual datatables.
Note: All the examples expect that there is already a OneDataApi object initialized and assigned to a variable called "onedata_api".
Create virtual datatable
from onedata.datatables.types import DataTable
### You can create a virtual datatable by specifying the name, the source datatable and the target project
target_project_id = "00000000-0000-0000-0000-000000000001"
vdt_source_id = "00000000-0000-0000-0000-000000000002"
vdt: DataTable = onedata_api.datatables.create_virtual(vdt_source_id=vdt_source_id,
vdt_name="New VDT via SDK",
target_project_id=target_project_id)
Update virtual datatable
from onedata.datatables.types import DataTable
vdt_id = "00000000-0000-0000-0000-000000000001"
vdt_source_id = "00000000-0000-0000-0000-000000000003"
update_config = {
"origin": "datatable",
"config": {
"schema": "table",
"dataId": vdt_source_id,
"dataOptions": {
'sql': 'Select * FROM inputTable'
}
}
}
### You can update a virtual datatable by ID with a specified datasource configuration
vdt: DataTable = onedata_api.datatables.update_virtual(vdt_id=vdt_id,
datasource_configuration=update_config)
### You can also update a virtual datatable by ID and specifying a sql and source_id
sql = "Select * FROM inputTable"
vdt: DataTable = onedata_api.datatables.update_virtual(vdt_id=vdt_id,
sql=sql,
vdt_source_id=vdt_source_id)
### You can also update a virtual datatable by name and project id and
vdt_project_id = "00000000-0000-0000-0000-000000000004"
vdt_name = "New Virtual Data Table"
vdt: DataTable = onedata_api.datatables.update_virtual(vdt_name=vdt_name,
vdt_project_id=vdt_project_id,
datasource_configuration=update_config)
Create virtual data table with specified configuration
from onedata.datatables.types import DataTable
vdt_source_id = "00000000-0000-0000-0000-000000000001"
vdt_name = "New Virtual Data Table"
target_project_id = "00000000-0000-0000-0000-000000000002"
config = {
"origin": "datatable",
"config": {
"schema": "table",
"dataId": vdt_source_id,
"dataOptions": {
'sql': 'Select * FROM inputTable'
}
}
}
### You can create a virtual data table with a specified datasource configuration
vdt: DataTable = onedata_api.datatables.create_virtual_with_configuration(vdt_source_id=vdt_source_id,
vdt_name=vdt_name,
target_project_id=target_project_id,
datasource_configuration=config)
### You can create a virtual data table with a specified sql
sql = "Select * FROM inputTable"
vdt: DataTable = onedata_api.datatables.create_virtual_with_configuration(vdt_source_id=vdt_source_id,
vdt_name=vdt_name,
target_project_id=target_project_id,
sql=sql)
Workflow Jobs
This page shows how to get the shallow and deep jobs of a workflow.
Note: All the examples expect that there is already a OneDataApi object initialized and assigned to a variable called "onedata_api".
Get deep workflow job by ID
from onedata.workflows.jobs.types import WorkflowJob,
wf_job_id = "00000000-0000-0000-0000-000000000003"
wf_job: WorkflowJob = od.workflows.jobs.get(id=wf_job_id)
Get latest shallow job of a workflow
from onedata.workflows.jobs.types import WorkflowJobShallow, JobExecutionState
wf_id = "00000000-0000-0000-0000-000000000001"
latest_job_shallow: WorkflowJobShallow = onedata_api.workflows.jobs.get_latest_job_shallow(wf_id=wf_id, job_execution_state=JobExecutionState.SUCCESS)
print(f"latest shallow job: {latest_job_shallow}")
Get latest deep job of a workflow
from onedata.workflows.jobs.types import WorkflowJob, JobExecutionState
wf_id = "00000000-0000-0000-0000-000000000001"
latest_job_deep: WorkflowJob = onedata_api.workflows.jobs.get_latest_job_deep(wf_id=wf_id, job_execution_state=JobExecutionState.SUCCESS)
print(f"latest deep job: {latest_job_deep}")
Workflows
This page describes how to list, get & execute workflows.
Note: All the examples expect that there is already a OneDataApi object initialized and assigned to a variable called "onedata_api".
Listing workflows
from onedata.workflows.types import WorkflowInformation
from onedata.common.types import Paginated
# Warning: The list command is deprecated due to missleading naming.
paginated_workflows: Paginated = onedata_api.workflows.list()
# Please use paginated()
paginated_workflows: Paginated = onedata_api.workflows.paginated()
wf_list: [WorkflowInformation] = paginated_workflows.items
Listing workflows with parameters
from onedata.workflows.types import WorkflowInformation
from onedata.common.types import Paginated, ResourceSortProperty, SortOrder
page = 3
limit = 5
sort_by = ResourceSortProperty.name
ordering = SortOrder.ASC
# Warning: The list command is deprecated due to missleading naming.
paginated_workflows: Paginated = onedata_api.workflows.list(page=page, limit=limit, sort_by=sort_by, ordering=ordering)
# Please use paginated()
paginated_workflows: Paginated = onedata_api.workflows.list(page=page, limit=limit, sort_by=sort_by, ordering=ordering)
wf_list: [WorkflowInformation] = paginated_workflows.items
Getting a workflow
from onedata.workflows.types import Workflow
wf_id = "00000000-0000-0000-0000-000000000001"
wf: Workflow = onedata_api.workflows.get(wf_id)
print(wf.name)
Executing a workflow asynchronously
from onedata.workflows.jobs.types import WorkflowJob
### You can execute a workflow by its ID
wf_id = "00000000-0000-0000-0000-000000000001"
wf_job: WorkflowJob = onedata_api.workflows.execute_async(id=wf_id)
### You can also specify a specific version of the workflow to execute
wf_version = 20
wf_job: WorkflowJob = onedata_api.workflows.execute_async(id=wf_id, version=wf_version)
### You can also execute a workflow by name and project ID
wf_name = "Some WF name"
project_id = "00000000-0000-0000-0000-000000000002"
wf_job: WorkflowJob = onedata_api.workflows.execute_async(name=wf_name, project_id=project_id, force_exact_match=True, force_distinct=True)
### You can also execute a workflow with variables
wf_id = "00000000-0000-0000-0000-000000000001"
wf_job: WorkflowJob = onedata_api.workflows.execute_async(id=wf_id, variable_assignments=
[{"variableType": "integer", "variableName": "myVar", "variableValue": 5}])
Executing a workflow synchronously
from onedata.workflows.jobs.types import WorkflowJob
### You can execute a workflow by its ID
wf_id = "00000000-0000-0000-0000-000000000001"
wf_job: WorkflowJob = onedata_api.workflows.execute_sync(id=wf_id)
### You can also specify a specific version of the workflow to execute
wf_version = 20
wf_job: WorkflowJob = onedata_api.workflows.execute_sync(id=wf_id, version=wf_version)
### You can also execute a workflow by name and project ID
wf_name = "Some WF name"
project_id = "00000000-0000-0000-0000-000000000002"
wf_job: WorkflowJob = onedata_api.workflows.execute_sync(name=wf_name, project_id=project_id, force_exact_match=True, force_distinct=True)
### You can also execute a workflow with variables
wf_id = "00000000-0000-0000-0000-000000000001"
wf_job: WorkflowJob = onedata_api.workflows.execute_sync(id=wf_id, variable_assignments=
[{"variableType": "integer", "variableName": "myVar", "variableValue": 5}])