Note: This article is an extension to the article Debug Mode - General Explanation
The main goal of this feature is to deliver an easily accessible and user-friendly visualization of debugging information to users, which is provided by the ONE DATA server. This will help have a more detailed Workflow diagnosis, thereby saving more time developing them. It may also be used to evaluate performance bottlenecks in existing and newly created Workflows.
Given below, are the new features along with a detailed description for each one of them.
- Display Debug Information and Intermediate Data Samples
- Workflow Edge Coloring for Visualized Execution Status
- Trigger the Debug Mode Directly From Within a Processor
Keep in mind that all these features are only accessible after performing a debug action.
Display Debug Information and Intermediate Data Samples
To provide a better debug experience, this feature enables the user to generate on-the-fly debugging information and counts on datasets 'flowing' over the edges connecting Processors.
The debugging result within the Processor no longer contains only JSON result objects (compared to previous ONE DATA versions), but in addition, it shows sample records of the data at input and output port level, displayed in different tabs.
Each tab shows data samples, row count, and execution time information.
The same tables can be seen by clicking on the input or output ports of a Processor within the workflow.
The bug icons on ports will be visible only when the job result contains "dataDebuggerResult".
Also by executing a Workflow, you can see the execution time on Processors as a tooltip (hover your mouse over the desired processor).
Note: OD has many transformation Processors. These do not cause computation on the data inside themselves. Instead, all collected transformations will be executed in a later Processor, where data gets materialized (e.g. Row Count, Result Table, Save, etc.). So while the preceding Processors don't have much execution time themselves, they contribute to the complexity (and therefore the execution time) of terminal Processors. Full Debug can give some insights into the complexity added by each Processor (by comparing the increase in execution times).
Workflow Edge Coloring for Visualized Execution Status
Showing colored Workflow edges based on the execution status generates a user-friendly visualization experience. This helps to better locate Processors that affect the job and cause a relative failure (especially in complex workflows). Usually, this is used to narrow down an issue to specific Processors, so that isolated debug actions can be performed exactly there in later stages.
Note on the priority of visualization: The decreased importance goes from ERROR to WARNING to SUCCESS.
In general, Workflow edge colors depend on the execution status of the connected Processor. More specifically, the Processor affects the edge connected to its input port. Coloring is divided as follows:
Edge has a successful Processor as target.
Edge has a warning Processor as target.
Edge has a failed Processor as target.
Edge has a non executed Processor as target
Trigger the Debug Mode Directly From Within a Processor
This feature already exists in the old version of the Debug Mode (check the following link). What's new is that this functionality is now available within the Processor configuration to offer ease of using Debug Mode and reduce the user transition.
Presented below, are some example scenarios for using the Debug Mode's new features:
- Executing the Workflow in "save and debug" mode
- Executing Workflow showing different edge colors as per the execution status
- Executing partially using "fast debug" or "full debug" options on a specific Processor
- If for some reason the server cannot provide debug data, the user obviously cannot visualize it through the UI.
- Currently when a partial execution is performed (right-click on a Processor with any Debug Mode), the whole Workflow is validated before execution. Invalid (or missing but mandatory) configuration values anywhere in the Workflow will prevent the execution.
- At the present, the above functionalities do not support grouped Processors yet.