Note: This article is an extension to the article Debug Mode - General Explanation


Overview

The main goal of this feature is to deliver an easily accessible and user-friendly visualization of debug information to users, which is provided by the ONE DATA server. This will help have a more detailed Workflow diagnosis, thereby save more time developing them. It may also be used to evaluate performance bottlenecks in existing and newly created Workflows.


New Features

Given below, are the new features along with a detailed descriptions for each one of them.


Keep in mind that all these features are only accessible after performing a debug action.


Display Debug Information and Intermediate Data Samples

In order to provide a better debug experience, this feature enables the user to generate on-the-fly debug information and counts on datasets 'flowing' over the edges connecting Processors.

The debugging result within the Processor no longer contains only JSON result objects (compared to previous ONE DATA versions), but in addition it shows sample records of the data at input and output port level, displayed in different tabs.

Each tab shows data samples, row count and execution time information.


The same tables can be seen by clicking on the input or output ports of a Processor within the workflow.



The bug icons on ports will be visible only when the job result contains "dataDebuggerResult".


Also by executing the Workflow in Debug Mode, you can see the row count on edges as tooltip (hover over one of the Workflow edges using the mouse cursor).



Workflow Edge Coloring for Visualized Execution Status

Showing colored Workflow edges based on the execution status generates a user-friendly visualization experience. This helps to better locate Processors that affect the job and cause a relative failure (especially in complex workflows). Usually this is used to narrow down an issue to specific Processors, so that isolated debug actions can be performed exactly there in later stages.


Note on priority of visualization: The decreased importance goes from ERROR to WARNING to SUCCESS.


In general, Workflow edge colors depend on the execution status of the connected Processor. More specifically, the Processor affects the edge connected to it's input port. Coloring is divided as follows:


ColorStatusDefinition
GreenSuccessful

Edge has a successful Processor as target.

  • Processor has a corresponding on-the-fly result
  • Processor does not have any corresponding warnings

OrangeWarning

Edge has a warning Processor as target.

  • Processor does not have any corresponding errors
  • Processor has corresponding warnings

RedFailed

Edge has a failed Processor as target.

  • Processor has corresponding errors

GreyNot Executed

Edge has a non executed Processor as target

  • Processor does not have any corresponding errors
  • Processor does not have any corresponding results 
  • Processor does not have any corresponding warnings


Trigger the Debug Mode Directly From Within a Processor

This feature already exists in the old version of the Debug Mode (check the following link). What's new is that this functionality is now available within the Processor configuration to offer an ease of using Debug Mode and reduce the user transition.



Example Scenarios

Presented below, are some example scenarios for using the Debug Mode's new features:

  • Executing the Workflow in "save and debug" mode


  • Executing Workflow showing different edge colors as per the execution status


  • Executing partially using "fast debug" or "full debug" options on a specific Processor


  • Triggering Debug Modes from within a Processor configuration


Some Constraints

  • If for some reason the server cannot provide debug-data, the user obviously cannot visualize it through the UI.
  • Currently when a partial execution is performed (right click on a Processor with any Debug Mode), the whole Workflow is validated before execution. Invalid (or missing but mandatory) configuration values anywhere in the Workflow will prevent the execution.
  • In the present, the above functionalities do not support grouped Processors yet.