The ONE DATA platform is implemented on a client / server architecture. Our central Apache Spark component manages the parallelization and execution logic, including the available physical and virtual infrastructure components. The client is based on a HTML5 / JavaScript frontend and the server component is sub-divided in modules and is implemented in Java using Spring.io as the main application framework. The ONE LOGIC platform uses Spark, Python and R computation depending on the required context, which helps to achieve scalable and efficient workflow executions. A HDFS and Apache Parquet are used to save intermediate results and datasets. User management and meta information is stored in a DBMS (data base management system). The ONE DATA platform can scale-up and scale-out by design. Depending only on the availability of minimal hardware requirements, a nearly unlimited amount of data can be processed using ONE DATA.
