Extract & Load Overview

The Extract & Load objects in the Data Workshop tools are specifically designed to streamline the process of transferring data from various sources, such as databases, APIs, or files, and loading it into designated destinations, inside the ARPIA platform. These tools simplify complex data movements by automating the extraction and loading steps, allowing for faster, more efficient data handling. By centralizing these operations, Data Workshop ensures data consistency and accuracy across different systems, enabling teams to focus more on data analysis and less on data preparation.

Development Environment

The tools available for these objects are organized into two main development environment categories:

Extract

These objects automate the extraction process from MySQL-compatible databases that are registered as Data Sources within the ARPIA workspace. Extractions can be configured to either transfer data directly from table to table or to use custom SQL queries, enabling precise retrieval of the desired information from the database.

• AP DataPipe Engine - MySQL
• AP DataPipe Engine - File
• Python 3.12 DataPipe Engine

High Performance Computing

These objects provide open development environments where you can write and execute code to solve complex, nested problems and create customized processes tailored to your specific needs.

• PHP 7.4 Application
• Python 3.8 Advanced ML Application
• NodeJS16 Application