Extract & Load in Arpia — Knowledge Base Overview

Arpia’s Extract & Load (E&L) framework simplifies moving data from where it lives (databases, APIs, or files) to where your team needs it (tables, apps, or analysis). These tools automate extraction, transformation, and loading (ETL/ELT) so that:

Data stays consistent and accurate across systems.
Teams spend less time on prep work and more on analysis and insights.
Workflows can scale from simple data copies to complex, logic-heavy applications.

Tool Categories

Arpia provides several tool types for Extract & Load, depending on the complexity and purpose of your task.

1. AP DataPipe Engines (MySQL / File)

Purpose: Quick, no-code data movement.
Interface: Form-based configuration in the UI.
Best for: Copying database tables or files directly into destination tables.
Use cases: Simple data transfers, column selection, light filtering, or field mapping.

2. Python 3.12 DataPipe Engine

Purpose: Flexible pipelines with Python scripting.
Interface: Python code plus configurations.
Best for: Adding custom logic or transformations during extract & load.
Use cases: Data cleaning, conditional transformations, API calls before load, merging multiple sources.

3. High Performance Computing (HPC) Applications

Purpose: Full development environments for custom workloads.
Interface: Scripting in PHP or Python with full access.
Best for: Advanced logic, app-like processes, or heavy compute tasks.
Use cases: Training ML models, running pipelines, building APIs or webhooks, external integrations.

4. Arpia Notebook

Purpose: Interactive, browser-based exploration.
Interface: Notebook-style coding with Markdown + Python support.
Best for: Rapid prototyping, testing, and visual analysis.
Use cases: Exploratory data analysis, testing queries or scripts, building custom visualizations, documenting workflows.

Quick Decision Guide

If you need to…	Use this tool
Copy tables/files quickly with no code	AP DataPipe (MySQL/File)
Transform or clean data while loading	Python 3.12 DataPipe
Run ML models, APIs, or custom apps	HPC Applications
Explore, test, or visualize interactively	Arpia Notebook

How They Fit Together

Start in a Notebook → Explore data, test ideas, visualize results.
Move to Python 3.12 DataPipe → Turn working logic into a repeatable, automated pipeline.
Use AP DataPipe (MySQL/File) → When you only need straightforward data ingestion.
Deploy HPC Applications → For production-grade apps, integrations, or machine learning workloads.

By aligning the tool choice with the task, teams can keep workflows simple where possible and powerful where necessary.