AI & Machine Learning Overview

The Reasoning Flows layer dedicated to intelligent systems, model training, and generative AI.

---

Purpose

The AI & Machine Learning objects in Reasoning Flows empower users to build, deploy, and leverage intelligent systems directly within their workflows. These tools support a wide range of use cases — from generating embeddings and training predictive models to real-time inference, time-series forecasting, and generative AI applications.

With both GUI-based and GPU-accelerated options, teams can move seamlessly from data preparation to advanced analytics and generative reasoning.

---

Where It Fits in Reasoning Flows

In the Reasoning Flows architecture:

1. Extract & Load brings data into the platform.
2. Repository Tables register datasets for reuse.
3. Transform & Prepare cleans and structures data for modeling.
4. AI & Machine Learning builds, trains, and deploys intelligent models.
5. Reasoning Atlas integrates model outputs into ARPIA's Generative AI and Semantic reasoning systems.

Goal: This stage turns structured and prepared data into intelligent models, predictions, and AI-driven applications.

---

Available Tools

AutoML

GUI-based machine learning tools for building models without writing code.

AP AutoML Engine

Low-code tool for training ML models on structured datasets with zero-code setup.

Capabilities:

• Classification (predict categories)
• Regression (predict numeric values)
• Basic time-series forecasting

Best for:

• Rapid prototyping
• Business users without ML expertise
• Standard prediction tasks on tabular data

---

SingularAI Text Embeddings

Generates semantic vector embeddings from raw text, enabling semantic search, classification, and clustering.

Capabilities:

• Convert text to numerical vectors that capture meaning
• Enable similarity-based search
• Support text classification and clustering

Best for:

• Natural Language Processing (NLP) tasks
• Retrieval-Augmented Generation (RAG) workflows
• Semantic search implementations
• Text similarity analysis

Example use case:
Convert product descriptions into embeddings, then find similar products based on meaning rather than exact keyword matches.

---

ARPIA Sequential GenAI Worker

Framework for building sequential generative AI workflows where steps execute in order.

Capabilities:

• Chain multiple LLM calls in sequence
• Pass outputs from one step as inputs to the next
• Build multi-step reasoning pipelines

Best for:

• Step-by-step document processing
• Sequential reasoning tasks
• Pipelines where each step depends on the previous

---

ARPIA WorkFlow GenAI Worker

Framework for building complex generative AI workflows with branching and parallel execution.

Capabilities:

• Orchestrate multiple LLM components
• Support conditional branching
• Enable parallel processing paths

Best for:

• Complex AI applications with multiple paths
• Chat assistants with routing logic
• Content generation pipelines
• Summarization and analysis workflows

---

AutoML GPU

GPU-accelerated machine learning for compute-intensive tasks.

AP AutoML GPU Engine

GPU-accelerated version of the standard AutoML Engine for larger datasets and more complex models.

Capabilities:

• Same as AP AutoML Engine but with GPU acceleration
• Faster training on large datasets
• Support for more complex model architectures

Best for:

• Datasets exceeding 100,000 rows
• Deep learning models
• When standard AutoML training time is too slow

---

AutoML TimeSeries GPU

Specialized GPU-accelerated tools for time-series forecasting.

AP AutoML TimeSeries GPU Engine

GPU-accelerated engine specifically optimized for time-series prediction tasks.

Capabilities:

• Forecast future values based on historical patterns
• Handle seasonality and trends
• Process multiple time series simultaneously

Best for:

• Sales forecasting
• Demand prediction
• Financial time-series analysis
• Any prediction task where historical sequence matters

---

Image Analyzer

Computer vision tools for image classification and analysis.

AP AutoML Vision Predictor

GUI-driven object that classifies images based on visual content. References image URLs stored in Repository Tables.

Capabilities:

• Image classification
• Content tagging
• Visual pattern recognition

Best for:

• Product image categorization
• Quality control validation
• Content moderation
• Any task requiring automated image analysis

---

Choosing the Right Tool

---

When to Use GPU

GPU-accelerated engines provide significant performance benefits but consume more resources. Use this guide:

---

Typical Workflow Examples

Example 1: Sales Prediction Model

sales_gold (GOLD)
    │
    ▼
AP AutoML Engine
    • Target: net_sales
    • Features: category, region, month
    • Type: Regression
    │
    ▼
sales_model (trained model)
    │
    ▼
sales_predictions_optimized (OPTIMIZED)

Example 2: Semantic Search with RAG

documents_clean (CLEAN)
    │
    ▼
SingularAI Text Embeddings
    • Generate vectors for each document
    │
    ▼
documents_embeddings_optimized (OPTIMIZED)
    │
    ▼
Reasoning Atlas
    • Enable semantic search
    • Power LLM-based Q&A

Example 3: Time-Series Forecasting

historical_sales_gold (GOLD)
    │
    ▼
AP AutoML TimeSeries GPU Engine
    • Time column: date
    • Target: units_sold
    • Forecast horizon: 30 days
    │
    ▼
sales_forecast_optimized (OPTIMIZED)

Example 4: Generative AI Document Processing

documents_raw (RAW)
    │
    ▼
ARPIA Sequential GenAI Worker
    • Step 1: Extract key information
    • Step 2: Summarize content
    • Step 3: Generate recommendations
    │
    ▼
processed_documents_gold (GOLD)

---

Integration with Data Layers

AI & ML objects consume and produce data at specific layers:

Important: Only GOLD and OPTIMIZED tables are indexed in the Knowledge Catalog for reasoning and LLM-powered exploration.

---

Best Practices

Prepare data before training. Use Transform & Prepare objects to clean and structure data. Models perform better on high-quality GOLD layer data.

Start with standard AutoML. Use GPU engines only when standard engines are too slow or dataset size requires it.

Tag outputs appropriately. Model predictions and embeddings should be tagged as OPTIMIZED and documented in the Knowledge Catalog.

Version your models. Keep track of which data and parameters produced each model for reproducibility.

Monitor model performance. Regularly validate that models continue to perform well as new data arrives.

Document in the Knowledge Catalog. Register model outputs as Nodes so they're discoverable and governed.

---

Related Documentation

• Transform & Prepare
  Learn how data is refined before model training.

• Repository Tables
  Understand how structured data is managed and shared across workflows.

• ARPIA Data Layer Framework
  Review the governance standards for RAW, CLEAN, GOLD, and OPTIMIZED datasets.

• Reasoning Atlas Overview
  Discover how AI and ML outputs integrate into ARPIA's Semantic Reasoning and Generative AI ecosystem.