dcupl Console

The dcupl Console is a web-based platform for managing your data infrastructure. It provides visual tools for designing data models, building automated workflows, deploying REST APIs, and leveraging AI-powered features.

Architecture Overview

flowchart TB
  subgraph Browser["Browser"]
    SDK["dcupl SDK"]
    Console["Console Frontend
(Angular)"]
  end

  subgraph Backend["Backend Services"]
    API["Console API
(NestJS)"]
    DB[(Firestore)]
  end

  subgraph Runner["Workflow Runner"]
    Engine["V3 Execution Engine"]
    RAPI["RAPI Endpoints"]
  end

  SDK <-->|postMessage| Console
  Console <-->|REST API| API
  API <--> DB
  API -->|Deploy Workflows| Runner
  SDK -->|LoaderConfiguration| CDN[(CDN)]
  CDN --> Runner

The Console consists of three main components:

Console Frontend - Angular-based web application for visual model design, workflow building, and project management
Console API - NestJS backend handling authentication, project storage, and workflow deployment
Workflow Runner - Execution engine for workflows and RAPI (REST API) endpoints

Core Features

Data Modeling

Create and manage your data structures visually with the Model Editor:

Visual model designer with drag-and-drop property creation
Property types including strings, numbers, dates, arrays, and references
Validation rules for data quality
AI-powered model generation from sample data

The Model Editor supports both visual form editing and JSON view for full control over your data model definitions.

Workflows

Automate data processing and business logic with the V3 workflow system:

flowchart LR
  T[Trigger] --> N1[Fetch Data]
  N1 --> N2[Transform]
  N2 --> N3{Validate}
  N3 -->|Valid| N4[Save]
  N3 -->|Invalid| N5[Log Error]

Graph-based visual editor for workflow design
Multiple node types including HTTP requests, scripts, and file operations
Parallel execution for improved performance
90x memory improvement with item-based data flow

RAPI (REST API)

Deploy auto-generated REST APIs from your data models:

Automatic endpoint generation based on dcupl models
OpenAPI/Swagger documentation for API exploration
API key authentication with configurable security
v-Guard caching for version-based cache invalidation

AI Features

Leverage AI to accelerate development:

Model generation from sample CSV/JSON data
Data quality linting with AI-powered recommendations
Workflow proposals from natural language descriptions
Mock data generation for testing

Console Workflow

flowchart TD
  A[Create Project] --> B[Define Models]
  B --> C[Load Data]
  C --> D{Choose Path}
  D -->|SDK Only| E[Use dcupl SDK]
  D -->|Automation| F[Build Workflows]
  D -->|API Access| G[Deploy RAPI]
  F --> H[Deploy to Runner]
  G --> H
  H --> I[Monitor & Manage]

Key Concepts

Projects

Projects are the top-level container for all your dcupl resources:

Models - Data structure definitions
Data files - CSV, JSON, or external data sources
Workflows - Automation logic
RAPIs - REST API deployments
Variables - Environment-specific configuration

File Versioning

The Console uses a draft/published versioning system:

Draft - Work-in-progress changes visible only in Console
Published - Stable version deployed to CDN and accessible by SDK

Authentication

Console uses Firebase Authentication with project-level authorization:

Role	Permissions
Owner	Full access, billing, delete project
Editor	Create, edit, deploy resources
Viewer	Read-only access to project resources

Getting Started

Create a project at console.dcupl.com
Define your models using the visual Model Editor
Upload data via CSV, JSON, or connect external sources
Build workflows for data processing and automation
Deploy RAPIs to expose your data via REST endpoints

What's Next?

Workflows - Build automated data processing pipelines
RAPI - Deploy and manage REST APIs
AI Features - Leverage AI for model and workflow generation
SDK Integration - Connect the SDK to your Console project