Data API

The Data API (dcupl.data) provides methods for managing data in dcupl. It supports various operations including setting, updating, upserting, and removing data.

Overview

The Data API offers different strategies for data management:

set: Replace all existing data for a model
update: Update specific items (requires matching keys)
upsert: Update existing or insert new items
remove: Remove specific items
reset: Clear all data for a model

All data operations are queued and processed when you call dcupl.init() or dcupl.update().

Methods

set()

Replaces all existing data for a model with the provided data.

Signature:

set(data: RawItem[], options: DataOptions): void

Parameters:

Parameter	Type	Required	Description
`data`	`RawItem[]`	Yes	Array of data items
`options`	`DataOptions`	Yes	Data operation options

Returns: void

Examples:

Basic set operation

dcupl.data.set(
  [
    { key: 'p1', name: 'Product 1', price: 100 },
    { key: 'p2', name: 'Product 2', price: 200 },
  ],
  { model: 'Product' }
);

With custom key property

dcupl.data.set(
  [
    { id: 'p1', name: 'Product 1' },
    { id: 'p2', name: 'Product 2' },
  ],
  {
    model: 'Product',
    keyProperty: 'id',
  }
);

With auto-generated keys

dcupl.data.set([{ name: 'Product 1' }, { name: 'Product 2' }], {
  model: 'Product',
  autoGenerateKey: true,
});

Replace all products

dcupl.data.set(newProducts, { model: 'Product' });
await dcupl.update();

update()

Updates specific items by their keys. Items must already exist in the model.

Signature:

update(data: RawItem[], options: DataOptions): void

Parameters:

Parameter	Type	Required	Description
`data`	`RawItem[]`	Yes	Array of items to update
`options`	`DataOptions`	Yes	Data operation options

Returns: void

Examples:

Update single item

dcupl.data.update([{ key: 'p1', price: 150 }], { model: 'Product' });

Update multiple items

dcupl.data.update(
  [
    { key: 'p1', price: 150, stock: 20 },
    { key: 'p2', price: 250, stock: 15 },
  ],
  { model: 'Product' }
);

Partial update - only specified fields

dcupl.data.update(
  [
    { key: 'p1', price: 150 }, // Only price is updated
  ],
  { model: 'Product' }
);
await dcupl.update();

Update with custom key property

dcupl.data.update([{ id: 'p1', price: 150 }], {
  model: 'Product',
  keyProperty: 'id',
});

upsert()

Updates existing items or inserts new ones. If an item with the key exists, it's updated; otherwise, it's inserted.

Signature:

upsert(data: RawItem[], options: DataOptions): void

Parameters:

Parameter	Type	Required	Description
`data`	`RawItem[]`	Yes	Array of items to upsert
`options`	`DataOptions`	Yes	Data operation options

Returns: void

Examples:

Upsert items (update if exists, insert if not)

dcupl.data.upsert(
  [
    { key: 'p1', name: 'Updated Product 1', price: 150 }, // Updates if exists
    { key: 'p4', name: 'New Product 4', price: 400 }, // Inserts if not exists
  ],
  { model: 'Product' }
);

Upsert with auto-key generation for new items

dcupl.data.upsert(
  [
    { key: 'p1', price: 150 },
    { name: 'New Product' }, // Gets auto-generated key
  ],
  {
    model: 'Product',
    autoGenerateKey: true,
  }
);

Safe data synchronization

dcupl.data.upsert(fetchedData, { model: 'Product' });
await dcupl.update();

remove()

Removes specific items from the model.

Signature:

remove(data: RawItem[], options: DataOptions): void

Parameters:

Parameter	Type	Required	Description
`data`	`RawItem[]`	Yes	Array of items to remove (only key needed)
`options`	`DataOptions`	Yes	Data operation options

Returns: void

Examples:

Remove single item

dcupl.data.remove([{ key: 'p1' }], { model: 'Product' });

Remove multiple items

dcupl.data.remove([{ key: 'p1' }, { key: 'p2' }, { key: 'p3' }], { model: 'Product' });

Remove with custom key property

dcupl.data.remove([{ id: 'p1' }], {
  model: 'Product',
  keyProperty: 'id',
});

Remove items and update

dcupl.data.remove(itemsToDelete, { model: 'Product' });
await dcupl.update();

reset()

Removes all data for a specific model.

Signature:

reset(options: { model: string }): void

Parameters:

Parameter	Type	Required	Description
`options`	`{ model: string }`	Yes	Options with model key

Returns: void

Examples:

Clear all products

dcupl.data.reset({ model: 'Product' });

Reset and reload

dcupl.data.reset({ model: 'Product' });
dcupl.data.set(newData, { model: 'Product' });
await dcupl.update();

Clear multiple models

dcupl.data.reset({ model: 'Product' });
dcupl.data.reset({ model: 'Category' });
await dcupl.update();

apply()

Applies a data container directly (advanced usage).

Signature:

apply(container: DataContainer): void

Parameters:

Parameter	Type	Required	Description
`container`	`DataContainer`	Yes	Data container to apply

Returns: void

Examples:

Apply custom container

const container: DataContainer = {
  model: 'Product',
  type: 'upsert',
  data: products,
  keyProperty: 'id',
};
dcupl.data.apply(container);

Advanced container with options

dcupl.data.apply({
  model: 'Product',
  type: 'update',
  data: updates,
  keyProperty: 'productId',
  autoGenerateKey: false,
  autoGenerateProperties: true,
});

Configuration Types

DataOptions

Configuration options for data operations.

Type Definition:

type DataOptions = {
  model: string;
  keyProperty?: string;
  autoGenerateKey?: boolean;
};

Properties:

Property	Type	Default	Description
`model`	`string`	-	Model key (required)
`keyProperty`	`string`	`'key'`	Property name to use as item key
`autoGenerateKey`	`boolean`	`false`	Auto-generate keys for items without one

Examples:

Basic options

{
  model: 'Product';
}

Custom key property

{
  model: 'Product',
  keyProperty: 'productId'
}

Auto-generate keys

{
  model: 'Product',
  autoGenerateKey: true
}

Complete options

{
  model: 'Product',
  keyProperty: 'id',
  autoGenerateKey: true
}

DataContainer

Advanced data container for direct application.

Type Definition:

type DataContainer = {
  model: string;
  type?: DataContainerType;
  data: RawItem[];
  keyProperty?: string;
  autoGenerateKey?: boolean;
  autoGenerateProperties?: boolean;
  placeholderUid?: string;
};

type DataContainerType = 'upsert' | 'update' | 'set' | 'remove';

Properties:

Property	Type	Description
`model`	`string`	Model key
`type`	`DataContainerType`	Operation type
`data`	`RawItem[]`	Data items
`keyProperty`	`string`	Key property name
`autoGenerateKey`	`boolean`	Auto-generate keys
`autoGenerateProperties`	`boolean`	Auto-generate properties from data
`placeholderUid`	`string`	Internal placeholder identifier

Data Operation Patterns

Initial Data Load

Loading data for the first time.

// Define models
dcupl.models.set({
  key: 'Product',
  properties: [
    { key: 'name', type: 'string' },
    { key: 'price', type: 'float' },
    { key: 'status', type: 'string' },
  ],
});

dcupl.models.set({
  key: 'Category',
  properties: [{ key: 'name', type: 'string' }],
});

// Load data
dcupl.data.set(products, { model: 'Product' });
dcupl.data.set(categories, { model: 'Category' });

// Initialize
await dcupl.init();

// Now data is ready to query

Incremental Updates

Updating data after initial load.

// Initial load
dcupl.data.set(initialProducts, { model: 'Product' });
await dcupl.init();

// Later: Update specific items
dcupl.data.update([{ key: 'p1', price: 150, stock: 20 }], { model: 'Product' });
await dcupl.update();

// The update is efficient and only processes changed items

Batch Operations

Performing multiple data operations in sequence.

// Queue multiple operations
dcupl.data.update(priceUpdates, { model: 'Product' });
dcupl.data.remove(discontinuedProducts, { model: 'Product' });
dcupl.data.upsert(newProducts, { model: 'Product' });

// Process all at once
await dcupl.update();

// All operations are applied in order

Sync from API

Synchronizing data from an external API.

async function syncProducts() {
  // Fetch from API
  const response = await fetch('/api/products');
  const products = await response.json();

  // Upsert (update existing, insert new)
  dcupl.data.upsert(products, { model: 'Product' });
  await dcupl.update();
}

// Sync periodically
setInterval(syncProducts, 60000);

Real-time Updates

Handling real-time data updates.

// WebSocket connection
const ws = new WebSocket('wss://api.example.com/products');

ws.onmessage = async (event) => {
  const update = JSON.parse(event.data);

  switch (update.type) {
    case 'create':
      dcupl.data.upsert([update.data], { model: 'Product' });
      break;
    case 'update':
      dcupl.data.update([update.data], { model: 'Product' });
      break;
    case 'delete':
      dcupl.data.remove([{ key: update.data.key }], { model: 'Product' });
      break;
  }

  await dcupl.update();
};

Partial Updates

Updating only specific fields efficiently.

// Update only price and stock fields
dcupl.data.update(
  [
    {
      key: 'p1',
      price: 150,
      stock: 20,
      // Other fields remain unchanged
    },
  ],
  { model: 'Product' }
);
await dcupl.update();

// Batch partial updates
const priceUpdates = products.map((p) => ({
  key: p.key,
  price: p.price * 1.1, // 10% increase
}));
dcupl.data.update(priceUpdates, { model: 'Product' });
await dcupl.update();

Data Migration

Migrating data between different schemas.

// Old data
const oldProducts = await fetchOldProducts();

// Transform to new schema
const newProducts = oldProducts.map((p) => ({
  key: p.id,
  name: p.productName,
  price: p.cost,
  category: p.categoryId,
  status: p.active ? 'active' : 'inactive',
}));

// Load transformed data
dcupl.data.set(newProducts, { model: 'Product' });
await dcupl.init();

Bulk Import

Importing large datasets efficiently.

// Import in batches
const batchSize = 1000;

for (let i = 0; i < largeDataset.length; i += batchSize) {
  const batch = largeDataset.slice(i, i + batchSize);

  if (i === 0) {
    dcupl.data.set(batch, { model: 'Product' });
    await dcupl.init();
  } else {
    dcupl.data.upsert(batch, { model: 'Product' });
    await dcupl.update();
  }
}

Auto-Key Generation

Working with data that doesn't have keys.

// Data without keys
const rawData = [
  { name: 'Product 1', price: 100 },
  { name: 'Product 2', price: 200 },
];

// Set with auto-generated keys
dcupl.data.set(rawData, {
  model: 'Product',
  autoGenerateKey: true,
});
await dcupl.init();

// Items now have auto-generated keys
const products = dcupl.query.execute({ modelKey: 'Product' });
console.log(products[0].key); // Auto-generated key

Custom Key Properties

Using different property names as keys.

// Data with 'id' instead of 'key'
const products = [
  { id: 'prod-1', name: 'Product 1', price: 100 },
  { id: 'prod-2', name: 'Product 2', price: 200 },
];

// Set with custom key property
dcupl.data.set(products, {
  model: 'Product',
  keyProperty: 'id',
});

// Or configure in model
dcupl.models.set({
  key: 'Product',
  keyProperty: 'id',
  properties: [
    { key: 'name', type: 'string' },
    { key: 'price', type: 'float' },
  ],
});

dcupl.data.set(products, { model: 'Product' });

Data Validation

Ensuring data quality on import.

// Import with validation enabled
const dcupl = new Dcupl({
  quality: {
    enabled: true,
  },
});

dcupl.models.set({
  key: 'Product',
  properties: [
    {
      key: 'name',
      type: 'string',
      quality: {
        required: true,
        validators: {
          minLength: { value: 3 },
        },
      },
    },
    {
      key: 'price',
      type: 'float',
      quality: {
        required: true,
        validators: {
          min: { value: 0 },
        },
      },
    },
  ],
  quality: {
    enabled: true,
  },
});

// Data is validated on set/update
dcupl.data.set(products, { model: 'Product' });
await dcupl.init();
// Invalid items trigger quality errors

Replace All Data

Completely replacing a model's data.

// Clear and replace
dcupl.data.set(newProducts, { model: 'Product' });
await dcupl.update();

// Alternative: explicit reset
dcupl.data.reset({ model: 'Product' });
dcupl.data.set(newProducts, { model: 'Product' });
await dcupl.update();

Conditional Updates

Updating data based on conditions.

// Fetch current state
const products = dcupl.query.execute({ modelKey: 'Product' });

// Prepare conditional updates
const updates = products
  .filter((p) => p.stock < 10)
  .map((p) => ({
    key: p.key,
    status: 'low_stock',
  }));

// Apply updates
dcupl.data.update(updates, { model: 'Product' });
await dcupl.update();

Complete Examples

E-Commerce Product Management

// Initial setup
const dcupl = new Dcupl();

dcupl.models.set({
  key: 'Product',
  properties: [
    { key: 'name', type: 'string' },
    { key: 'price', type: 'float' },
    { key: 'stock', type: 'int' },
    { key: 'status', type: 'string' },
  ],
  references: [{ key: 'category', type: 'singleValued', model: 'Category' }],
});

// Load products
dcupl.data.set(products, { model: 'Product' });
await dcupl.init();

// Update prices
const priceUpdates = [
  { key: 'p1', price: 150 },
  { key: 'p2', price: 250 },
];
dcupl.data.update(priceUpdates, { model: 'Product' });
await dcupl.update();

// Add new products
const newProducts = [{ key: 'p4', name: 'Product 4', price: 400, stock: 50, status: 'active' }];
dcupl.data.upsert(newProducts, { model: 'Product' });
await dcupl.update();

// Remove discontinued
dcupl.data.remove([{ key: 'p1' }], { model: 'Product' });
await dcupl.update();

Real-Time Dashboard

// Initialize with initial data
dcupl.data.set(initialMetrics, { model: 'Metric' });
await dcupl.init();

// Setup real-time updates
const eventSource = new EventSource('/api/metrics/stream');

eventSource.onmessage = async (event) => {
  const metric = JSON.parse(event.data);

  // Upsert new metrics
  dcupl.data.upsert([metric], { model: 'Metric' });
  await dcupl.update();

  // UI automatically updates via list subscriptions
};

Data Synchronization Service

class DataSyncService {
  private dcupl: Dcupl;
  private syncInterval: number;

  constructor(dcupl: Dcupl) {
    this.dcupl = dcupl;
  }

  async fullSync(model: string) {
    const data = await this.fetchAll(model);
    this.dcupl.data.set(data, { model });
    await this.dcupl.update();
  }

  async incrementalSync(model: string, lastSync: Date) {
    const updates = await this.fetchUpdates(model, lastSync);

    if (updates.created.length > 0) {
      this.dcupl.data.upsert(updates.created, { model });
    }

    if (updates.modified.length > 0) {
      this.dcupl.data.update(updates.modified, { model });
    }

    if (updates.deleted.length > 0) {
      this.dcupl.data.remove(
        updates.deleted.map((id) => ({ key: id })),
        { model }
      );
    }

    await this.dcupl.update();
  }

  startAutoSync(model: string, intervalMs: number) {
    let lastSync = new Date();

    this.syncInterval = setInterval(async () => {
      await this.incrementalSync(model, lastSync);
      lastSync = new Date();
    }, intervalMs);
  }

  stopAutoSync() {
    clearInterval(this.syncInterval);
  }

  private async fetchAll(model: string) {
    const response = await fetch(`/api/${model}`);
    return response.json();
  }

  private async fetchUpdates(model: string, since: Date) {
    const response = await fetch(`/api/${model}/updates?since=${since.toISOString()}`);
    return response.json();
  }
}

Offline-First Application

class OfflineDataManager {
  private dcupl: Dcupl;
  private pendingOperations: DataContainer[] = [];

  constructor(dcupl: Dcupl) {
    this.dcupl = dcupl;
    this.loadFromLocalStorage();
  }

  async create(model: string, item: any) {
    const operation = {
      model,
      type: 'upsert' as const,
      data: [item],
    };

    // Apply locally
    this.dcupl.data.apply(operation);
    await this.dcupl.update();

    // Queue for sync
    this.pendingOperations.push(operation);
    this.saveToLocalStorage();

    // Try to sync if online
    if (navigator.onLine) {
      await this.sync();
    }
  }

  async update(model: string, item: any) {
    const operation = {
      model,
      type: 'update' as const,
      data: [item],
    };

    this.dcupl.data.apply(operation);
    await this.dcupl.update();

    this.pendingOperations.push(operation);
    this.saveToLocalStorage();

    if (navigator.onLine) {
      await this.sync();
    }
  }

  async sync() {
    if (this.pendingOperations.length === 0) return;

    try {
      // Send pending operations to server
      await fetch('/api/sync', {
        method: 'POST',
        body: JSON.stringify(this.pendingOperations),
      });

      // Clear pending operations
      this.pendingOperations = [];
      this.saveToLocalStorage();
    } catch (error) {
      console.error('Sync failed:', error);
    }
  }

  private saveToLocalStorage() {
    localStorage.setItem('pendingOperations', JSON.stringify(this.pendingOperations));
  }

  private loadFromLocalStorage() {
    const saved = localStorage.getItem('pendingOperations');
    if (saved) {
      this.pendingOperations = JSON.parse(saved);
    }
  }
}

Performance Considerations

Partial Updates

dcupl automatically optimizes updates by detecting which items changed:

// Initial load
dcupl.data.set(largeDataset, { model: 'Product' });
await dcupl.init();

// Later: Update only 10 items out of 10,000
dcupl.data.update(smallUpdateBatch, { model: 'Product' });
await dcupl.update();
// Only processes the 10 changed items

Batch Operations

Combine multiple operations for better performance:

Less efficient: Multiple updates

dcupl.data.update([item1], { model: 'Product' });
await dcupl.update();
dcupl.data.update([item2], { model: 'Product' });
await dcupl.update();

More efficient: Batch update

dcupl.data.update([item1, item2], { model: 'Product' });
await dcupl.update();

Auto-Update Control

Control when lists update:

Disable auto-update for bulk operations

const dcupl = new Dcupl({
  performance: {
    autoUpdateLists: { enabled: false },
  },
});

Perform bulk updates

dcupl.data.update(batch1, { model: 'Product' });
dcupl.data.update(batch2, { model: 'Product' });
await dcupl.update();

Manually update lists when ready

myList.update();

Data API

Overview

Methods

set()

update()

upsert()

remove()

reset()

apply()

Configuration Types

DataOptions

DataContainer

Data Operation Patterns

Initial Data Load

Incremental Updates

Batch Operations

Sync from API

Real-time Updates

Partial Updates

Data Migration

Bulk Import

Auto-Key Generation

Custom Key Properties

Data Validation

Replace All Data

Conditional Updates

Complete Examples

E-Commerce Product Management

Real-Time Dashboard

Data Synchronization Service

Offline-First Application

Performance Considerations

Partial Updates

Batch Operations

Auto-Update Control

See Also