Quality & Validation
dcupl provides a comprehensive quality and validation system to ensure data integrity, enforce business rules, and catch errors early. Configure validation rules at both the model and property level.
What is Quality & Validation?
Quality configuration enables:
- Required fields - Ensure critical data is present
- Nullable control - Allow or prevent null/undefined values
- Type enforcement - Strict or loose data type checking
- Custom validators - Email, min/max, pattern matching, and more
- Error tracking - Collect and report validation errors
- Quality reports - Analyze data quality metrics
Enabling Quality Validation
Quality validation uses a two-tier system:
Global Enable (dcupl level)
import { Dcupl } from '@dcupl/core';
const dcupl = new Dcupl({
quality: {
enabled: true, // Enable quality tracking globally
},
});Model Enable (model level)
const userModel: ModelDefinition = {
key: 'User',
quality: {
enabled: true, // Enable validation for this model
},
properties: [
// ... properties with validation rules
],
};Both must be true for validation to occur.
Property Quality Configuration
Basic Quality Options
{
key: 'email',
type: 'string',
quality: {
required: boolean, // Field must be present
nullable: boolean, // Field can be null/undefined
forceStrictDataType: boolean, // Strict type checking
validators: {}, // Validation rules
validatorHandling: 'loose' | 'strict', // Error handling
},
}Required Fields
Mark fields as required:
const userModel: ModelDefinition = {
key: 'User',
quality: { enabled: true },
properties: [
{
key: 'email',
type: 'string',
quality: {
required: true, // Email is required
},
},
{
key: 'username',
type: 'string',
quality: {
required: true,
},
},
{
key: 'bio',
type: 'string',
quality: {
required: false, // Bio is optional
},
},
],
data: [
{ key: 'u1', email: 'alice@example.com', username: 'alice' }, // ✅ Valid
{ key: 'u2', username: 'bob' }, // ❌ Missing required email
],
};
dcupl.models.set(userModel);
await dcupl.init();
// Check for errors
const errors = dcupl.quality.getModelErrors('User');
console.log(errors);
// [{ errorType: 'UndefinedValue', itemKey: 'u2', attribute: 'email', ... }]Nullable Fields
Control null/undefined values:
const productModel: ModelDefinition = {
key: 'Product',
quality: { enabled: true },
properties: [
{
key: 'name',
type: 'string',
quality: {
required: true,
nullable: false, // Cannot be null
},
},
{
key: 'description',
type: 'string',
quality: {
nullable: true, // Can be null
},
},
],
data: [
{ key: 'p1', name: 'Laptop', description: null }, // ✅ Valid
{ key: 'p2', name: null }, // ❌ name cannot be null
],
};Strict Data Type Checking
Enforce exact data types:
const orderModel: ModelDefinition = {
key: 'Order',
quality: { enabled: true },
properties: [
{
key: 'quantity',
type: 'int',
quality: {
forceStrictDataType: true, // Must be exact int
},
},
],
data: [
{ key: 'o1', quantity: 5 }, // ✅ Valid int
{ key: 'o2', quantity: '5' }, // ❌ String not allowed
{ key: 'o3', quantity: 5.5 }, // ❌ Float not allowed
],
};Validators
Email Validator
Validate email format:
const contactModel: ModelDefinition = {
key: 'Contact',
quality: { enabled: true },
properties: [
{
key: 'email',
type: 'string',
quality: {
required: true,
validators: {
email: {}, // Email format validation
},
},
},
],
data: [
{ key: 'c1', email: 'alice@example.com' }, // ✅ Valid
{ key: 'c2', email: 'invalid-email' }, // ❌ Invalid format
{ key: 'c3', email: 'bob@domain' }, // ❌ Invalid format
],
};
dcupl.models.set(contactModel);
await dcupl.init();
const errors = dcupl.quality.getModelErrors('Contact');
console.log(errors.filter((e) => e.errorType === 'InvalidValidator'));Min/Max Validators
Validate numeric ranges:
const productModel: ModelDefinition = {
key: 'Product',
quality: { enabled: true },
properties: [
{
key: 'price',
type: 'float',
quality: {
validators: {
min: { value: 0 }, // Price >= 0
max: { value: 10000 }, // Price <= 10000
},
},
},
{
key: 'rating',
type: 'float',
quality: {
validators: {
min: { value: 1 },
max: { value: 5 },
},
},
},
],
data: [
{ key: 'p1', price: 99.99, rating: 4.5 }, // ✅ Valid
{ key: 'p2', price: -10, rating: 3 }, // ❌ price < 0
{ key: 'p3', price: 500, rating: 6 }, // ❌ rating > 5
],
};MinLength/MaxLength Validators
Validate string length:
const userModel: ModelDefinition = {
key: 'User',
quality: { enabled: true },
properties: [
{
key: 'username',
type: 'string',
quality: {
validators: {
minLength: { value: 3 }, // At least 3 characters
maxLength: { value: 20 }, // At most 20 characters
},
},
},
{
key: 'password',
type: 'string',
quality: {
validators: {
minLength: { value: 8 }, // At least 8 characters
},
},
},
],
data: [
{ key: 'u1', username: 'alice', password: 'secure123' }, // ✅ Valid
{ key: 'u2', username: 'ab', password: '123' }, // ❌ Both too short
],
};Pattern Validator
Validate against regex patterns:
const productModel: ModelDefinition = {
key: 'Product',
quality: { enabled: true },
properties: [
{
key: 'sku',
type: 'string',
quality: {
validators: {
pattern: {
value: '^[A-Z]{3}-[0-9]{4}, // Format: ABC-1234
},
},
},
},
{
key: 'zipCode',
type: 'string',
quality: {
validators: {
pattern: {
value: '^\\d{5}(-\\d{4})?, // 5 or 9 digit zip
},
},
},
},
],
data: [
{ key: 'p1', sku: 'ABC-1234', zipCode: '94102' }, // ✅ Valid
{ key: 'p2', sku: 'invalid', zipCode: '1234' }, // ❌ Both invalid
],
};Enum Validator
Restrict to specific values:
const orderModel: ModelDefinition = {
key: 'Order',
quality: { enabled: true },
properties: [
{
key: 'status',
type: 'string',
quality: {
validators: {
enum: {
value: ['pending', 'processing', 'shipped', 'delivered', 'cancelled'],
},
},
},
},
{
key: 'priority',
type: 'string',
quality: {
validators: {
enum: {
value: ['low', 'medium', 'high', 'critical'],
},
},
},
},
],
data: [
{ key: 'o1', status: 'pending', priority: 'high' }, // ✅ Valid
{ key: 'o2', status: 'invalid', priority: 'urgent' }, // ❌ Both invalid
],
};StartsWith/EndsWith Validators
const fileModel: ModelDefinition = {
key: 'File',
quality: { enabled: true },
properties: [
{
key: 'filename',
type: 'string',
quality: {
validators: {
startsWith: { value: 'report_' }, // Must start with 'report_'
endsWith: { value: '.pdf' }, // Must end with '.pdf'
},
},
},
],
data: [
{ key: 'f1', filename: 'report_2025.pdf' }, // ✅ Valid
{ key: 'f2', filename: 'document.pdf' }, // ❌ Wrong prefix
{ key: 'f3', filename: 'report_2024.txt' }, // ❌ Wrong extension
],
};Includes Validator
Check if string contains substring:
const productModel: ModelDefinition = {
key: 'Product',
quality: { enabled: true },
properties: [
{
key: 'tags',
type: 'string',
quality: {
validators: {
includes: { value: 'approved' }, // Must contain 'approved'
},
},
},
],
data: [
{ key: 'p1', tags: 'new, approved, featured' }, // ✅ Valid
{ key: 'p2', tags: 'new, pending' }, // ❌ Missing 'approved'
],
};Unique Validator
Ensure unique values across all items:
const userModel: ModelDefinition = {
key: 'User',
quality: { enabled: true },
properties: [
{
key: 'email',
type: 'string',
quality: {
validators: {
unique: {}, // Email must be unique
},
},
},
{
key: 'username',
type: 'string',
quality: {
validators: {
unique: {},
},
},
},
],
data: [
{ key: 'u1', email: 'alice@example.com', username: 'alice' }, // ✅ Valid
{ key: 'u2', email: 'alice@example.com', username: 'bob' }, // ❌ Duplicate email
],
};Multiple Validators
Combine multiple validators:
const userModel: ModelDefinition = {
key: 'User',
quality: { enabled: true },
properties: [
{
key: 'email',
type: 'string',
quality: {
required: true,
nullable: false,
validators: {
email: {},
unique: {},
minLength: { value: 5 },
},
},
},
{
key: 'age',
type: 'int',
quality: {
required: true,
validators: {
min: { value: 18 },
max: { value: 120 },
},
},
},
],
};Error Handling
Getting Errors
await dcupl.init();
// Get all errors
const allErrors = dcupl.quality.getErrors();
// Get errors for specific model
const userErrors = dcupl.quality.getModelErrors('User');
// Get specific error
const error = dcupl.quality.getError('error-key');
console.log(userErrors);
// [
// {
// type: 'model',
// errorType: 'UndefinedValue',
// errorGroup: 'PropertyDataError',
// model: 'User',
// itemKey: 'u1',
// attribute: 'email',
// title: 'Undefined Value',
// description: 'Required field is missing'
// },
// ...
// ]Error Types
Common error types:
UndefinedValue- Required field is missingNullValue- Field is null when nullable: falseWrongDataType- Type mismatch with forceStrictDataTypeInvalidValidator- Validator rule failedNonUniqueKey- Duplicate value when unique validator usedMissingModel- Referenced model doesn't existMissingProperty- Property doesn't existMissingReference- Reference doesn't exist
Resetting Errors
// Reset all errors
dcupl.quality.resetErrors();
// Reset model errors only
dcupl.quality.resetModelErrors();
// Reset validation errors only
dcupl.quality.resetValidationErrors();Quality Reports
Analytics Values
Get quality metrics:
await dcupl.init();
const analytics = dcupl.quality.values;
console.log(analytics);
// {
// loading: 150, // Load time in ms
// processing: 50, // Processing time in ms
// startup: 200, // Total startup time
// counts: {
// models: 5,
// rows: 1000,
// attributes: 50,
// dataEntries: 1000
// },
// errors: {
// size: 5,
// UndefinedValue: 2,
// InvalidValidator: 3
// },
// sizes: {
// totalEncoded: 50000,
// totalDecoded: 75000,
// unusedDecoded: 5000
// }
// }Unused Attributes
Find unused attributes:
const unusedData = dcupl.quality.calculateUnusedData();
console.log(dcupl.quality.values.unusedAttributes);
// [
// { model: 'Product', attribute: 'legacyField' },
// { model: 'User', attribute: 'oldField' }
// ]Practical Examples
Example 1: User Registration Form
const userModel: ModelDefinition = {
key: 'User',
quality: { enabled: true },
properties: [
{
key: 'email',
type: 'string',
quality: {
required: true,
nullable: false,
validators: {
email: {},
unique: {},
minLength: { value: 5 },
maxLength: { value: 100 },
},
},
},
{
key: 'username',
type: 'string',
quality: {
required: true,
validators: {
unique: {},
minLength: { value: 3 },
maxLength: { value: 20 },
pattern: { value: '^[a-zA-Z0-9_]+ },
},
},
},
{
key: 'age',
type: 'int',
quality: {
required: true,
validators: {
min: { value: 18 },
max: { value: 120 },
},
},
},
],
};Example 2: Product Catalog
const productModel: ModelDefinition = {
key: 'Product',
quality: { enabled: true },
properties: [
{
key: 'sku',
type: 'string',
quality: {
required: true,
validators: {
unique: {},
pattern: { value: '^[A-Z]{3}-[0-9]{4} },
},
},
},
{
key: 'name',
type: 'string',
quality: {
required: true,
validators: {
minLength: { value: 3 },
maxLength: { value: 100 },
},
},
},
{
key: 'price',
type: 'float',
quality: {
required: true,
validators: {
min: { value: 0.01 },
max: { value: 999999.99 },
},
},
},
{
key: 'stock',
type: 'int',
quality: {
required: true,
validators: {
min: { value: 0 },
},
},
},
],
};Example 3: Order Management
const orderModel: ModelDefinition = {
key: 'Order',
quality: { enabled: true },
properties: [
{
key: 'orderNumber',
type: 'string',
quality: {
required: true,
validators: {
unique: {},
pattern: { value: '^ORD-[0-9]{6} },
},
},
},
{
key: 'status',
type: 'string',
quality: {
required: true,
validators: {
enum: {
value: ['pending', 'processing', 'shipped', 'delivered', 'cancelled'],
},
},
},
},
{
key: 'total',
type: 'float',
quality: {
required: true,
validators: {
min: { value: 0 },
},
},
},
],
};Example 4: Contact Information
const contactModel: ModelDefinition = {
key: 'Contact',
quality: { enabled: true },
properties: [
{
key: 'email',
type: 'string',
quality: {
required: true,
validators: {
email: {},
},
},
},
{
key: 'phone',
type: 'string',
quality: {
validators: {
pattern: { value: '^\\+?[1-9]\\d{1,14} }, // E.164 format
},
},
},
{
key: 'zipCode',
type: 'string',
quality: {
validators: {
pattern: { value: '^\\d{5}(-\\d{4})? },
},
},
},
],
};Example 5: Employee Records
const employeeModel: ModelDefinition = {
key: 'Employee',
quality: { enabled: true },
properties: [
{
key: 'employeeId',
type: 'string',
quality: {
required: true,
validators: {
unique: {},
pattern: { value: '^EMP-[0-9]{5} },
},
},
},
{
key: 'email',
type: 'string',
quality: {
required: true,
validators: {
email: {},
unique: {},
endsWith: { value: '@company.com' },
},
},
},
{
key: 'department',
type: 'string',
quality: {
required: true,
validators: {
enum: {
value: ['Engineering', 'Sales', 'Marketing', 'HR', 'Finance'],
},
},
},
},
],
};Example 6: Blog Articles
const articleModel: ModelDefinition = {
key: 'Article',
quality: { enabled: true },
properties: [
{
key: 'slug',
type: 'string',
quality: {
required: true,
validators: {
unique: {},
pattern: { value: '^[a-z0-9-]+ },
minLength: { value: 3 },
maxLength: { value: 100 },
},
},
},
{
key: 'title',
type: 'string',
quality: {
required: true,
validators: {
minLength: { value: 10 },
maxLength: { value: 200 },
},
},
},
{
key: 'content',
type: 'string',
quality: {
required: true,
validators: {
minLength: { value: 100 },
},
},
},
],
};Example 7: File Uploads
const fileModel: ModelDefinition = {
key: 'File',
quality: { enabled: true },
properties: [
{
key: 'filename',
type: 'string',
quality: {
required: true,
validators: {
pattern: { value: '^[a-zA-Z0-9_.-]+ },
maxLength: { value: 255 },
},
},
},
{
key: 'extension',
type: 'string',
quality: {
required: true,
validators: {
enum: {
value: ['jpg', 'jpeg', 'png', 'gif', 'pdf', 'doc', 'docx'],
},
},
},
},
{
key: 'size',
type: 'int',
quality: {
required: true,
validators: {
min: { value: 1 },
max: { value: 10485760 }, // 10MB max
},
},
},
],
};Example 8: Financial Transactions
const transactionModel: ModelDefinition = {
key: 'Transaction',
quality: { enabled: true },
properties: [
{
key: 'transactionId',
type: 'string',
quality: {
required: true,
validators: {
unique: {},
pattern: { value: '^TXN-[0-9]{10} },
},
},
},
{
key: 'amount',
type: 'float',
quality: {
required: true,
forceStrictDataType: true,
validators: {
min: { value: 0.01 },
max: { value: 1000000 },
},
},
},
{
key: 'currency',
type: 'string',
quality: {
required: true,
validators: {
enum: { value: ['USD', 'EUR', 'GBP', 'JPY'] },
pattern: { value: '^[A-Z]{3} },
},
},
},
],
};Example 9: Inventory Items
const inventoryModel: ModelDefinition = {
key: 'Inventory',
quality: { enabled: true },
properties: [
{
key: 'barcode',
type: 'string',
quality: {
required: true,
validators: {
unique: {},
pattern: { value: '^[0-9]{12,13} },
},
},
},
{
key: 'quantity',
type: 'int',
quality: {
required: true,
forceStrictDataType: true,
validators: {
min: { value: 0 },
},
},
},
{
key: 'location',
type: 'string',
quality: {
required: true,
validators: {
pattern: { value: '^[A-Z]{1,2}-[0-9]{1,3}-[0-9]{1,3} },
},
},
},
],
};Example 10: Medical Records
const patientModel: ModelDefinition = {
key: 'Patient',
quality: { enabled: true },
properties: [
{
key: 'mrn',
type: 'string',
quality: {
required: true,
validators: {
unique: {},
pattern: { value: '^MRN-[0-9]{8} },
},
},
},
{
key: 'ssn',
type: 'string',
quality: {
validators: {
pattern: { value: '^\\d{3}-\\d{2}-\\d{4} },
},
},
},
{
key: 'bloodType',
type: 'string',
quality: {
validators: {
enum: {
value: ['A+', 'A-', 'B+', 'B-', 'AB+', 'AB-', 'O+', 'O-'],
},
},
},
},
],
};More Examples
Example 11: Vehicle Registration
const vehicleModel: ModelDefinition = {
key: 'Vehicle',
quality: { enabled: true },
properties: [
{
key: 'vin',
type: 'string',
quality: {
required: true,
validators: {
unique: {},
pattern: { value: '^[A-HJ-NPR-Z0-9]{17} },
minLength: { value: 17 },
maxLength: { value: 17 },
},
},
},
{
key: 'licensePlate',
type: 'string',
quality: {
required: true,
validators: {
unique: {},
minLength: { value: 2 },
maxLength: { value: 8 },
},
},
},
{
key: 'year',
type: 'int',
quality: {
required: true,
validators: {
min: { value: 1900 },
max: { value: 2030 },
},
},
},
],
};Example 12: Course Enrollment
const enrollmentModel: ModelDefinition = {
key: 'Enrollment',
quality: { enabled: true },
properties: [
{
key: 'studentId',
type: 'string',
quality: {
required: true,
validators: {
pattern: { value: '^STU-[0-9]{6} },
},
},
},
{
key: 'courseCode',
type: 'string',
quality: {
required: true,
validators: {
pattern: { value: '^[A-Z]{2,4}[0-9]{3} },
},
},
},
{
key: 'grade',
type: 'string',
quality: {
validators: {
enum: {
value: ['A', 'A-', 'B+', 'B', 'B-', 'C+', 'C', 'C-', 'D', 'F'],
},
},
},
},
],
};Example 13: Real Estate Listings
const propertyModel: ModelDefinition = {
key: 'Property',
quality: { enabled: true },
properties: [
{
key: 'mls',
type: 'string',
quality: {
required: true,
validators: {
unique: {},
pattern: { value: '^MLS-[0-9]{8} },
},
},
},
{
key: 'price',
type: 'int',
quality: {
required: true,
validators: {
min: { value: 1000 },
max: { value: 100000000 },
},
},
},
{
key: 'bedrooms',
type: 'int',
quality: {
required: true,
validators: {
min: { value: 0 },
max: { value: 20 },
},
},
},
{
key: 'bathrooms',
type: 'float',
quality: {
required: true,
validators: {
min: { value: 0.5 },
max: { value: 20 },
},
},
},
],
};Example 14: Event Tickets
const ticketModel: ModelDefinition = {
key: 'Ticket',
quality: { enabled: true },
properties: [
{
key: 'ticketNumber',
type: 'string',
quality: {
required: true,
validators: {
unique: {},
pattern: { value: '^TKT-[0-9]{10} },
},
},
},
{
key: 'ticketType',
type: 'string',
quality: {
required: true,
validators: {
enum: {
value: ['general', 'vip', 'backstage', 'student', 'senior'],
},
},
},
},
{
key: 'price',
type: 'float',
quality: {
required: true,
validators: {
min: { value: 0 },
},
},
},
],
};Example 15: Social Media Posts
const postModel: ModelDefinition = {
key: 'Post',
quality: { enabled: true },
properties: [
{
key: 'content',
type: 'string',
quality: {
required: true,
validators: {
minLength: { value: 1 },
maxLength: { value: 280 }, // Twitter-style limit
},
},
},
{
key: 'hashtags',
type: 'string',
quality: {
validators: {
pattern: { value: '^(#[a-zA-Z0-9_]+\\s*)* },
},
},
},
{
key: 'visibility',
type: 'string',
quality: {
required: true,
validators: {
enum: {
value: ['public', 'followers', 'private'],
},
},
},
},
],
};Form Validation Integration
Use quality validation with forms:
function UserForm() {
const [formData, setFormData] = useState({});
const [errors, setErrors] = useState([]);
const validateForm = async () => {
const dcupl = new Dcupl({
quality: { enabled: true },
});
dcupl.models.set({
key: 'User',
quality: { enabled: true },
properties: [
{
key: 'email',
type: 'string',
quality: {
required: true,
validators: { email: {} },
},
},
],
data: [formData],
});
await dcupl.init();
const validationErrors = dcupl.quality.getModelErrors('User');
setErrors(validationErrors);
return validationErrors.length === 0;
};
const handleSubmit = async (e) => {
e.preventDefault();
if (await validateForm()) {
// Form is valid, submit
}
};
return (
<form onSubmit={handleSubmit}>
<input name="email" onChange={e => setFormData({...formData, email: e.target.value})} />
{errors.map(err => <div key={err.attribute}>{err.description}</div>)}
<button type="submit">Submit</button>
</form>
);
}Data Import Validation
Validate imported data:
async function importData(csvData) {
const dcupl = new Dcupl({
quality: { enabled: true },
});
dcupl.models.set({
key: 'Product',
quality: { enabled: true },
properties: [
{
key: 'sku',
type: 'string',
quality: {
required: true,
validators: {
unique: {},
pattern: { value: '^[A-Z]{3}-[0-9]{4} },
},
},
},
],
data: csvData,
});
await dcupl.init();
const errors = dcupl.quality.getModelErrors('Product');
if (errors.length > 0) {
console.error('Import failed:', errors);
return { success: false, errors };
}
return { success: true, data: csvData };
}Best Practices
1. Enable Quality Globally
// ✅ Good: Enable globally
const dcupl = new Dcupl({
quality: { enabled: true },
});
// ❌ Bad: Forget to enable
const dcupl = new Dcupl();2. Enable Per Model
// ✅ Good: Enable for models that need validation
{
key: 'User',
quality: { enabled: true },
properties: [...]
}
// ⚠️ Optional: Disable for models that don't need it
{
key: 'LogEntry',
quality: { enabled: false },
properties: [...]
}3. Validate Critical Fields
// ✅ Good: Validate important fields
{
key: 'email',
type: 'string',
quality: {
required: true,
validators: { email: {}, unique: {} }
}
}
// ❌ Bad: No validation on critical fields
{
key: 'email',
type: 'string'
}4. Use Appropriate Validators
// ✅ Good: Specific validators
{
key: 'age',
type: 'int',
quality: {
validators: {
min: { value: 0 },
max: { value: 120 }
}
}
}
// ❌ Bad: No range validation
{
key: 'age',
type: 'int'
}5. Check Errors After Init
// ✅ Good: Check for errors
await dcupl.init();
const errors = dcupl.quality.getErrors();
if (errors.length > 0) {
console.error('Validation errors:', errors);
}
// ❌ Bad: Ignore errors
await dcupl.init();
// Continue without checkingTroubleshooting
Validation Not Running
Problem: No validation errors despite invalid data
Solution: Check both global and model-level enablement:
// Global
const dcupl = new Dcupl({
quality: { enabled: true }, // ✅
});
// Model
{
key: 'User',
quality: { enabled: true }, // ✅
properties: [...]
}Too Many Errors
Problem: Overwhelming number of validation errors
Solution: Start with critical fields only:
// Phase 1: Validate critical fields
{ key: 'email', quality: { required: true, validators: { email: {} } } }
// Phase 2: Add more validation graduallyPerformance Impact
Problem: Validation slows down init
Solution: Disable for non-critical models:
{
key: 'LogEntry',
quality: { enabled: false }, // Skip validation
}Type Coercion Issues
Problem: Strict type checking too restrictive
Solution: Adjust forceStrictDataType:
{
key: 'quantity',
type: 'int',
quality: {
forceStrictDataType: false, // Allow coercion
}
}What's Next?
- Models - Complete property configuration
- Data Management - Loading and updating data
- Derived Properties - Computed values
- Expression Properties - Template-based fields
- Performance - Optimization strategies