Skip to main content

Schema Registry

Schema registry and validation

Stop Bad Data Before It Reaches Your Warehouse​

Define schemas and validate events in real-time. Data quality issues get caught at the source - not discovered days later in your warehouse.

Schema Registry with Version Control​

Centralized Data Catalog​

Maintain a comprehensive catalog of all your data (event) models in a centralized schema registry. Your schema registry serves as the single source of truth for your data contracts, ensuring all teams - from engineering to analytics - work from consistent definitions.

Key Features:

  • Schema Browser: Browse and search all registered schemas
  • Auto-Generated Documentation: Automatically generate human-readable documentation from schemas
  • Visual Schema Viewer: Review schemas through an intuitive UI or import JSON Schema definitions
  • Schema Lineage: Track which applications and destinations use each schema
  • Access Controls: Manage who can view, edit and publish schemas

Version Control & Evolution​

Track schema changes over time and manage multiple versions of the same schema. As your business evolves, your data models need to evolve too - but in a controlled, backward-compatible way.

Version Management:

  • Semantic Versioning: Major, minor and patch versions for schema changes
  • Version History: Complete audit trail of all schema modifications
  • Rollback Support: Revert to previous schema versions if needed
  • Deprecation Workflow: Mark old versions as deprecated while maintaining support

Real-Time Event Validation​

Validate at the Source​

Every event is validated against its registered JSON schema before it reaches your data warehouse. This prevents bad data from ever entering your analytics pipeline, saving countless hours of data cleanup and debugging.

Validation Process:

  1. Event Ingestion: Event arrives at DataBridge collector
  2. Schema Lookup: System identifies the appropriate schema version
  3. Real-Time Validation: Event is validated against JSON schema
  4. Success Path: Valid events are enriched and routed to destinations
  5. Failure Path: Invalid events are sent to dead-letter queue

Comprehensive Validation Rules​

DataBridge supports the full JSON Schema specification, enabling comprehensive validation:

  • Data Type Validation
  • Format Validation
  • RegExp Pattern Matching
  • Range Validation
  • Enum Validation
  • Required Fields

Validation Feedback​

When validation fails, DataBridge provides detailed error messages to help developers quickly identify and fix issues:

Example Validation Error:

{
  "event_id": "evt_abc123",
  "validation_status": "failed",
  "errors": [
    {
      "field": "email",
      "message": "Value must be a valid email format",
      "invalid_value": "not-an-email",
      "schema_path": "#/properties/email"
    },
    {
      "field": "age",
      "message": "Value must be an integer between 0 and 150",
      "invalid_value": -5,
      "schema_path": "#/properties/age"
    }
  ],
  "timestamp": "2025-01-15T10:30:00Z"
}

Dead-Letter Queue Management​

Capture and Review Failed Events​

Events that fail validation are automatically routed to dead-letter queues (DLQs) for review and debugging, so you never lose important data - even when it doesn't match your schema.

DLQ Features:

  • Automatic Routing: Failed events are immediately sent to DLQ
  • Detailed Error Context: Each failed event includes validation errors and metadata
  • Search & Filter: Find specific failures by error type, schema, or time range
  • Manual Review: Review failed events in the UI

Benefits of Schema Validation​

Data Quality Guarantee​

  • Catch Errors Early: Prevent bad data from reaching your warehouse
  • Consistent Data Types: Ensure all data conforms to expected types
  • Reduced Data Debt: Eliminate need for extensive data cleanup
  • Trust in Analytics: Make decisions with confidence in data quality

Developer Experience​

  • Immediate Feedback: Know instantly if events are malformed
  • Clear Error Messages: Detailed validation errors speed up debugging
  • Type Safety: Generate types from schemas for compile-time checking
  • Documentation: Schemas serve as living documentation

Operational Efficiency​

  • Reduce Debugging Time: Fix issues before they reach production
  • Lower Storage Costs: Don't store invalid data
  • Faster Analytics: No need to clean data before analysis
  • Team Alignment: Single source of truth for data contracts

Ready to Get Started?​

Start with our free tier (100K events/month) or explore paid plans starting at $79/month.