Skip to main content

Create Data Source

A Data Source is any application or system that generates events and sends them to DataBridge. This can be a website, mobile app, backend server, IoT device, or third-party service.

What is a Data Source?​

Data Sources are event producers that:

  • Generate events based on user actions (clicks, views, purchases)
  • Send automated events based on system logic (cron jobs, triggers)
  • Forward events from third-party services (webhooks)

Each Data Source has:

  • Unique API key for authentication
  • Ingestion endpoint to send events
  • Rate limits and quotas based on your plan
  • Monitoring dashboard for volume, errors and latency

Types of Data Sources​

1. Client-Side SDKs​

For web browsers and mobile apps:

  • JavaScript/TypeScript (Browser, Node.js)
  • iOS (Swift, Objective-C)
  • Android (Kotlin, Java)

2. Server-Side SDKs​

Coming Soon

For backend applications:

  • Python
  • Ruby
  • Go
  • Java
  • PHP
  • .NET

3. HTTP API​

For custom integrations:

  • Direct HTTP POST to ingestion endpoint
  • Flexible for any language or platform

4. Webhooks​

For third-party services:

  • Stripe, Shopify, Twilio, etc.
  • Forward events from external platforms

Creating Your First Data Source​

Step 1: Navigate to Sources​

Go to Ingestion Sources and click Configure New Source.

Step 2: Configure Source Details​

Provide the following information:

Source Name

  • Example: Production Website, Mobile App (iOS), Backend API
  • Use descriptive names to identify the source easily

Description (optional)

  • Add context about what this source tracks
  • Example: "Production e-commerce website tracking user behavior"

Step 3: Schema Validation Settings​

Choose validation mode:

Strict Validation (Default, recommended)

  • All events must match a schema in your registry
  • Invalid events are rejected and sent to dead-letter queue
  • Ensures 100% data quality

Permissive Mode

  • Events without schemas are still ingested
  • Useful for gradual migration or testing
  • ⚠️ Not recommended for production

Best practice: Always use strict validation to maintain data quality.

Step 4: Get API Key and Endpoint​

After creating the source, you'll receive:

API Key (Write Key)

dbr_source_c01d22f8f21...

⚠️ Keep this secret! It allows writing events to your source.

Ingestion Endpoint

https://cloud.databridge.tech/capture

Step 5: Install SDK (Optional)​

If using an SDK, install it in your application:

JavaScript (Browser):

npm install @databridge/browser

JavaScript (Node.js):

npm install @databridge/node

Sending Events​

Using JavaScript SDK​

import DataBridge from '@databridge/browser';

// Initialize with your API key
const databridge = new DataBridge('dbr_source_c01d22f8f21...');

// Track an event
databridge.capture('com.yourcompany/purchase_completed/1-0-0', {
  user_id: 'user_12345',
  order_id: 'ORD-98765',
  amount: 149.99,
  currency: 'USD',
  items: [
    {product_id: 'PROD-001', quantity: 2, price: 49.99},
    {product_id: 'PROD-002', quantity: 1, price: 50.01}
  ],
  timestamp: new Date().toISOString()
});

Using HTTP API​

curl -X POST https://cloud.databridge.tech/capture \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer dbr_source_c01d22f8f21..." \
  -d '{
    "event": "com.yourcompany/purchase_completed/1-0-0",
    "payload": {
      "user_id": "user_12345",
      "order_id": "ORD-98765",
      "amount": 149.99,
      "currency": "USD",
      "items": [
        {"product_id": "PROD-001", "quantity": 2, "price": 49.99}
      ],
      "timestamp": "2024-01-15T10:30:00Z"
    }
  }'

Event Validation Flow​

When events arrive at DataBridge:

  1. Event received
  2. Find matching schema
  3. Validate against schema
  4. Accept or Reject

If Valid:

  • Event passes to transformation pipeline
  • Delivered to configured destinations
  • Counted toward your event quota

If Invalid:

  • Event is rejected
  • Sent to dead-letter queue for review
  • Error logged with detailed validation failure message
  • Alert sent (if configured)

Monitoring Your Source​

After sending events, monitor your source:

Volume Metrics:

  • Events received per hour/day
  • Validation success rate
  • Error rate

Error Tracking:

  • Invalid events with reasons
  • Dead-letter queue contents
  • Schema mismatch details

Performance:

  • Average ingestion latency
  • Peak throughput
  • API response times

Testing Your Source​

Quick Test with HTTP API​

Send a test event using curl:

curl -X POST https://cloud.databridge.tech/capture \
  -H "Content-Type: application/json" \
  -H "Api-Secret: YOUR_API_KEY" \
  -d '{
    "event": "com.yourcompany/test_event/1-0-0",
    "properties": {
      "test": true,
      "timestamp": "2024-01-15T10:30:00Z"
    }
  }'

Check DataBridge Dashboard​

  1. Go to Ingestion Sources
  2. Click on your source
  3. View Recent Events tab
  4. Verify event was received and validated

Best Practices​

1. Use Environment-Specific Sources​

Create separate sources for each environment:

  • Production Website
  • Staging Website
  • Development Website

This prevents test data from polluting production.

2. Implement Error Handling​

try {
  databridge.capture('com.yourcompany/purchase_completed/1-0-0', eventData);
} catch (error) {
  console.error('Failed to capture event:', error);
  // Log to your error monitoring service
}

3. Add Context to Events​

Include metadata to help debugging:

databridge.capture('com.yourcompany/page_view/1-0-0', {
  page_url: window.location.href,
  user_id: currentUser.id,
  session_id: sessionId,
  environment: 'production',
  app_version: '2.1.0',
  timestamp: new Date().toISOString()
});

4. Monitor Validation Errors​

Set up alerts for validation failures:

  • High error rates indicate schema mismatches
  • Review dead-letter queue regularly
  • Update schemas or fix event tracking code

Common Issues​

Events Not Appearing​

Check:

  1. API key is correct
  2. Ingestion endpoint URL is correct
  3. Event matches a schema in registry
  4. Schema validation is not failing (check dead-letter queue)

Validation Errors​

Solutions:

  1. Review schema in Events section
  2. Test event payload in Validate tab
  3. Check for required fields, types and formats
  4. Update schema or fix event structure

Rate Limiting​

Symptoms:

  • 429 Too Many Requests responses
  • Events being dropped

Solutions:

  • Upgrade to higher plan with more quota
  • Implement batching to reduce request count
  • Use server-side SDK for better performance

Next Steps​

Now that you've created a data source and sent events:

  1. Connect a Data Warehouse to deliver events
  2. Configure a Pipeline to connect source to destination
  3. Monitor event quality in the dashboard
  4. Add more event schemas for other tracking needs

Your events are now flowing into DataBridge and being validated in real-time!