Tips & Troubleshooting

Practical guidance for getting the most out of DataBridge, along with solutions for common issues.

---

Known Limitations

Understanding current limitations helps you plan your integration and avoid surprises.

Supported Data Stores

DataBridge currently supports the following destination types:

Destination	Event Delivery	Data Quality (Agent)
PostgreSQL	Yes	Yes
ClickHouse	Yes	Yes
MySQL	Yes	Yes

Kafka and S3 destinations are under active development and not yet available in the UI.

Billing Plan Limits

Each plan has a monthly cap on delivered events (events that successfully reach a destination). See the Pricing page for current plan limits.

Billing Limit Behavior

When your organization exceeds its monthly event limit, pipelines pause delivery for 15 minutes before rechecking. Events continue to be accepted and buffered during this time, but no new events are delivered until the billing period resets or you upgrade your plan.

Event Payload Limits

Each data source has a configurable Max Payload Size (default: 64 KB). Events exceeding this size are rejected at ingestion. If you need to send larger payloads, increase the limit when creating or editing the source.

Schema Validation

• Events must reference a registered schema in the schema registry. Events with an unrecognized $id (schema reference URL) are rejected.
• Schema compilation results are cached for up to 3 hours in the gateway. After updating a schema, allow up to 3 hours for all gateway workers to pick up the change (or restart the gateway for immediate effect).

Passthrough Mode

When a source has Passthrough enabled, invalid events are still ingested and delivered rather than being discarded. This is useful for testing but can pollute your destination with bad data.

warning

Passthrough mode bypasses schema validation. Only use it during initial testing or migration. Disable it for production sources.

Agent Mode

• The DataBridge Agent must be running and connected to the cloud API for data profiling and quality checks to execute.
• The agent polls for tasks every 10 seconds. If it encounters errors, it backs off exponentially up to 5 minutes before retrying.
• Data quality checks (profiling and validation) are only supported for PostgreSQL, ClickHouse and MySQL destinations.

Alert Destinations

Supported alert channels for data quality notifications:

• Webhook -- generic HTTP POST
• Slack -- via incoming webhook URL
• Email -- to configured recipients
• Telegram -- via bot integration

---

Useful Tips

Getting Started

Start with Passthrough, Then Add Schemas

When integrating a new event source, create the source with Passthrough enabled first. Send a few test events, verify they appear in your destination, then create schemas and disable passthrough for strict validation.

Use Separate Sources per Environment

Create distinct sources for production, staging and development. This prevents test data from mixing with production events and lets you apply different settings per environment.

Data Sources

• Test your API secret with a quick curl request before wiring up an SDK. This confirms that ingestion, validation and delivery all work end-to-end.
• Use descriptive names for sources (e.g., Production Website, Mobile App iOS) so the dashboard is easy to scan.

Pipelines

• Batch size tuning: The default batch size is 256 events. Increase it for higher throughput when delivery latency isn't critical. Decrease it if you need events to reach the destination faster.
• One pipeline per source-destination pair: Each pipeline connects exactly one source to one destination. To send the same events to multiple destinations, create a pipeline for each.
• Attach Functions to pipelines when you need to transform or enrich events before delivery. Functions run as a chain -- the output of one feeds into the next.

Data Quality

• Profile your datasets to establish baselines before creating quality checks. Profiling gives you column-level statistics (row count, nulls, min/max, cardinality) that inform good threshold values.
• Use the visual check builder in the dashboard rather than typing check expressions manually. It helps you pick the right function, column, operator and threshold.
• Standard check expressions follow the format: function(column) operator threshold. For example: not_null(email) >= 0.99 means "at least 99% of email values should be non-null".

Dashboard

Set Your Organization Timezone

Go to Settings and set your organization's timezone. All dashboard timestamps, charts and time-bucketed metrics display in this timezone. The default is UTC.

• The time range selector on the main dashboard affects all metrics shown: Data Quality percentage, failed checks, event volume charts and destination delivery stats. Available ranges are Last Week (default), Last Month, Last 3 Months and Last Year.
• Dataset overview on the dashboard shows up to 10 datasets with their latest row counts and trend direction.

ClickHouse Connections

ClickHouse supports multiple protocols with different default ports:

Protocol	Default Port
http	8123
https	8443
clickhouse	9000
clickhouse-secure	9440

tip

If you're connecting to ClickHouse Cloud, use the clickhouse-secure protocol. For local development, http on port 8123 is the simplest option.

---

Troubleshooting

Events Not Appearing in Destination

Symptoms: You're sending events but nothing shows up in your data store.

Check these in order:

1. Is the source enabled? Disabled sources don't accept events. Go to Pipelines > Sources and verify the source is active.
2. Is the API secret correct? The gateway validates API secrets on every request. An invalid secret results in a silent rejection. Double-check the Authorization: Bearer <secret> header.
3. Does a matching schema exist? Unless the source has Passthrough enabled, events must reference a schema registered in your organization's schema registry. Check the $id field in your event payload matches an existing schema.
4. Is the pipeline enabled? Go to Pipelines and confirm the pipeline connecting your source to your destination is toggled on.
5. Has billing been exceeded? If your organization has hit its monthly event limit, pipelines pause delivery. Check the dashboard for billing warnings.
6. Is the destination reachable? Use the Test Connection feature when editing a data store to verify connectivity.

Pipeline Not Delivering Events

Symptoms: Events are received (source metrics increase) but destination shows no new data.

• Check pipeline status: A disabled pipeline won't process events.
• Check destination errors: Go to the pipeline detail page. If the bridge can't write to the destination, errors are recorded in pipeline metrics.
• Billing limits: When exceeded, pipelines sleep for 15 minutes between billing checks. The dashboard may show a billing warning.
• Destination table creation: DataBridge auto-creates tables in the destination. If the destination user lacks CREATE TABLE permissions, delivery fails silently on the first batch for each new event type.

Schema Validation Failures

Symptoms: Events are rejected; you see increasing invalid event counts.

• Review the failed events panel on the dashboard. It shows the event name, source, failure reason and timestamp.
• Check required fields: Missing a field marked required in your JSON schema causes rejection.
• Check data types: Sending a string where the schema expects a number (or vice versa) causes rejection.
• Check format constraints: Fields with format, enum, minimum, maximum, or pattern constraints are validated strictly.
• Schema version mismatch: Ensure the event's $id references the correct schema version (namespace/event-name/major-minor-patch).

Data Quality Checks Not Running

Symptoms: Check results are stale or no results appear.

1. Is the agent running? Quality checks are executed by the DataBridge Agent. If the agent process is down, no checks run.
2. Is the agent connected? The agent polls the cloud API for tasks. Check agent logs for connection errors or repeated backoff messages.
3. Check the NextRunAt schedule: Each check has a scheduled next run time. If it's in the future, the check hasn't come due yet.
4. Connection resolution: In Agent mode, verify that the connection_alias in the agent's local config matches the alias configured for the destination in the dashboard.

Connection Test Fails

Symptoms: "Test Connection" button returns an error when configuring a data store.

PostgreSQL:

• Verify host, port (default: 5432), database, username and password.
• Check ssl_mode: use require for cloud-hosted databases, disable for local development.
• Ensure the database user has CREATE TABLE and INSERT permissions on the target schema.

ClickHouse:

• Verify the protocol matches your ClickHouse deployment. Using the wrong protocol (e.g., http when the server expects native clickhouse protocol) causes connection failures.
• Ensure the port matches the protocol (see the ports table above).
• For ClickHouse Cloud, use clickhouse-secure with port 9440.
• If using a cluster, provide the cluster name. Leave it empty for single-node deployments.

MySQL:

• Verify host, port (default: 3306), database, username and password.
• Check ssl_mode: use require for cloud-hosted databases.
• Ensure the database user has CREATE TABLE and INSERT permissions.

Dashboard Shows No Data

Symptoms: Dashboard panels are empty or show zeros.

• Check the time range: The dashboard defaults to Last Week. If your events are older or you just started, switch to a broader range (Last Month, Last 3 Months).
• Verify events are flowing: Go to Pipelines > Sources and check if any source shows recent received event counts.
• Check your timezone setting: If the organization timezone is misconfigured, chart buckets may appear shifted or empty for the expected date range.

Transformation Function Errors

Symptoms: Events arrive at the destination without expected transformations, or events are dropped.

• Functions execute in a sandboxed JavaScript environment. Runtime errors in a function cause the event to pass through without transformation (the original payload is preserved).
• Check the pipeline's function configuration to ensure functions are attached in the correct order.
• If a function returns null, the event is filtered out and won't be delivered.
• The system runs up to 16 concurrent function invocations per pipeline batch. Long-running functions can slow down the entire pipeline.

Agent Backing Off Repeatedly

Symptoms: Agent logs show increasing backoff delays.

• The agent uses exponential backoff when it encounters consecutive errors polling for tasks. Delays start at 10 seconds and cap at 5 minutes.
• Check network connectivity between the agent and the DataBridge Cloud API.
• Check agent credentials: Invalid or expired authentication causes every poll to fail.
• Check agent logs for specific error messages. Connection resolution failures (missing alias, unreachable destination) are reported per-task and don't trigger backoff. Only poll-level failures (network, auth) cause backoff.

---

Getting Help

If you can't resolve an issue using this guide:

• Email us at hi@databridge.tech
• Include relevant error messages, your organization ID and the steps you've already tried