Skip to main content

Quick Start Guide

Cloud Dashboard is the recommended approach

The DataBridge Dashboard handles profiling and validation tasks through a managed Agent - no CLI installation needed. See DataBridge Cloud to connect a data destination and run your first profile.

This page documents dbqctl, the open-source CLI for local development and CI/CD workflows.

Get started with DataBridge data quality monitoring in minutes using dbqctl, our open-source CLI tool.

Installation​

macOS​

# Using Homebrew (recommended)
brew install databridge/tap/dbqctl

# Or download the binary directly
curl -L https://github.com/DataBridgeTech/dbqctl/releases/latest/download/dbqctl-darwin-amd64 -o dbqctl
chmod +x dbqctl
sudo mv dbqctl /usr/local/bin/

Linux​

# Download the binary
curl -L https://github.com/DataBridgeTech/dbqctl/releases/latest/download/dbqctl-linux-amd64 -o dbqctl
chmod +x dbqctl
sudo mv dbqctl /usr/local/bin/

Windows​

# Download from GitHub releases
# https://github.com/DataBridgeTech/dbqctl/releases/latest

Verify Installation​

dbqctl version

Configuration​

Create a configuration file to define your data sources. dbqctl looks for configuration in:

  • $HOME/.dbq.yaml (global)
  • ./dbq.yaml (project-specific)
  • Custom path via --config flag

Example Configuration​

Create ~/.dbq.yaml:

version: "1"
datasources:
  # PostgreSQL example
  - id: pg
    type: postgresql
    configuration:
      host: localhost
      port: 5432
      username: postgres
      password: ${PG_PASSWORD}  # Use environment variables for secrets
      database: mydb
    datasets:
      - public.users
      - public.orders

  # ClickHouse example
  - id: ch
    type: clickhouse
    configuration:
      host: localhost
      port: 9000
      username: default
      password: ${CH_PASSWORD}
      database: default
    datasets:
      - nyc_taxi.trips_small

  # MySQL example
  - id: mysql
    type: mysql
    configuration:
      host: localhost
      port: 3306
      username: root
      password: ${MYSQL_PASSWORD}
      database: ecommerce
    datasets:
      - ecommerce.orders

Security Tip: Always use environment variables for passwords, never hardcode credentials.

Your First Data Profile​

Profile a table to understand its data quality characteristics:

# Check connection
dbqctl ping -d pg

# Import available datasets
dbqctl import -d pg --filter "public"

# Profile a specific table
dbqctl profile -d pg --dataset public.users

Expected output:

Dataset: public.users
Rows: 10,542

Column: user_id (bigint)
  Null Count: 0 (0.0%)
  Unique Values: 10,542
  Min: 1
  Max: 10,542

Column: email (varchar)
  Null Count: 3 (0.03%)
  Blank Count: 0
  Unique Values: 10,539
  Most Frequent: user@example.com (5)

Column: created_at (timestamp)
  Null Count: 0
  Min: 2023-01-15 08:23:11
  Max: 2024-10-01 14:32:09

Column: country (varchar)
  Null Count: 127 (1.2%)
  Unique Values: 43
  Most Frequent: US (3,421), UK (1,892), CA (876)

Your First Quality Check​

Create a checks file checks.yaml:

version: "1"
rules:
  # Check a PostgreSQL table
  - dataset: pg@[public.users]
    checks:
      # Schema validation
      - schema_check:
          expect_columns:
            columns: [user_id, email, created_at, country]
        desc: "Users table must have core columns"
        on_fail: error

      # Table-level checks
      - row_count > 1000:
          desc: "Should have at least 1000 users"
          on_fail: error

      # Column-level checks
      - not_null(user_id):
          desc: "User ID is required"

      - not_null(email):
          desc: "Email is required"

      - uniqueness(email):
          desc: "Emails must be unique"
          on_fail: error

      - freshness(created_at) < 7d:
          desc: "Should have new users in the last week"
          on_fail: warn

Run the checks:

dbqctl check --checks ./checks.yaml

Expected output:

Running checks on pg@[public.users]...

✓ Schema check passed: Users table must have core columns
✓ Row count check passed: Should have at least 1000 users (10,542 rows)
✓ Not null check passed: User ID is required (0 nulls)
✗ Not null check FAILED: Email is required (3 nulls found)
✓ Uniqueness check passed: Emails must be unique (10,539 unique values)
✓ Freshness check passed: Should have new users in the last week (last: 2 hours ago)

Results: 5 passed, 1 failed

Next Steps​

Now that you've run your first quality checks, explore:

Quick Tips​

  • Start with profiling to understand your data before writing checks
  • Use descriptive check descriptions to make failures self-explanatory
  • Validate in CI/CD by integrating dbqctl into your deployment pipeline
  • Monitor critical tables by running checks on a schedule (cron/Airflow)
  • Version your checks alongside your schema definitions in git

Getting Help​