Boltbase

Appearance

CLI Reference ​

Commands accept either DAG names (from YAML name field) or file paths.

  • Both formats: start, stop, status, retry
  • File path only: dry, enqueue
  • DAG name only: restart

Global Options ​

bash
boltbase [global options] command [command options] [arguments...]
  • --config, -c - Config file (default: ~/.config/boltbase/config.yaml)
  • --boltbase-home - Override BOLTBASE_HOME for this command invocation
  • --quiet, -q - Suppress output
  • --cpu-profile - Enable CPU profiling
  • --help, -h - Show help
  • --version, -v - Print version

Commands ​

exec ​

Run a command without writing a YAML file.

bash
boltbase exec [options] -- <command> [args...]

Options:

  • --name, -N - DAG name (default: exec-<command>)
  • --run-id, -r - Custom run ID
  • --env KEY=VALUE - Set environment variable (repeatable)
  • --dotenv <path> - Load dotenv file (repeatable)
  • --workdir <path> - Working directory
  • --shell <path> - Shell binary
  • --base <file> - Custom base config file (default: ~/.config/boltbase/base.yaml)
  • --worker-label key=value - Set worker selector labels (repeatable)
bash
# Basic usage
boltbase exec -- python script.py

# With environment variables
boltbase exec --env DB_HOST=localhost -- python etl.py

See the exec guide for detailed documentation.

start ​

Run a DAG workflow.

bash
boltbase start [options] DAG_NAME_OR_FILE [-- PARAMS...]

Interactive Mode:

  • If no DAG file is specified, opens an interactive selector
  • Only available in terminal (TTY) environments
  • Shows enhanced progress display during execution

Options:

  • --params, -p - Parameters as JSON
  • --name, -N - Override the DAG name (default: name from DAG definition or filename)
  • --run-id, -r - Custom run ID
  • --from-run-id - Re-run using the DAG snapshot and parameters captured from a historic run

Note: --from-run-id cannot be combined with --params, --parent, or --root. Provide exactly one DAG name or file so the command can look up the historic run.

bash
# Basic run
boltbase start my-workflow.yaml

# Interactive mode (no file specified)
boltbase start

# With parameters (note the -- separator)
boltbase start etl.yaml -- DATE=2024-01-01 ENV=prod

# Custom run ID
boltbase start --run-id batch-001 etl.yaml

# Override DAG name
boltbase start --name my_custom_name my-workflow.yaml

# Clone parameters from a historic run
boltbase start --from-run-id 20241031_235959 example-dag.yaml

stop ​

Stop a running DAG.

bash
boltbase stop [options] DAG_NAME_OR_FILE

Options:

  • --run-id, -r - Specific run ID (optional)
bash
boltbase stop my-workflow                     # Stop current run
boltbase stop --run-id=20240101_120000 etl   # Stop specific run

restart ​

Restart a DAG run with a new ID.

bash
boltbase restart [options] DAG_NAME

Options:

  • --run-id, -r - Run to restart (optional)
bash
boltbase restart my-workflow                  # Restart latest
boltbase restart --run-id=20240101_120000 etl # Restart specific

retry ​

Retry a failed DAG execution.

bash
boltbase retry [options] DAG_NAME_OR_FILE

Options:

  • --run-id, -r - Run to retry (required)
bash
boltbase retry --run-id=20240101_120000 my-workflow

status ​

Display current status of a DAG.

bash
boltbase status [options] DAG_NAME_OR_FILE

Options:

  • --run-id, -r - Check specific run (optional)
bash
boltbase status my-workflow  # Latest run status

Output:

Status: running
Started: 2024-01-01 12:00:00
Steps:
  ✓ download     [completed]
  ⟳ process      [running]
  ○ upload       [pending]

history ​

Display execution history of DAG runs with filtering and pagination.

Usage:

bash
boltbase history [flags] [DAG_NAME]

Flags:

  • --from - Start date/time in UTC (formats: 2006-01-02 or 2006-01-02T15:04:05Z)
  • --to - End date/time in UTC (formats: 2006-01-02 or 2006-01-02T15:04:05Z)
  • --last - Relative time period (examples: 7d, 24h, 1w, 30d). Cannot combine with --from/--to
  • --status - Filter by status: running, succeeded, failed, aborted, queued, waiting, rejected, not_started, partially_succeeded
    • Aliases: success (succeeded), failure (failed), canceled/cancelled/cancel (aborted)
  • --run-id - Filter by run ID (partial match supported)
  • --tags - Filter by tags, comma-separated with AND logic (e.g., prod,critical)
  • --format, -f - Output format: table (default), json, or csv
  • --limit, -l - Max results (default: 100, max: 1000)

Default Behavior:

  • Shows last 30 days of runs
  • Table format with columns: DAG NAME, RUN ID, STATUS, STARTED (UTC), DURATION, PARAMS
  • Sorted newest first
  • Limit 100 results
  • Run IDs are never truncated

Examples:

bash
# All runs from last 30 days
boltbase history

# Specific DAG runs
boltbase history my-workflow

# Recent failures for debugging
boltbase history my-workflow --status failed --last 7d

# Date range query
boltbase history --from 2026-01-01 --to 2026-01-31

# JSON export for analysis
boltbase history --format json --limit 500 > history.json

# CSV export for spreadsheets
boltbase history --format csv --limit 500 > history.csv

# Tag filtering (AND logic)
boltbase history --tags "prod,critical"

# Combined filters
boltbase history my-workflow --status failed --last 24h --limit 10

Output (table):

DAG NAME      RUN ID                                STATUS     STARTED (UTC)        DURATION  PARAMS
my-workflow   019c1ca4-ba96-7599-80c9-773862801abc  Succeeded  2026-02-02 04:38:03  2m30s     -
my-workflow   019c1ca3-f123-4567-89ab-cdef01234567  Failed     2026-02-01 14:22:15  45s       env=prod

Output (JSON):

json
[
  {
    "name": "my-workflow",
    "dagRunId": "019c1ca4-ba96-7599-80c9-773862801abc",
    "status": "succeeded",
    "startedAt": "2026-02-02T04:38:03Z",
    "finishedAt": "2026-02-02T04:40:33Z",
    "duration": "2m30s",
    "params": "",
    "tags": ["prod", "backend"],
    "workerId": "",
    "error": ""
  }
]

Output (CSV):

csv
DAG NAME,RUN ID,STATUS,STARTED (UTC),DURATION,PARAMS
my-workflow,019c1ca4-ba96-7599-80c9-773862801abc,Succeeded,2026-02-02 04:38:03,2m30s,-
my-workflow,019c1ca3-f123-4567-89ab-cdef01234567,Failed,2026-02-01 14:22:15,45s,env=prod

Note: CSV output follows RFC 4180. Fields containing commas, quotes, or newlines are automatically quoted and escaped.

Error Examples:

bash
# Conflicting flags
$ boltbase history --last 7d --from 2026-01-01
Error: cannot use --last with --from or --to

# Invalid status
$ boltbase history --status invalid
Error: invalid status 'invalid'. Valid values: running, succeeded, failed, ...

# Date validation
$ boltbase history --from 2026-02-01 --to 2026-01-01
Error: --from date (2026-02-01) must be before --to date (2026-01-01)

See Also:

server ​

Start the web UI server.

bash
boltbase server [options]

Options:

  • --host, -s - Host (default: localhost)
  • --port, -p - Port (default: 8080)
  • --dags, -d - DAGs directory
bash
boltbase server                               # Default settings
boltbase server --host=0.0.0.0 --port=9000  # Custom host/port

scheduler ​

Start the DAG scheduler daemon.

bash
boltbase scheduler [options]

Options:

  • --dags, -d - DAGs directory
bash
boltbase scheduler                  # Default settings
boltbase scheduler --dags=/opt/dags # Custom directory

start-all ​

Start scheduler, web UI, and optionally coordinator service.

bash
boltbase start-all [options]

Options:

  • --host, -s - Host (default: localhost)
  • --port, -p - Port (default: 8080)
  • --dags, -d - DAGs directory
  • --coordinator.host - Coordinator bind address (default: 127.0.0.1)
  • --coordinator.advertise - Address to advertise in service registry
  • --coordinator.port - Coordinator gRPC port (default: 50055)
bash
# Single instance mode (coordinator disabled)
boltbase start-all

# Distributed mode with coordinator enabled
boltbase start-all --coordinator.host=0.0.0.0 --coordinator.port=50055

# Production mode
boltbase start-all --host=0.0.0.0 --port=9000 --coordinator.host=0.0.0.0

Note: The coordinator service is only started when --coordinator.host is set to a non-localhost address (not 127.0.0.1 or localhost). By default, start-all runs in single instance mode without the coordinator.

validate ​

Validate a DAG specification for structural correctness.

bash
boltbase validate [options] DAG_FILE

Checks structural correctness and references (e.g., step dependencies) without evaluating variables or executing the DAG. Returns validation errors in a human-readable format.

bash
boltbase validate my-workflow.yaml

Output when valid:

DAG spec is valid: my-workflow.yaml (name: my-workflow)

Output when invalid:

Validation failed for my-workflow.yaml
- Step 'process' depends on non-existent step 'missing_step'
- Invalid cron expression in schedule: '* * * *'

dry ​

Validate a DAG without executing it.

bash
boltbase dry [options] DAG_FILE [-- PARAMS...]

Options:

  • --params, -p - Parameters as JSON
  • --name, -N - Override the DAG name (default: name from DAG definition or filename)
bash
boltbase dry my-workflow.yaml
boltbase dry etl.yaml -- DATE=2024-01-01  # With parameters
boltbase dry --name my_custom_name my-workflow.yaml  # Override DAG name

enqueue ​

Add a DAG to the execution queue.

bash
boltbase enqueue [options] DAG_FILE [-- PARAMS...]

Options:

  • --run-id, -r - Custom run ID
  • --params, -p - Parameters as JSON
  • --name, -N - Override the DAG name (default: name from DAG definition or filename)
  • --queue, -u - Override DAG-level queue name for this enqueue
bash
boltbase enqueue my-workflow.yaml
boltbase enqueue --run-id=batch-001 etl.yaml -- TYPE=daily
# Enqueue to a specific queue (override)
boltbase enqueue --queue=high-priority my-workflow.yaml
# Override DAG name
boltbase enqueue --name my_custom_name my-workflow.yaml

dequeue ​

Remove a DAG from the execution queue.

bash
boltbase dequeue <queue-name> --dag-run=<dag-name>:<run-id>  # remove specific run
boltbase dequeue <queue-name>                                # pop the oldest item

Example:

bash
boltbase dequeue default --dag-run=my-workflow:batch-001
boltbase dequeue default

version ​

Display version information.

bash
boltbase version

cleanup ​

Remove old DAG run history for a specified DAG.

bash
boltbase cleanup [options] DAG_NAME

Options:

  • --retention-days - Number of days to retain (default: 0 = delete all)
  • --dry-run - Preview what would be deleted without actually deleting
  • --yes, -y - Skip confirmation prompt

Active runs (running, queued) are never deleted for safety.

bash
# Delete all history (with confirmation prompt)
boltbase cleanup my-workflow

# Keep last 30 days of history
boltbase cleanup --retention-days 30 my-workflow

# Preview what would be deleted
boltbase cleanup --dry-run my-workflow

# Delete without confirmation (for scripts)
boltbase cleanup -y my-workflow

# Combine options
boltbase cleanup --retention-days 7 -y my-workflow

Output:

# Dry run output
Dry run: Would delete 5 run(s) for DAG "my-workflow":
  - 019b1c4b-1b1e-7232-b12d-e822dac72613
  - 019b1c4b-13e1-7251-a713-aaad60dfa88c
  ...

# Actual deletion output
Successfully removed 5 run(s) for DAG "my-workflow"

migrate ​

Migrate legacy data to new format.

bash
boltbase migrate history  # Migrate v1.16 -> v1.17+ format

coordinator ​

Start the coordinator gRPC server for distributed task execution.

bash
boltbase coordinator [options]

Options:

  • --coordinator.host - Host address to bind (default: 127.0.0.1)
  • --coordinator.advertise - Address to advertise in service registry (default: auto-detected hostname)
  • --coordinator.port - Port number (default: 50055)
  • --peer.cert-file - Path to TLS certificate file for peer connections
  • --peer.key-file - Path to TLS key file for peer connections
  • --peer.client-ca-file - Path to CA certificate file for client verification (mTLS)
  • --peer.insecure - Use insecure connection (h2c) instead of TLS (default: true)
  • --peer.skip-tls-verify - Skip TLS certificate verification (insecure)
bash
# Basic usage
boltbase coordinator --coordinator.host=0.0.0.0 --coordinator.port=50055

# Bind to all interfaces and advertise service name (for containers/K8s)
boltbase coordinator \
  --coordinator.host=0.0.0.0 \
  --coordinator.advertise=boltbase-server \
  --coordinator.port=50055

# With TLS
boltbase coordinator \
  --peer.insecure=false \
  --peer.cert-file=server.pem \
  --peer.key-file=server-key.pem

# With mutual TLS
boltbase coordinator \
  --peer.insecure=false \
  --peer.cert-file=server.pem \
  --peer.key-file=server-key.pem \
  --peer.client-ca-file=ca.pem

The coordinator service enables distributed task execution by:

  • Automatically registering in the service registry system
  • Accepting task polling requests from workers
  • Matching tasks to workers based on labels
  • Tracking worker health via heartbeats (every 10 seconds)
  • Providing task distribution API with automatic failover
  • Managing worker lifecycle through file-based registry

worker ​

Start a worker that polls the coordinator for tasks.

bash
boltbase worker [options]

Options:

  • --worker.id - Worker instance ID (default: hostname@PID)
  • --worker.max-active-runs - Maximum number of active runs (default: 100)
  • --worker.labels, -l - Worker labels for capability matching (format: key1=value1,key2=value2)
  • --peer.insecure - Use insecure connection (h2c) instead of TLS (default: true)
  • --peer.cert-file - Path to TLS certificate file for peer connections
  • --peer.key-file - Path to TLS key file for peer connections
  • --peer.client-ca-file - Path to CA certificate file for server verification
  • --peer.skip-tls-verify - Skip TLS certificate verification (insecure)
bash
# Basic usage
boltbase worker

# With custom configuration
boltbase worker \
  --worker.id=worker-1 \
  --worker.max-active-runs=50

# With labels for capability matching
boltbase worker --worker.labels gpu=true,memory=64G,region=us-east-1
boltbase worker --worker.labels cpu-arch=amd64,instance-type=m5.xlarge

# With TLS connection
boltbase worker \
  --peer.insecure=false

# With mutual TLS
boltbase worker \
  --peer.insecure=false \
  --peer.cert-file=client.pem \
  --peer.key-file=client-key.pem \
  --peer.client-ca-file=ca.pem

# With self-signed certificates
boltbase worker \
  --peer.insecure=false \
  --peer.skip-tls-verify

Workers automatically register in the service registry system, send regular heartbeats, and poll the coordinator for tasks matching their labels to execute them locally.

Configuration ​

Priority: CLI flags > Environment variables > Config file

Using Custom Home Directory ​

The --boltbase-home flag allows you to override the application home directory for a specific command invocation. This is useful for:

  • Testing with different configurations
  • Running multiple Boltbase instances with isolated data
  • CI/CD scenarios requiring custom directories
bash
# Use a custom home directory for this command
boltbase --boltbase-home=/tmp/boltbase-test start my-workflow.yaml

# Start server with isolated data
boltbase --boltbase-home=/opt/boltbase-prod start-all

# Run scheduler with specific configuration
boltbase --boltbase-home=/var/lib/boltbase scheduler

When --boltbase-home is set, it overrides the BOLTBASE_HOME environment variable and uses a unified directory structure:

$BOLTBASE_HOME/
├── dags/              # DAG definitions
├── logs/              # All log files
├── data/              # Application data
├── suspend/           # DAG suspend flags
├── config.yaml        # Main configuration
└── base.yaml          # Shared DAG defaults

Key Environment Variables ​

  • BOLTBASE_HOME - Set all directories to this path
  • BOLTBASE_HOST - Server host (default: 127.0.0.1)
  • BOLTBASE_PORT - Server port (default: 8080)
  • BOLTBASE_DAGS_DIR - DAGs directory
  • BOLTBASE_LOG_DIR - Log directory
  • BOLTBASE_DATA_DIR - Data directory
  • BOLTBASE_AUTH_BASIC_USERNAME - Basic auth username
  • BOLTBASE_AUTH_BASIC_PASSWORD - Basic auth password

Released under the MIT License.

Copyright © 2024 Boltbase Contributors