Workflow Basics

Learn the fundamentals of writing Boltbase workflows.

Your First Workflow

Create hello.yaml:

steps:
  - command: echo "Hello from Boltbase!"

Run it:

boltbase start hello.yaml

Workflow Structure

A complete workflow contains:

# Metadata
name: data-pipeline
description: Process daily data
tags: [etl, production]

# Configuration  
schedule: "0 2 * * *"
params:
  - DATE: ${DATE:-today}

# Steps
steps:
  - name: process
    command: python process.py ${DATE}

# Handlers
handler_on:
  failure:
    command: notify-error.sh

Steps

The basic unit of execution.

Step Names

Step names are optional. When omitted, Boltbase automatically generates names based on the step type:

steps:
  - command: echo "First step"              # Auto-named: cmd_1
  - script: |                      # Auto-named: script_2
      echo "Multi-line"
      echo "Script"
  - name: explicit-name            # Explicit name
    command: echo "Third step"
  - type: http                     # Auto-named: http_4
    config:
      url: https://api.example.com
  - call: child-workflow            # Auto-named: dag_5

Auto-generated names follow the pattern {type}_{number}:

• cmd_N - Command steps
• script_N - Script steps
• http_N - HTTP steps
• dag_N - DAG steps
• container_N - Docker/container steps
• ssh_N - SSH steps
• mail_N - Mail steps
• jq_N - JQ steps

For parallel steps (see below), the pattern is parallel_{group}_{type}_{index}.

Shorthand Command Syntax

For simple commands, you can use an even more concise syntax:

steps:
  - command: echo "Hello World"
  - command: ls -la
  - command: python script.py

This is equivalent to:

type: graph
steps:
  - name: step 1
    command: echo "Hello World"
  - name: step 2
    command: ls -la
    depends: step 1
  - name: step 3
    command: python script.py
    depends: step 2

Multiple Commands

Multiple commands share the same step configuration:

steps:
  - name: build-and-test
    command:
      - npm install
      - npm run build
      - npm test
    env:
      - NODE_ENV: production
    working_dir: /app
    retry_policy:
      limit: 3

Instead of duplicating env, working_dir, retry_policy, preconditions, container, etc. across multiple steps, combine commands into one step.

Commands run in order and stop on first failure. Retries restart from the first command.

Trade-off: You lose the ability to retry or resume from the middle of the command list. If you need granular control over individual command retries, use separate steps.

Supported step types: shell, command, docker, container, ssh

Not supported: jq, http, archive, mail, github_action, dag (these only accept single commands)

Multi-line Scripts

steps:
  - script: |
      #!/bin/bash
      set -e

      echo "Processing..."
      python analyze.py data.csv
      echo "Complete"

If you omit shell, Boltbase uses the interpreter declared in the script's shebang (#!) when present.

Shell Selection

Set a default shell for every step at the DAG level, and override it per step when needed:

shell: ["/bin/bash", "-e", "-u"]  # Default shell + args for the whole workflow
steps:
  - name: bash-task
    command: echo "Runs with bash -e -u"

  - name: zsh-override
    shell: /bin/zsh                # Step-level override
    command: echo "Uses zsh instead"

The shell value accepts either a string ("bash -e") or an array (["bash", "-e"]). Arrays avoid quoting issues when you need multiple flags.

When you omit a step-level shell, Boltbase runs through the DAG shell (or system default) and automatically adds -e on Unix-like shells so scripts stop on first error. If you explicitly set shell on a step, include -e yourself if you want the same errexit behavior.

steps:
  - shell: python3
    script: |
      import pandas as pd
      df = pd.read_csv('data.csv')
      print(df.head())

Dependencies

Steps run sequentially by default. Use depends for parallel execution or to control order.

steps:
  - name: download
    command: wget data.csv
    
  - name: process
    command: python process.py
    
  - name: upload
    command: aws s3 cp output.csv s3://bucket/

Parallel Execution

You can run steps in parallel using explicit dependencies:

type: graph
steps:
  - name: setup
    command: echo "Setup"

  - name: task1
    command: echo "Task 1"
    depends: setup

  - name: task2
    command: echo "Task 2"
    depends: setup

  - name: finish
    command: echo "All tasks complete"
    depends: [task1, task2]

Working Directory

Set where commands execute:

steps:
  - name: in-project
    working_dir: /home/user/project
    command: python main.py
    
  - name: in-data
    working_dir: /data/input
    command: ls -la

Environment Variables

Define environment variables at DAG-level or step-level:

env:
  - API_KEY: secret123
  - ENV: production

steps:
  - name: dev-test
    command: echo "Running in $ENV"
    env:
      - ENV: development  # Overrides DAG-level

TIP

Boltbase filters system environment variables for security. See Environment Variables for details on filtering, inheritance, and .env file support.

Capturing Output

Store command output in variables:

steps:
  - name: get-version
    command: git rev-parse --short HEAD
    output: VERSION
    
  - name: build
    command: docker build -t app:${VERSION} .

Basic Error Handling

Continue on Failure

steps:
  - name: optional-step
    command: maybe-fails.sh
    continue_on:
      failure: true
      
  - name: always-runs
    command: cleanup.sh

Simple Retry

steps:
  - name: flaky-api
    command: curl https://unstable-api.com
    retry_policy:
      limit: 3

Timeouts

Prevent steps from running forever:

steps:
  - name: long-task
    command: echo "Processing data"
    timeout_sec: 300  # 5 minutes

Step Descriptions

Document your steps:

steps:
  - name: etl-process
    description: |
      Extract data from API, transform to CSV,
      and load into data warehouse
    command: python etl.py

Tags and Organization

Group related workflows:

name: customer-report
tags: 
  - reports
  - customer
  - daily

group: Analytics  # UI grouping

See Also

• Control Flow - Conditionals and loops
• Data & Variables - Pass data between steps
• Error Handling - Advanced error recovery
• Parameters - Make workflows configurable