A lightweight and powerful workflow engine for enterprise & small teams

Deployable on-premise, in the cloud, or on IoT devices. Low-code declarative workflow definitions that anyone can understand.

Docs | Quick Start | Features | Installation | Community

Dagu is a powerful workflow engine designed to be deployable in environments where Airflow cannot be installed, such as small devices, on-premise servers, and legacy systems. It allows you to declaratively define any batch job as a single DAG (Directed Acyclic Graph) in a simple YAML format.

steps:
  - name: step1
    command: sleep 1 && echo "Hello, Dagu!"
    
  - name: step2
    command: sleep 1 && echo "This is a second step"

By declaratively defining the processes within a job, complex workflows become visualized, making troubleshooting and recovery easier. Viewing log and retry can be performed from the Web UI, eliminating the need to manually log into a server via SSH.

It is equipped with many features to meet the highly detailed requirements of enterprise environments. It operates even in environments without internet access and, being statically compiled, includes all dependencies, allowing it to be used in any environment, including on-premise, cloud, and IoT devices. It is a lightweight workflow engine that meets enterprise requirements.

Note: For a list of features, please refer to the documentation.

Workflow jobs are defined as commands. Therefore, legacy scripts that have been in operation for a long time within a company or organization can be used as-is without modification. There is no need to learn a complex new language, and you can start using it right away.

Dagu is designed for small teams of 1-3 people to easily manage complex workflows. It aims to be an ideal choice for teams that find large-scale, high-cost infrastructure like Airflow to be overkill and are looking for a simpler solution.

CLI Demo: Create a simple DAG workflow and execute it using the command line interface.

Web UI Demo: Create and manage workflows using the web interface, including real-time monitoring and control.

Docs on CLI

Docs on Web UI

• OpenID Connect (OIDC) Support: OpenID Connect authentication for secure access (implemented by @Arvintian)
• Distributed Execution: Run steps in a DAG across multiple machines

• OpenTelemetry Support (OTEL): Distributed tracing with W3C trace context propagation (requested by @jeremydelattre59)
• Windows Support (Beta): Initial PowerShell and cmd.exe compatibility - basic functionality works but may have limitations (@pdoronila)
• Hierarchical DAG: Nested workflows with parameter passing
• Individual Step Retry: Retry individual steps without re-running entire workflow (@thefishhat)
• Repeat Policy Enhancement: The 'while'/'until' modes are added to repeat policy setting (@thefishhat)
• Queueing functionality: Enqueue DAGs without running immediately to limit resource usage of the workflow (@kriyanshii)

Full changelog

Dagu is designed to orchestrate workflows across various domains, particularly those involving multi-step batch jobs and complex data dependencies. Example applications include:

• AI/ML & Data Science - Automating machine learning workflows, including data ingestion, feature engineering, model training, validation, and deployment.
• Geospatial & Environmental Analysis - Processing datasets from sources such as satellites (earth observation), aerial/terrestrial sensors, seismic surveys, and ground-based radar. Common uses include numerical weather prediction and natural resource management.
• Finance & Trading - Implementing time-sensitive workflows for stock market analysis, quantitative modeling, risk assessment, and report generation.
• Medical Imaging & Bioinformatics - Creating pipelines to process and analyze large volumes of medical scans (e.g., MRI, CT) or genomic data for clinical and research purposes.
• Data Engineering (ETL/ELT) - Building, scheduling, and monitoring pipelines for moving and transforming data between systems like databases, data warehouses, and data lakes.
• IoT & Edge Computing - Orchestrating workflows that collect, process, and analyze data from distributed IoT devices, sensors, and edge nodes.
• Media & Content Processing - Automating workflows for video transcoding, image processing, and content delivery, including tasks like format conversion, compression, and metadata extraction.

• DAG definition - Express complex dependencies in readable YAML
• Scheduling - Cron expressions with timezone support
• Queueing - Control concurrency with named queues
• Error handling - Retries, failure handlers, cleanup hooks
• Conditional execution - Run steps based on conditions
• Parallel execution - Control concurrent step execution
• Variables & Parameters - Pass data between steps, parameterize workflows
• Docker support - Run steps in containers
• SSH executor - Execute commands on remote hosts
• HTTP requests - Integrate with APIs
• Email notifications - SMTP integration
• Hierarchical workflows - Nest DAGs to any depth
• Distributed execution - Scale across multiple machines
• Authentication - Basic auth, API tokens, TLS
• Web UI - Real-time monitoring and control
• REST API - Full programmatic access

# Install
curl -L https://raw.githubusercontent.com/dagu-org/dagu/main/scripts/installer.sh | bash
# Create dagu configuration directory
mkdir -p ~/.config/dagu/dags
# Create your first workflow
mkdir -p ~/.config/dagu/dags
cat > ~/.config/dagu/dags/hello.yaml << 'EOF'
steps:
  - name: hello
    command: echo "Hello from Dagu!"
    
  - name: world  
    command: echo "Running step 2"
EOF
# Execute it
dagu start hello
# Check the status
dagu status hello
# Start the web UI
dagu start-all
# Visit https://localhost:8080

# Latest
curl -L https://raw.githubusercontent.com/dagu-org/dagu/main/scripts/installer.sh | bash
# Specific version
curl -L https://raw.githubusercontent.com/dagu-org/dagu/main/scripts/installer.sh | bash -s -- --version v1.17.0
# Install to a specific directory
curl -L https://raw.githubusercontent.com/dagu-org/dagu/main/scripts/installer.sh | bash -s -- --prefix /path/to/install
# Homebrew
brew install dagu-org/brew/dagu
# Homebrew (upgrade)
brew upgrade dagu-org/brew/dagu

docker run -d \
  --name dagu \
  -p 8080:8080 \
  -v ~/.dagu:/var/lib/dagu \
  ghcr.io/dagu-org/dagu:latest dagu start-all

Download from releases and add to PATH.

• Getting Started - Tutorial and first steps
• Core Concepts - Architecture and design
• Writing Workflows - Complete authoring guide
• CLI Reference - Command-line usage
• API Reference - REST API documentation
• Configuration - Configuration options
• Changelog - Release notes and version history

Find more in our examples documentation.

name: daily-etl
schedule: "0 2 * * *"
steps:
  - name: extract
    command: python extract.py
    output: DATA_FILE
    
  - name: validate
    command: python validate.py ${DATA_FILE}
    
  - name: transform
    command: python transform.py ${DATA_FILE}
    retryPolicy:
      limit: 3
      
  - name: load
    command: python load.py ${DATA_FILE}

steps:
  - name: data-pipeline
    run: etl
    params: "ENV=prod REGION=us-west-2"
    
  - name: parallel-jobs
    run: batch
    parallel:
      items: ["job1", "job2", "job3"]
      maxConcurrency: 2
    params: "JOB=${ITEM}"
---
name: etl
params:
  - ENV
  - REGION
steps:
  - name: process
    command: python etl.py --env ${ENV} --region ${REGION}
---
name: batch
params:
  - JOB
steps:
  - name: process
    command: python process.py --job ${JOB}

name: ml-pipeline
steps:
  - name: prepare-data
    executor:
      type: docker
      config:
        image: python:3.11
        autoRemove: true
        volumes:
          - /data:/data
    command: python prepare.py
    
  - name: train-model
    executor:
      type: docker
      config:
        image: tensorflow/tensorflow:latest-gpu
    command: python train.py
    
  - name: deploy
    command: kubectl apply -f model-deployment.yaml
    preconditions:
      - condition: "`date +%u`"
        expected: "re:[1-5]"  # Weekdays only

name: distributed-workflow
steps:
  - name: gpu-training
    run: gpu-training
    params: "REGION=${ITEM}"
    parallel: ["us-east-1", "eu-west-1"]
    
---
name: gpu-training
params: "REGION=region"
workerSelector:
  gpu: "true"
steps:
  - name: gpu-training
    command: python train_model.py ${REGION}

• Run steps in a DAG across multiple machines (distributed execution) - Available in v1.18.0
• Track artifacts by dropping files in $DAGU_ARTIFACTS
• Pause executions for webhooks, approvals, or any event (human-in-the-loop, event-driven workflows)
• Integrate with AI agents via MCP (Model Context Protocol)

Prerequisites: Go 1.24+, Node.js, pnpm

git clone https://github.com/dagu-org/dagu.git && cd dagu
make build
make run

• @Arvintian for implementing OpenID Connect (OIDC) support and HTTP executor
• @kriyanshii for implementing the queueing functionality
• @pdoronila for adding Windows support
• @thefishhat for various improvements on the scheduler, retry policy, and more
• @arky for managing and improving the community, documentation, and examples
• @ghansham for providing valuable feedback and suggestions

Any contributions are welcome! If you have ideas, suggestions, or improvements, please open an issue or submit a pull request.

• Discord Community
• GitHub Issues

Thanks for supporting Dagu’s development! Join our supporters: GitHub Sponsors

GNU GPLv3 - See LICENSE