Running in Docker

Base images

DipDup provides multiple prebuilt images for different environments hosted on Docker Hub. Choose the one according to your needs from the table below.

The default DipDup image is suitable for development and testing. It also includes some tools to make package management easier. If unsure, use this image.

-slim image based on Alpine Linux, thus is much smaller than the default one. Also, it doesn't include codegen functionality (init command, unlikely to be useful in production). This image will eventually become the default one.

-pytezos image includes pre-installed PyTezos library. DipDup doesn't provide any further PyTezos integration. Having some patience, you can build a trading robot or something like that using this image.

Nightly builds (ghcr.io)

In addition to Docker Hub we also publish images on GitHub Container Registry. Builds are triggered on push to any branch for developers' convenience, but only Alpine images are published. Do not use nightlies in production!

# Latest image for `aux/arm64` branch
FROM ghcr.io/dipdup-io/dipdup:aux-arm64

Writing Dockerfile

Start with creating .dockerignore for your project if it's missing.

# Ignore all
*

# Add build files
!Makefile
!pyproject.toml
!poetry.lock
!requirements**
!README.md

# Add code
!src

# Add configs
!*.yml

# Ignore caches
**/.mypy_cache
**/.pytest_cache
**/__pycache__

A typical Dockerfile looks like this:

FROM dipdup/dipdup:6
# FROM dipdup/dipdup:6-pytezos
# FROM dipdup/dipdup:6-slim

# Optional: install additional dependencies using poetry
# COPY pyproject.toml poetry.lock .
# RUN install_dependencies

# Optional: install additional dependencies using pip
# COPY requirements.txt .
# RUN install_dependencies requirements.txt

COPY . .

Note that Poetry integration is not available in the slim image.

Deploying with docker-compose

Make sure you have docker run and docker-compose installed.

Example docker-compose.yml file:

version: "3.8"

services:
  dipdup:
    build: .
    depends_on:
      - db
    command: ["-c", "dipdup.yml", "-c", "dipdup.prod.yml", "run"]
    restart: always
    environment:
      - POSTGRES_PASSWORD=${POSTGRES_PASSWORD:-changeme}
      - ADMIN_SECRET=${ADMIN_SECRET:-changeme}
    ports:
      - 127.0.0.1:9000:9000

  db:
    image: postgres:14
    ports:
      - 127.0.0.1:5432:5432
    volumes:
      - db:/var/lib/postgresql/data
    restart: always
    environment:
      - POSTGRES_USER=dipdup
      - POSTGRES_DB=dipdup
      - POSTGRES_PASSWORD=${POSTGRES_PASSWORD:-changeme}
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U dipdup"]
      interval: 10s
      timeout: 5s
      retries: 5

  hasura:
    image: hasura/graphql-engine:v2.28.0
    ports:
      - 127.0.0.1:8080:8080
    depends_on:
      - db
    restart: always
    environment:
      - HASURA_GRAPHQL_DATABASE_URL=postgres://dipdup:${POSTGRES_PASSWORD:-changeme}@db:5432
      - HASURA_GRAPHQL_ENABLE_CONSOLE=true
      - HASURA_GRAPHQL_DEV_MODE=true
      - HASURA_GRAPHQL_ENABLED_LOG_TYPES=startup, http-log, webhook-log, websocket-log, query-log
      - HASURA_GRAPHQL_ADMIN_SECRET=${HASURA_SECRET:-changeme}
      - HASURA_GRAPHQL_UNAUTHORIZED_ROLE=user
      - HASURA_GRAPHQL_STRINGIFY_NUMERIC_TYPES=true

volumes:
  db:

Environment variables are expanded in the DipDup config file; Postgres password and Hasura secret are forwarded in this example.

Create a separate dipdup.<environment>.yml file for this stack:

database:
  kind: postgres
  host: db
  port: 5432
  user: ${POSTGRES_USER:-dipdup}
  password: ${POSTGRES_PASSWORD:-changeme}
  database: ${POSTGRES_DB:-dipdup}

hasura:
  url: http://hasura:8080
  admin_secret: ${HASURA_SECRET:-changeme}
  allow_aggregations: false
  camel_case: true

sentry:
  dsn: ${SENTRY_DSN:-""}
  environment: ${SENTRY_ENVIRONMENT:-prod}

prometheus:
  host: 0.0.0.0

advanced:
  early_realtime: True
  crash_reporting: False

Note the hostnames (resolved in the docker network) and environment variables (expanded by DipDup).

Build and run the containers:

docker-compose up -d --build

Try lazydocker tool to manage Docker containers interactively.

Deploying with Docker Swarm

Example stack:

version: "3.8"

services:
  dipdup:
    image: ${DOCKER_REGISTRY:-ghcr.io}/dipdup-io/dipdup:${TAG:-master}
    depends_on:
      - db
      - hasura
    command: ["-c", "dipdup.yml", "-c", "dipdup.prod.yml", "run"]
    environment:
      - "POSTGRES_USER=dipdup"
      - "POSTGRES_PASSWORD=changeme"
      - "POSTGRES_DB=dipdup"
      - "HASURA_SECRET=changeme"
    networks:
      - dipdup-private
      - prometheus-private
    deploy:
      mode: replicated
      replicas: ${INDEXER_ENABLED:-1}
      labels:
        - prometheus-job=${SERVICE}
        - prometheus-port=8000
      placement: &placement
        constraints:
          - node.labels.${SERVICE} == true
    logging: &logging
      driver: "json-file"
      options:
        max-size: "10m"
        max-file: "10"
        tag: "\{\{.Name\}\}.\{\{.ImageID\}\}"

  db:
    image: postgres:14
    volumes:
      - db:/var/lib/postgresql/data
    environment: 
      - POSTGRES_USER=dipdup
      - POSTGRES_DB=dipdup
      - POSTGRES_PASSWORD=${POSTGRES_PASSWORD:-changeme}
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 10s
      timeout: 5s
      retries: 5
    networks:
      - dipdup-private
    deploy:
      mode: replicated
      replicas: 1
      placement: *placement
    logging: *logging

  hasura:
    image: hasura/graphql-engine:v2.28.0
    depends_on:
      - db
    environment:
      - HASURA_GRAPHQL_DATABASE_URL=postgres://dipdup:${POSTGRES_PASSWORD:-changeme}@db:5432
      - HASURA_GRAPHQL_ENABLE_CONSOLE=true
      - HASURA_GRAPHQL_DEV_MODE=false
      - HASURA_GRAPHQL_ENABLED_LOG_TYPES=startup, http-log, websocket-log, query-log
      - HASURA_GRAPHQL_LOG_LEVEL=warn
      - HASURA_GRAPHQL_ENABLE_TELEMETRY=false
      - HASURA_GRAPHQL_ADMIN_SECRET=${HASURA_SECRET}
      - HASURA_GRAPHQL_UNAUTHORIZED_ROLE=user
      - HASURA_GRAPHQL_STRINGIFY_NUMERIC_TYPES=true
    networks:
      - dipdup-private
      - traefik-public
    deploy:
      mode: replicated
      replicas: 1
      labels:
        - traefik.enable=true
        - traefik.http.services.${SERVICE}.loadbalancer.server.port=8080
        - "traefik.http.routers.${SERVICE}.rule=Host(`${HOST}`) && (PathPrefix(`/v1/graphql`) || PathPrefix(`/api/rest`))"
        - traefik.http.routers.${SERVICE}.entrypoints=http,${INGRESS:-ingress}
        - "traefik.http.routers.${SERVICE}-console.rule=Host(`${SERVICE}.${SWARM_ROOT_DOMAIN}`)"
        - traefik.http.routers.${SERVICE}-console.entrypoints=https
        - traefik.http.middlewares.${SERVICE}-console.headers.customrequestheaders.X-Hasura-Admin-Secret=${HASURA_SECRET}
        - traefik.http.routers.${SERVICE}-console.middlewares=authelia@docker,${SERVICE}-console
      placement: *placement
    logging: *logging


volumes:
  db:

networks:
  dipdup-private:
  traefik-public:
    external: true
  prometheus-private:
    external: true