Quality Gates (CIT)

CIT (Container Integration Test) is the testing specification executed by dbuild during the test and ci-run commands. It defines what success looks like for each container image.

flowchart LR
    subgraph dbuild ["dbuild (The Engine)"]
        direction LR
        B[Build] --> T[Test]
        T --> P[Push]
    end

    subgraph CIT ["CIT (The Gates)"]
        direction TB
        T1[Shell]
        T2[Port]
        T3[Health]
        T4[Screenshot]
    end

    T -.-> CIT

While dbuild manages the container lifecycle (start, stop, cleanup), CIT defines the success criteria through a cumulative mode system.

Cumulative Modes

Each mode includes all checks from lower modes:

screenshot = health + visual regression
health     = port   + HTTP endpoint check
port       = shell  + TCP port listening
shell      = container starts successfully

Mode	What it checks	Use case
`shell`	Container starts, `echo ok` via exec	Base images, CLI tools
`port`	Shell + TCP port is listening	Services with network listeners
`health`	Port + HTTP endpoint returns non-error	Web apps with health endpoints
`screenshot`	Health + visual regression against baseline	Web UIs

Mode Auto-Detection

If no mode is set in config, CIT picks the highest applicable mode:

If a screenshot baseline image exists, use screenshot
If health is set, use health
If port is set, use port
Otherwise, use shell

Gate Details

Shell Test

Verifies the container starts successfully:

s6-overlay initializes
Services start without errors
echo ok succeeds via exec

This is the baseline gate -- every mode includes it.

Port Binding

Waits for the application to bind to a TCP port:

cit:
  port: 7878

CIT polls the socket until it becomes available or the wait timeout expires.

Health Check

Sends an HTTP GET request to the specified endpoint:

cit:
  port: 7878
  health: /ping

Expects a non-error HTTP response (2xx or 4xx). A 502 or 503 indicates the app isn't ready yet and CIT will retry.

For HTTPS-only applications:

cit:
  port: 8443
  health: /health
  https: true

Visual Regression (SSIM)

Captures a browser screenshot and compares it against a known-good baseline using Structural Similarity Index (SSIM):

cit:
  mode: screenshot
  port: 7878
  health: /ping
  screenshot_wait: 10

Threshold: SSIM >= 0.95 (95% structural similarity)
Catches UI regressions, broken CSS, missing assets
Uses scikit-image for SSIM comparison and Selenium + Chromium for capture
Requires pip install ".[dev]" on the dbuild installation
If screenshot dependencies are missing, the mode automatically downgrades to health

Baseline search order:

.daemonless/baseline-{tag}.png (per-variant)
.daemonless/baselines/baseline-{tag}.png
.daemonless/baseline.png (shared across variants)
.daemonless/baselines/baseline.png

If a baseline exists and no mode is configured, screenshot mode is auto-selected.

Readiness Detection

Before running port/health checks, CIT watches container logs for a readiness pattern. The default pattern matches common startup messages:

Warmup complete | services.d.*done | Application started | Startup complete | listening on | is ready

Override with the ready field:

cit:
  ready: "Server initialized"
  wait: 300

The wait timeout (default: 120 seconds) applies to the readiness check. If the pattern isn't seen within the timeout, CIT proceeds to port/health checks anyway -- the timeout is not fatal on its own.

Compose Testing

For multi-service stacks (e.g., app + database), set compose: true:

cit:
  compose: true
  port: 8080
  health: /api/health

This uses podman-compose with the compose file at .daemonless/compose.yaml or .daemonless/compose.yml. Shell exec tests are skipped for compose stacks since they don't support single-container exec.

Compose File for CIT

The compose file at .daemonless/compose.yaml is separate from the top-level compose.yaml. This lets you define a test-specific stack without modifying the production deployment file — for example, to include a local database or to override image references:

.daemonless/
└── compose.yaml   # CIT-only stack, used when cit.compose: true
compose.yaml       # Production deployment + x-daemonless metadata

If .daemonless/compose.yaml is absent when compose: true is set, the test fails immediately.

Testing Backends

By default, dbuild test runs the container using Podman. For images deployed to FreeBSD jails, you can also test with AppJail to validate behavior in a real jail context.

# Test with Podman only (default)
dbuild test

# Test with AppJail only
dbuild test --backend appjail

# Test with both Podman and AppJail
dbuild test --backend all

AppJail Backend

When --backend appjail is used, dbuild runs the container via appjail oci run instead of podman run. This exercises the full jail stack:

appjail oci run — starts an OCI jail directly from local Podman image storage
Shared host network — the jail is reachable at 127.0.0.1 on the configured port
Jail isolation — syscall restrictions, mount namespaces, and jail annotations all apply

AppJail backend is automatically selected when the image has appjail: true in compose.yaml and AppJail is installed on the host. If AppJail is configured but not installed, dbuild emits a warning and falls back to Podman — it does not error.

Note

compose: true is Podman-only. AppJail does not support multi-container stacks via compose, so the AppJail backend is skipped when compose: true is set.

Overriding Generated AppJail Files

dbuild generate produces three AppJail deployment files from bundled templates:

File	Purpose
`Makejail`	Jail build instructions (image source, `allow.*` params)
`appjail-director.yml`	Director deployment descriptor (networking, volumes, env)
`.env`	Environment variable defaults for director

If an image needs custom content in any of these files, place the override in .daemonless/appjail/:

.daemonless/
└── appjail/
    ├── Makejail              # replaces generated Makejail
    ├── appjail-director.yml  # replaces generated director config
    └── .env                  # replaces generated env defaults

Any file found in .daemonless/appjail/ is copied as-is; files not present fall back to the auto-generated template. Partial overrides are fine — you can override just Makejail and let the rest render from templates.

Backend Summary

Backend	Runtime	Use when
`podman`	`podman run`	Standard containers, all hosts
`appjail`	`appjail oci run`	Jail-deployed images on FreeBSD
`all`	Both	Full validation before release

Jail Annotations

Some applications require specific FreeBSD jail permissions to function:

cit:
  annotations:
    - "org.freebsd.jail.allow.mlock=true"
    - "org.freebsd.jail.allow.sysvipc=true"

Annotation	Required by
`allow.mlock`	.NET apps (Radarr, Sonarr, Prowlarr, Lidarr)
`allow.sysvipc`	PostgreSQL

These annotations are passed to podman run during testing. In production, they are set via --annotation in the deploy playbook.

Platform QA

CIT serves as a functional regression suite for the entire FreeBSD container stack:

FreeBSD 15 kernel -- validates syscalls, socket binding, process management
ocijail runtime -- ensures jail isolation works correctly
s6-overlay -- verifies init system behavior

Every image build runs CIT in a real FreeBSD VM, not emulation. The push step is unreachable if any gate fails, ensuring ghcr.io/daemonless/* contains only validated containers.

flowchart LR
    A[Build Image] --> B[Run CIT]
    B -->|All Pass| C[Push to ghcr.io]
    B -->|Any Fail| D[Build Fails]
    D --> E[No Push]

Configuration Reference

Full cit: schema for .daemonless/config.yaml:

cit:
  # Test mode: shell | port | health | screenshot
  # Auto-detected if omitted (see Mode Auto-Detection above)
  mode: health

  # TCP port to check (required for port, health, screenshot modes)
  port: 8080

  # Health endpoint path (required for health, screenshot modes)
  health: /health

  # Use HTTPS for health checks (default: false)
  https: false

  # Startup timeout in seconds (default: 120)
  wait: 120

  # Log pattern to wait for before testing (regex)
  ready: "Server started"

  # Use podman-compose for multi-service stacks (default: false)
  compose: false

  # Extra wait in seconds before screenshot capture
  screenshot_wait: 10

  # Jail annotations passed to the test container
  annotations:
    - "org.freebsd.jail.allow.mlock=true"
    - "org.freebsd.jail.allow.sysvipc=true"

Field	Default	Description
`mode`	Auto-detected	Test mode: `shell`, `port`, `health`, or `screenshot`
`port`		TCP port to check
`health`	`/`	Health endpoint path
`https`	`false`	Use HTTPS for health checks
`wait`	`120`	Startup timeout in seconds
`ready`	Built-in pattern	Log regex to wait for before testing
`compose`	`false`	Use podman-compose with `.daemonless/compose.yaml`
`screenshot_wait`		Extra seconds to wait before screenshot capture
`annotations`	`[]`	Jail annotations for the test container