Checks

Checks are scripts that test the environment from outside, the way a user would see it. They're what makes "same behavior on every target" verifiable.

checks:
  - checks/portal-answers.sh
  - checks/lan-is-blocked-from-outside.sh

Writing a check

A check is any executable script. It runs on the environment's networks, reaches machines by name, and exits 0 when the behavior holds:

#!/bin/sh
# The portal answers, and refuses a request without its API key.
set -eu
code=$(curl -s -o /dev/null -w '%{http_code}' http://web:8080/api/invoices/42)
[ "$code" = 401 ] || { echo "expected 401 without a key, got $code"; exit 1; }
curl -fsS -H "X-Api-Key: demo" http://web:8080/api/invoices/42 | grep -q amount
echo "portal answers and enforces its key"

Good checks test what should not happen too: a network that must stay unreachable, a service that must refuse anonymous access.

Where they run

Isoloom adds a small check runner to the targets it builds, attached to the environment's networks, so a project only ships the scripts. On Docker it's a check profile:

docker compose -f .isoloom/docker/compose.yml --profile check run --rm isoloom-check

In CI

The intended setup runs the checks on every change for the cheap target (Docker), and on the VM targets when the machines change. A target whose checks fail isn't offered.

Planned: the check runner for VM targets. On Vagrant today, run the scripts from one of the machines (vagrant ssh web -c 'cd /opt/isoloom && sh checks/…').