Skip to content
Project VELAProject VELA

VELA demo runbook

How to run a compelling, reliable VELA demo end to end — the 90-second story, exact commands, the live-LLM “wow”, both frontends, the ESA-ADB detection benchmark, and what to do when something misbehaves on stage.

Golden rule for a live demo: the deterministic air-gap path is the reliable demo — it always produces the same cited investigation with no network, no keys, no surprises. The live-LLM path is the wow — real, but it depends on a provider being reachable. Rehearse both; lead with whichever your audience needs and keep the deterministic path one keystroke away as the fallback.

Everything below is advisory-only by construction: VELA reports, cites, and drafts a recovery procedure for a human to execute. It never commands the spacecraft. All spacecraft data is the synthetic VELA-REF-1 reference spacecraft plus seeded replay scenarios — never a real mission.

0. One-time setup

Terminal window
uv venv --python 3.12 && uv pip install -e ".[dev]"   # Python 3.12 in .venv; installs httpx too
npm install                                            # root: husky hooks (once)

Use make (it hardcodes PY = .venv/bin/python) or activate the venv first (. .venv/bin/activate). The optional extras you might add for a fuller demo:

  • .[api] — the HTTP service (python -m vela.api), needed for the operator console.
  • .[cloud] — the Anthropic reasoning path (langchain-anthropic). The provider path (Gemini/Sarvam/OpenAI-compatible) needs only httpx, which the .[dev] install already brings.
  • .[backends] — real Neo4j + Qdrant stores (not required for the demo; in-memory is the default).

Confirm you are ready:

Terminal window
python -m vela.cli scenarios     # lists the 10 scenarios
make demo                        # the reliable deterministic investigation, held for approval

1. The 90-second story

The narrative arc, mapped to what appears on screen. Run one investigation and talk over it:

Terminal window
python -m vela.cli investigate --scenario CELL_DEGRADATION --approve --replay

  1. An anomaly arrives. Battery voltage collapses under transmit load with rising cell temperature. VELA forms a triggering AnomalyEvent — the same event a detector or an operator annotation would raise.

  2. A cited root cause + a drafted procedure. The graph runs a fixed, auditable path (supervisor → triage → investigator → validator → drafter → approval) and prints a root-cause narrative and a step-by-step recovery procedure — with every claim cited to the spacecraft’s own knowledge: FMECA failure modes, ops-manual sections, telemetry windows. This is the differentiator: not “an LLM thinks it’s the battery,” but a claim you can click through to the FMECA entry and the manual section that support it.

  3. The grounding gates catch fabrication. VELA prints, deterministically, lines like:

    Citation integrity: PASS (N citation(s) checked)
    Releasable to operator: yes
    Telemetry-value grounding [advisory]: PASS (N numeric assertion(s) checked)
    Citation entailment [advisory, lexical]: PASS (N doc span(s) checked)

    The rule is symmetric and load-bearing: a grounded step with no citation is a violation (it is forced to UNGROUNDED), and an UNGROUNDED step that carries a citation is also a violation. A single ungrounded step flips “Releasable to operator” to no (ungrounded steps blocked) — the fabrication never reaches the operator. A hallucinated reference is a sev-1 and makes investigate exit non-zero, failing any pipeline it runs in. Narrate this as the safety mechanism: VELA would rather say “I can’t support that” than hand an operator a confident, unsupported claim.

  4. A human decides. Without --approve, the drafter pauses at a real approval gate and prints Human approval: HELD — re-run with --approve to release the draft. Releasing resumes the exact checkpointed graph — a genuine resume, not a re-run. This is the advisory-only boundary made concrete: the one action a human takes is releasing a draft, never commanding.

  5. Audit and replay. --replay prints the forensic checkpoint sequence (Forensic replay — N checkpoints: … -> … -> …) and the full agent trace. Every investigation is one auditable record.

Differentiators to name out loud: citations to the spacecraft’s own FMECA and manuals; an UNGROUNDED step being blocked from release; the telecommand-response suppression (§2.2); the air-gap posture (deterministic, zero egress); and never commands — advisory only, enforced by a committed guardrail, not good intentions.

Want the artifacts on disk to show afterwards?

Terminal window
python -m vela.cli investigate --scenario CELL_DEGRADATION --approve --out ./out
# writes ./out/<investigation-id>.md and ./out/<investigation-id>.audit.json

2. CLI demo — the three best scenarios

Default mode is airgap (deterministic). Add --mode cloud for the deployment posture; add the live-LLM env from §3 to reason with a real model. Full scenario list: python -m vela.cli scenarios.

2.1 CELL_DEGRADATION — the full cited flow (EPS, warning)

The headline demo: a real anomaly (battery capacity fade under transmit load) that produces a cited narrative and a drafted recovery procedure held for approval.

Terminal window
# Deterministic (reliable): held for approval, then released
python -m vela.cli investigate --scenario CELL_DEGRADATION
python -m vela.cli investigate --scenario CELL_DEGRADATION --approve --replay


# Cloud deployment posture (still deterministic unless you set the §3 env)
python -m vela.cli investigate --scenario CELL_DEGRADATION --mode cloud --approve --replay

2.2 SUPPRESSED_MOMENTUM_DUMP — the telecommand filter kills a false alarm (AOCS)

Wide reaction-wheel speed swings look alarming — but they immediately follow a desaturation telecommand, so they are the expected response, not a fault. VELA’s telecommand-response filter suppresses the deviation instead of raising a spurious investigation. This is the “check get_command_history before crying hardware failure” discipline, automated.

Terminal window
# The investigation is suppressed at the supervisor (no false procedure drafted):
python -m vela.cli investigate --scenario SUPPRESSED_MOMENTUM_DUMP


# See the same suppression at the detector level (note the SUPPRESSED tag):
python -m vela.cli detect --scenario SUPPRESSED_MOMENTUM_DUMP --method ecod
python -m vela.cli detect --scenario SUPPRESSED_MOMENTUM_DUMP --method graph

Contrast it live with a scenario that is not command-explained (so you show the filter is discriminating, not just silencing everything):

Terminal window
python -m vela.cli detect --scenario WHEEL_FRICTION --method graph

2.3 BATTERY_OVERTEMP — a critical severity (EPS)

Cell temperature climbs past its limit while bus voltage stays nominal — a thermal-runaway precursor, the highest severity in the deck. Good for showing VELA escalates appropriately.

Terminal window
python -m vela.cli investigate --scenario BATTERY_OVERTEMP --approve --replay
python -m vela.cli investigate --scenario BATTERY_OVERTEMP --mode cloud --approve --replay

The three detection trigger paths (optional depth)

Every scenario can be triggered three ways, all converging on the same AnomalyEvent:

Terminal window
python -m vela.cli investigate --scenario CELL_DEGRADATION --detector annotation   # default
python -m vela.cli investigate --scenario MEMORY_SEU_STORM --detector graph         # KG-aggregated
python -m vela.cli detect      --scenario WHEEL_FRICTION   --method ecod            # per-channel

graph aggregates per-channel shift scores through the knowledge graph, catching distributed subsystem faults that per-channel ECOD misses — the training-free stand-in for the learned detector.

3. Live-LLM demo — the “wow”

Reason with a real model. This is a deployment posture, not a different pipeline: the same vela.cli investigate runs, only the reasoning backend changes, and it is selected entirely by environment. airgap mode refuses any non-deterministic backend (no egress path), so the live path requires cloud mode (--mode cloud).

Fail-closed is a feature to narrate. If the model errors, times out, returns malformed output, or the key/token is missing, the reasoning layer raises internally and the drafter degrades the verdict to INSUFFICIENT rather than emitting a confident, ungrounded answer. On top of that, the same release-blocking grounding gates (telemetry-value + citation-entailment) run on the model’s output, so a fabricated value or an unsupported citation is forced UNGROUNDED regardless of which provider produced it. The live model gets to be persuasive; it does not get to be unaccountable.

3.1 Gemini via Vertex AI (ADC bearer token)

The api-key env holds a short-lived gcloud auth print-access-token bearer; the project is your GCP project. No Google SDK required — the provider path calls Vertex generateContent over httpx.

Terminal window
export VELA_REASONING=provider
export VELA_REASONING_TRANSPORT=vertex
export VELA_REASONING_PROJECT=<your-gcp-project>
export VELA_REASONING_MODEL=gemini-3.6-flash
export VELA_REASONING_API_KEY_ENV=GOOGLE_TOKEN      # NAME of the env var holding the token
export GOOGLE_TOKEN="$(gcloud auth print-access-token)"
export VELA_REASONING_PROVIDER=gemini               # optional: the label shown in the report


python -m vela.cli investigate --scenario CELL_DEGRADATION --mode cloud --approve --replay
# or simply:  make demo-cloud

ADC tokens expire (~1 hour). If a run fails auth, re-run gcloud auth print-access-token and re-export GOOGLE_TOKEN.

3.2 Sarvam (or any OpenAI-compatible endpoint)

The openai transport (the default) posts to <endpoint>/chat/completions. This covers Sarvam, OpenAI, Gemini’s OpenAI-compatible endpoint, or a local vLLM/Ollama serving open weights (a sovereign / air-gap-adjacent posture — no public egress).

Terminal window
export VELA_REASONING=provider
export VELA_REASONING_TRANSPORT=openai
export VELA_REASONING_ENDPOINT=https://api.sarvam.ai/v1   # OpenAI-compatible base URL
export VELA_REASONING_MODEL=<provider-model-id>
export VELA_REASONING_API_KEY_ENV=SARVAM_API_KEY          # NAME of the env var holding the key
export SARVAM_API_KEY=<your-key>
export VELA_REASONING_PROVIDER=sarvam                      # optional: report label


python -m vela.cli investigate --scenario BATTERY_OVERTEMP --mode cloud --approve --replay
# or:  make demo-cloud DEMO_SCENARIO=BATTERY_OVERTEMP

3.3 Anthropic (Claude)

Terminal window
export VELA_MODE=cloud VELA_REASONING=anthropic
export ANTHROPIC_API_KEY=<your-key>       # needs the .[cloud] extra: uv pip install -e ".[cloud]"
python -m vela.cli investigate --scenario CELL_DEGRADATION --mode cloud --approve --replay
# or:  make demo-cloud

3.4 make demo-cloud

make demo-cloud composes the same investigate command, parameterised by the env above. With no provider env set it prints a clear message and exits — it never silently falls back to deterministic:

Terminal window
make demo-cloud                                   # prints setup help + exits if no provider env
VELA_REASONING=provider … make demo-cloud         # runs the live path (see §3.1 / §3.2)
make demo-cloud DEMO_SCENARIO=BATTERY_OVERTEMP     # pick the scenario

The report footer prints Reasoning: <label> · mode: cloud, so the audience can see a real model (gemini / sarvam / anthropic) drove the reasoning — while the citation and grounding verdicts above it prove it stayed accountable.

4. The operator console (apps/console/)

VELA has one UI: the production, advisory-only React console over the investigation service. It renders exactly what the service’s cited InvestigationReport contains — the verdict, the citation-integrity + release status, the narrative, the drafted procedure with each step’s citations (any UNGROUNDED step flagged and blocked), the assumed channel→sensor bindings, the approval gate, and the audit/replay view with a backend-readiness pill. Its one action is releasing a held draft — never commanding.

Terminal window
# Terminal A — the API (needs the .[api] extra):
uv pip install -e ".[api]"
python -m vela.api                 # serves http://127.0.0.1:8000  (interactive docs at /docs)


# Terminal B — the console:
cd apps/console && npm ci && npm run dev   # http://localhost:5173

The console opens in offline demo mode (in-memory fixtures — great when there is no backend or the wifi is hostile). Switch the source selector to live service to drive the running API. For the release step against the live API, note the approval resumes an in-process checkpoint — keep the same python -m vela.api process running between launch and approval.

An earlier investor-facing demo (apps/cockpit-poc/, React/Gemini) has been retired; its worthwhile surfaces — the reasoning-pipeline DAG, the severity verdict hero, the scenario card list — were ported into the console and rebound to the real contracts. The console is now the only UI.

5. ESA-ADB detection benchmark (honest framing)

VELA can score its unsupervised detectors against the real ESA-ADB anomaly labels. This makes detection real and measurable; it does not produce a cited investigation over ESA-ADB — that needs a per-mission knowledge graph, and VELA-REF-1 is a synthetic reference spacecraft.

The dataset is a separate operator download (large; git-ignored; never fetched at runtime):

Terminal window
make data                                              # downloads + checksum-verifies ESA-ADB
# or point at a disk with headroom:  make data ESA_DATA_DIR=/mnt/big/data

Then run the real-label detection benchmark. At true ESA-ADB scale (~15M points/channel) you must resample; --targets-only scores just the channels the manifest marks Target=YES:

Terminal window
VELA_ESA_DATA_DIR=data python -m vela.eval detect --mission ESA-Mission1 --resample 1h --targets-only
# writes results/bench/detection-ESA-Mission1.json  (precision / recall / F0.5, event-wise)

The honest caveats — state these; do not overclaim:

  • Resolution-limited by design. Resampling to a coarser cadence (e.g. 1h) is required for the pure-Python detectors; anomalies span days so detectability is preserved, but this is not a native-resolution result. Do not claim to beat the ESA-ADB paper — this is a working, honest baseline on downsampled data, not a state-of-the-art submission.
  • Detection only. The benchmark reports detection metrics it actually computes (F0.5 weights precision because operators drown in false alarms). A cited investigation over ESA-ADB is out of scope. See ADR 0017.
  • Attribution required. ESA-ADB is CC BY 3.0 IGO (Zenodo DOI 10.5281/zenodo.12528696). Keep the attribution wherever the data or derived metrics appear.

Separately, make bench runs the pipeline benchmark over the synthetic deck in both modes and verifies cloud/airgap parity → results/bench/<sha>.json. That is a determinism/parity check, not an ESA-ADB detection score — don’t conflate the two.

6. Pre-demo checklist

  • uv pip install -e ".[dev]" done; .venv active or you drive everything through make.
  • python -m vela.cli scenarios lists 10 scenarios.
  • make demo runs clean end to end (this is your reliable fallback — rehearse it).
  • The three scenarios you’ll show (§2) each run and look right on this machine.
  • If doing the live-LLM path: all VELA_REASONING_* env set; the key/token env is set and fresh (ADC tokens expire ~1h); the endpoint/project is reachable from the demo network. Do a dry run of make demo-cloud minutes before, not on stage.
  • If doing the console: .[api] installed, python -m vela.api running, npm ci done in apps/console, source selector on live service, and the API left running between launch and approval.
  • Terminal font large; --replay handy to show the audit trail.

7. Failure recovery on stage

  • Live model errors / times out / auth fails. This is safe — the drafter fails closed to INSUFFICIENT rather than fabricating. You can narrate that as the safety story, then immediately fall back to the reliable path: drop --mode cloud / unset VELA_REASONING, or just run make demo (deterministic air-gap). Same graph, same cited report, zero dependencies.
  • airgap mode requires reasoning='deterministic'. You left VELA_REASONING set while running an air-gap command. Unset it (unset VELA_REASONING) or run the live path with --mode cloud.
  • Vertex token expired. Re-run export GOOGLE_TOKEN="$(gcloud auth print-access-token)".
  • httpx / provider import error. The provider path needs httpx — uv pip install -e ".[dev]" (or add httpx). The Anthropic path needs .[cloud].
  • API port already in use. Rebind: VELA_API_PORT=8010 python -m vela.api (and point the console there). VELA_API_HOST is available too.
  • Console shows nothing / backend down. The console’s offline demo mode uses fixtures and needs no backend — leave the source selector on demo to keep presenting while you fix the API.
  • Approval “resume” does nothing on the live API. Checkpoints are process-local; don’t restart python -m vela.api between launching an investigation and approving it.
  • When in doubt, go deterministic. make demo is the demo that always works.

Command reference

Terminal window
# Scenarios & investigations
python -m vela.cli scenarios
python -m vela.cli investigate --scenario <KEY> [--mode cloud|airgap] [--detector annotation|ecod|graph]
                               [--approve] [--approver NAME] [--replay] [--out DIR]
python -m vela.cli detect      --scenario <KEY> [--method ecod|graph] [--mode cloud|airgap]


# Make convenience targets
make demo                                   # deterministic investigation (reliable)
make demo-cloud                             # live-LLM investigation (needs §3 env)
make demo-cloud DEMO_SCENARIO=BATTERY_OVERTEMP
make bench                                  # pipeline benchmark, both modes, parity check
make data                                   # download the real ESA-ADB dataset (operator step)


# Service + frontend
python -m vela.api                          # http://127.0.0.1:8000 (docs at /docs)
cd apps/console && npm ci && npm run dev    # operator console, http://localhost:5173


# Real ESA-ADB detection benchmark
VELA_ESA_DATA_DIR=data python -m vela.eval detect --mission ESA-Mission1 --resample 1h --targets-only

The ten scenarios: NOMINAL, CELL_DEGRADATION, STRING_LOSS, WHEEL_FRICTION, HEATER_STUCK_ON, MEMORY_SEU_STORM, GAIN_DEGRADATION, GYRO_BIAS_DRIFT, BATTERY_OVERTEMP, SUPPRESSED_MOMENTUM_DUMP.