Runs

A run is one logical execution of a workflow with a JSON input. Its run_id is stable: retries, repairs, and recoveries re-run the same run as new immutable attempts under that id — there is never a second public run to discover or follow. Runs are created from the API, MCP, web app, CLI, a schedule, or a workflow endpoint; a runner claims each queued run and executes it, streaming events, logs, and artifacts back to the Hub as the durable record.

Lifecycle

Terminal states are final: late reports racing a terminal state are absorbed idempotently rather than overwriting the first decision.

Paused runs (recoverable interruptions)

When execution is interrupted by a condition a human can fix — the classic case is the provider account ran out of credits — the run parks as paused instead of failing. Pause metadata lives on run.pause:

{
  "reason": "credits_exhausted",
  "message": "Provider returned 402: credit balance is too low",
  "pausedAt": "2026-07-09T12:00:00.000Z",
  "pausedBy": "gateway",
  "resumable": true,
  "resume": { "smithersRunId": "run-1751234567", "strategy": "smithers_resume" },
  "requiredAction": { "type": "add_credits", "label": "Add credits, then resume" }
}

• How runs pause. The metering gateway pauses a gateway-metered run when the upstream provider answers with a credit/quota-exhaustion response (e.g. HTTP 402) — a structured signal at the inference boundary. The runner pauses a run when the engine parks it as waiting-quota (a structured Smithers state since 0.27; the runner mirrors it as a pause with reason: "quota_exhausted" carrying the engine checkpoint), and still classifies engine failure text that clearly means exhausted credits/quota as a fallback. Operators can pause explicitly via POST /api/runs/{id}/pause (pause_run, runyard pause).
• What paused means. The run releases its runner slot (a parked run never occupies capacity), keeps its runner_id so resume lands on the runner holding the local engine checkpoint, and is exempt from liveness/stall/deadline reaping. run.paused / run.resumed timeline events record the transitions.
• Resume. POST /api/runs/{id}/resume (resume_run, runyard resume) re-queues the same run. With a recorded checkpoint it continues via the engine's resume (smithers up --resume); without one it re-runs from scratch and the response (resume.strategy: "rerun_from_scratch") plus the run timeline say so explicitly. Cancel still works from paused.
• Resume strategies. The resume body optionally forces a strategy: {"strategy": "rerun_from_scratch"} discards the recorded checkpoint and clears the runner pin so any live runner can take the run (runyard resume --from-scratch, the web "Restart from scratch" button); {"strategy": "smithers_resume"} insists on the checkpoint and answers 409 when none is recorded. Omitted, the strategy resolves automatically.
• Offline checkpoint runner. A checkpointed resume only executes once the runner holding the checkpoint claims it. If that runner is offline, the resume still succeeds but the response carries a warning (and the run.resumed event records it): the run stays queued until the runner reconnects, or you resume again with rerun_from_scratch to run anywhere.
• Resume failure. If the runner cannot find the recorded checkpoint in its local .smithers state (state cleaned, workspace replaced), it does not hang or fake a timeout: it emits runner.resume_checkpoint_missing and re-parks the run as paused with reason: "resume_failed" and the stale checkpoint dropped, so the next resume honestly re-runs from scratch. (The engine validates --resume inside its detached child, so the Hub runner verifies the checkpoint with smithers inspect before launching.)
• Not a budget stop. budget_exceeded remains a distinct terminal status for the Hub's own hard spend ceiling; a budget breach is never converted into a pause.

Failure classes

Failures are classified into specific terminal statuses so operators can tell config problems from real failures: failed (generic), blocked_by_gate (a verification gate failed), blocked_by_preflight (the run could never start as configured), provider_limited (rate limits/quota), timed_out, invalid_output (schema mismatch), infra_unavailable (network/runner infrastructure), needs_human, and budget_exceeded (the run's metered usage reached its spend budget and it was hard-stopped).

Usage and budgets

Runs also meter the model calls they consume: per-call usage records stream as run.usage events and aggregate onto the run as usage (totalTokens, costMicros, byModel, ...), and run creation accepts an optional budget: { maxTokens?, maxCostMicros? } hard ceiling. Budgeted runs additionally carry a computed budgetStatus (spent vs limit, remaining, percent used, nearLimit at 80%) in list and detail payloads. See Usage metering & budgets for the data model, the fleet rollup (GET /api/usage/summary), the metering gateway, and budget enforcement.

The needs-attention queue

GET /api/runs/attention (list_attention_runs, runyard attention) returns every run whose next step is a human action, grouped with counts:

• paused — resume them once the interruption is fixed;
• waitingApproval — a decision is pending on an approval card;
• budgetStopped — hit their spend ceiling in the last 7 days; raise the budget and re-run to finish the work.

counts also includes pendingApprovals (open decision cards). The web Home view renders this as a triage strip above the runs list with resume/review/inspect actions inline; it disappears entirely when nothing needs a human. GET /api/runs?status=paused (or any other status) remains the raw filter underneath.

Preflight and run drafts (negotiation)

Before anything is enqueued, Runyard can run a deterministic preflight: schema checks on required input fields, runner/tag availability, secret presence, hook eligibility, and workflow source resolution. The result is ready, needs_input (with questions[] the caller can answer), or blocked (with blockers[] only an operator can fix), plus warnings, suggested defaults, and a nextAction hint.

Three ways to use it:

• POST /api/workflows/{id}/preflight — stateless report, nothing created.
• POST /api/workflows/{id}/run with negotiate: true — enqueue only when ready; otherwise the negotiation state and a saved draft come back instead of a doomed run.
• Run drafts — a draft is a proposed run that has not been enqueued. Create it, patch its input to answer questions (each patch re-preflights), then submit; submit re-preflights and returns 202 with the run only when ready (422 needs_input / 409 blocked otherwise).

curl -X POST -H "Authorization: Bearer $RUNYARD_TOKEN" -H "Content-Type: application/json" \
  -d '{"input": {"title": "Nightly digest", "topic": "release notes"}, "executionMode": "remote"}' \
  https://hub.example.com/api/workflows/idea-to-product/run

input.title is recommended for agent-created runs: a short human-readable title used in run lists, approval cards, and handoff.

Events, timeline, logs, artifacts

Runners append structured events as the run progresses. Every event carries a monotonic per-run cursor (seq); GET /api/runs/{id}/events/stream serves them live over SSE — replaying persisted history after ?afterSeq= (or a standard Last-Event-ID reconnect header), tagging frames with id:<seq>, sending keepalive comments, and closing with a final run-terminal frame once a terminal run is fully drained. Run payloads expose the transport as eventsStreamUrl, and runyard run --follow / runyard logs --follow are the CLI front ends. GET /api/runs/{id}/timeline merges status transitions, events, and artifacts into one ascending feed (with since/limit paging). Log lines are at /logs, a condensed log summary and diagnostics at /diagnostics, and uploaded artifacts (outputs, traces, files) at /artifacts, downloadable by artifact id.

Stable run identity and attempts

A replacement/recovery is the same logical run, run again. Every execution try is an immutable attempt (att_…) under the stable run_id:

• Normal clients never change anything. Polling GET /api/runs/{id}, the SSE event stream (whose seq cursor stays monotonic across attempts), cancel, approvals, artifacts, webhooks, and work-item links all target the stable run id straight through a recovery. attemptId/attemptCount on the run payload can be ignored entirely.
• Retry — POST /api/runs/{id}/retry (retry_run, runyard retry) starts the next attempt on the same run. It answers 409 while an attempt is still active unless force: true, which supersedes the live attempt first (its pending approval cards are retired as superseded, its workspace gets an ownership-checked cleanup intent, and its runner acknowledges cancellation before the new attempt is claimable — the recovering status above). Recovery classification fields (failureClass, workspaceHealthy, …) select the strategy: gate_retry, same_workspace_repair, continuation (reuses the engine checkpoint and runner pin), or clean_room.
• Attempt history — GET /api/runs/{id}/attempts (get_run_attempts, runyard attempts) lists every try with status, recovery reason, runner, timestamps, archived input, failure evidence, and cleanup state; GET /api/runs/{id}/events?attemptId= filters one attempt's events. Prior attempts are auditable history and are never overwritten. The web run page shows the same under Attempts.
• The lifecycle reads coherently — e.g. running → recovering → running → succeeded; the terminal outcome is the final attempt's outcome.
• Legacy ids keep resolving. Databases from v0.23.1 and earlier modeled each retry as a separate successor run; the migration collapses those chains into attempts under the original run, and the retired successor ids resolve to the same logical run forever (responses note resolvedFrom).

Reruns and promotion

• Rerun — POST /api/runs/{id}/rerun with the same (or omitted) input retries the SAME stable run as a new attempt (retried: true, run.id unchanged) — an active attempt is superseded like v0.23.1 replaced its predecessor, and double-submits dedupe to the pending retry. An edited input is different work: it creates a new linked run with input.rerunOf provenance.
• Promotion — for runs that mutated an isolated git worktree, POST /api/runs/{id}/promote merges the successful run's work into its target branch, runs gates, pushes, and cleans up the branch/worktree. Merging to the default branch always goes through promotion, never through a hook.

Response endpoints

A run request may include responseEndpoint: {type: "http" | "telegram", config} so the terminal-state reply is delivered to the caller when the run finishes. Delivery state is reported in responseEndpoints[] on GET /api/runs/{id}; polling that endpoint remains the canonical fallback.

API & MCP

The start/complete/fail/event-append and artifact-upload endpoints under /api/runs/{id} are the runner protocol (runner-scoped tokens, run ownership enforced) — see Runners.