Architecture
This page is for people working on ctrlrelay, not just with it. For an operator-level view of the BLOCKED protocol see Feedback loop.
Layered overview
flowchart TD
CLI["CLI / daemons<br/>cli.py (Typer app)<br/>bridge/__main__.py (bridge daemon)"]
PIPE["Pipelines — pipelines/<br/>dev · secops · task · post_merge · base"]
BRIDGE["Bridge — bridge/<br/>server · protocol · factory · node_tag<br/>telegram_handler · mattermost_handler"]
CORE["Core — core/<br/>Dispatcher (claude / opencode) · WorktreeManager<br/>GitHub CLI wrapper · Checkpoint · StateDB (SQLite)<br/>Poller · PRWatcher · Config (pydantic)"]
TRANS["Transports — transports/<br/>SocketTransport (to the bridge)<br/>FileMockTransport (tests)"]
DASH["Dashboard (optional, push-only) — dashboard/<br/>DashboardClient"]
CHAT["Chat: Telegram or Mattermost"]
CLI --> PIPE
CLI --> BRIDGE
PIPE --> CORE
PIPE --> TRANS
PIPE --> DASH
TRANS -- "Unix socket" --> BRIDGE
BRIDGE --> CHAT
The CLI is a thin shell over async functions in pipelines/ and core/. The
bridge is a separate daemon that pipelines reach over a Unix socket — it is
deliberately decoupled so it can be restarted without affecting in-flight
pipeline work.
Dispatcher ↔ agent interaction
src/ctrlrelay/core/dispatcher.py
owns the shared subprocess lifecycle. Thin adapters provide CLI-specific argv
and native-session parsing: ClaudeDispatcher, OpenCodeDispatcher, and
GenericCliDispatcher for local manifests. All expose the same async
spawn_session shape, so pipelines stay agent-agnostic.
make_repo_agent_dispatcher enrolls the configured plugins and selects the
standalone default per repo: agent.type is the global default and
repos[].agent_type overrides it. select_agent additionally resolves an
allowlisted assignment override or a previously pinned identity to one concrete
adapter at the run boundary.
- Spawn shape: Claude Code runs
claude -p <prompt> --output-format stream-json --verbose --dangerously-skip-permissions [--resume <id>]; OpenCode runsopencode run ... --format json --dangerously-skip-permissions [--session <id>]. - Working directory: the session’s worktree (so file edits land on the right branch).
- Environment additions:
CTRLRELAY_SESSION_IDandCTRLRELAY_STATE_FILEare the only contract between dispatcher and child. Anything else the agent needs (issue body, PR rules, etc.) goes into the prompt. - Credentials (#219): every spawn says whether it gets GitHub access
(
spawn_session(..., github_access=...), no default). dev and task spawn without it: GitHub tokens,gh’s login, git credential helpers, SSH and the desktop keyring are cut off (core/agent_env.py), and ctrlrelay pushes the branch, opens the PR or posts the report itself afterwards (core/publish.py,WorktreeManager.push_run_branch). secops still spawns with access. See What the agent can reach. - Result:
SessionResult(session_id, exit_code, state, stdout, stderr, agent_session_id).session_idis our composite orchestrator id;agent_session_idis the agent’s own session id, parsed from its JSON stdout and the value its resume flag actually expects.stateis the parsedCheckpointStateif a state file was written, elseNone. The wrapper exposes.success,.blocked,.failedproperties for pattern-matching pipelines. - Resume:
resume_session_idtranslates to--resume <id>(Claude Code),--session <id>(OpenCode), or a generic manifest’sresume_args. Pipelines look up the agent’s own id (persisted assessions.agent_session_idby the prior spawn) and pass it through. A generic plugin without a resume mapping refuses the request instead of silently starting fresh. Claude retains its existing fresh-session fallback when no native id was captured. - Identity and capabilities (#379): every concrete adapter reports an
AgentIdentity(agent_type, plugin id/version, model, provider and an invocation fingerprint) and conservativeAgentCapabilities(resume, structured events, usage and sandbox). The selected identity is written tosessionsbefore dispatch and checked again before storing a native session id. A conflicting identity is an error, never a silent backend switch. Capabilities default to false; missing usage or structured-event support is not guessed.
The dispatcher does NOT proxy or wrap the agent session — it shells out the
same way you’d shell out to gh or git. This is deliberate.
State DB shape
Schema in
src/ctrlrelay/core/state.py.
SQLite, single file at paths.state_db.
| Table | Columns | Notes |
|---|---|---|
sessions |
id, pipeline, repo, issue_number, worktree_path, status, blocked_question, started_at, ended_at, claude_exit_code, summary, agent_session_id, agent_type, agent_plugin_id, agent_plugin_version, agent_model, agent_provider, agent_fingerprint, output_tail |
Indexed on repo and status. One row per pipeline run. agent_session_id stores the agent’s own session id for resume; the agent_* columns pin the selected plugin identity and invocation fingerprint (#379); claude_exit_code holds the worker’s exit code whichever backend ran; output_tail is a scrubbed, capped tail of its output (#204). |
repo_locks |
repo (PK), session_id, acquired_at |
Acquired with acquire_lock, released in the pipeline’s finally. |
github_cursor |
repo, last_checked_at, last_seen_issue_update |
Bounds GitHub API calls in the poller. |
pending_resumes |
session_id (PK), pipeline, repo, question, created_at, answer, answered_at, resumed_at, expired_at, expired_reason, held_until, posted, last_answered_at |
One row per BLOCKED session awaiting an answer, whichever chat it was asked on. An answer arriving after the in-session wait lands here and the poller’s sweeper drives the resume. |
pr_watches |
repo, pr_number (PK pair), session_id, issue_number, pr_url, started_at, cleanup_phase |
In-flight post-merge watchers, rehydrated on poller start. |
automation_decisions |
id, repo, operation, policy, item_id, decision, decided_by, decided_at, context |
Audit trail for ask-policy decisions. |
dispatch_health |
single row: consecutive_failures, alerted_at, alert_claim_token, alert_delivered_at, last_failure_at, last_success_at |
Whether the relay can still run a worker at all (#204). |
relay_pushes |
id, repo, branch, session_id, sha, pushed_at |
Every branch head ctrlrelay pushed for an agent (#219); the push lease is taken from here. |
events |
id, ts, kind, repo, issue_number, session_id, detail |
Feed for the status page, written best-effort. |
telegram_pending |
request_id, session_id, question, asked_at, answered_at, answer |
Legacy: still created by the schema, but no code reads or writes it. Outstanding questions live in pending_resumes. |
Methods on StateDB: acquire_lock, release_lock, get_lock_holder,
list_locks, set_agent_session_id, get_agent_session_id,
pin_agent_identity, get_agent_identity, plus raw execute / commit.
The pipeline code uses raw SQL for the sessions
table on purpose — kept thin and grep-able.
Worktree lifecycle
WorktreeManager (in
src/ctrlrelay/core/worktree.py)
is a thin wrapper over git worktree. Conventions:
- Bare repo path:
<paths.bare_repos>/<owner>-<repo>.git. Cloned on first use, fetched on each subsequent use. - Worktree path:
<paths.worktrees>/<owner>-<repo>-<session_id>. - Branch: for the dev pipeline, derived from
dev_branch_templatewith{n}→ issue number. Defaultfix/issue-{n}.
Lifecycle in run_dev_issue:
ensure_bare_repo(repo)
branch_preexisted = await branch_exists_locally(repo, branch)
worktree_path = create_worktree_with_new_branch(repo, session_id, branch)
... agent session runs ...
on DONE: remove_worktree (keep branch — open PR refs it)
on BLOCKED: keep both (operator may resume)
on FAILED: remove_worktree
if not branch_preexisted and not on remote: delete_branch
The “preexisted” guard prevents a retry from clobbering a branch that already holds an open PR from a previous successful session.
PR verifier and post-merge
After a DONE-with-pr_url checkpoint, the dev pipeline runs
PRVerifier
to confirm the PR is mergeable and CI is green. If not, it builds a fix prompt
(merge conflicts → “rebase and resolve”, failing checks → “investigate and
fix”), resumes the same agent session with that prompt, and re-verifies. Up
to DEFAULT_MAX_FIX_ATTEMPTS (3) attempts.
Then the proof step
(src/ctrlrelay/core/proof.py,
#221) runs the test the agent named, in fresh copies of the pushed commit
(must pass) and of the base with only the branch’s test changes (must fail),
and scans the diff for weakened tests, when the repo turns it on. Only a proven run’s PR is marked
ready; any other outcome leaves it a draft and fails the run. See
automation.require_proof.
PRWatcher (in
src/ctrlrelay/core/pr_watcher.py)
polls a PR for merge status; the post-merge handler in
src/ctrlrelay/pipelines/post_merge.py
closes the originating issue and notifies the operator once the PR lands.
The poller starts one pr_watch_task per PR the dev pipeline hands off, and
persists it in pr_watches so a poller restart resumes the watch.
Bridge architecture
The bridge daemon (ctrlrelay/bridge/server.py) listens on a Unix socket
(mode 0o600 — owner-only). It speaks newline-delimited JSON, defined in
bridge/protocol.py:
The bridge builds exactly one chat handler from transport.type
(bridge/factory.py): TelegramHandler or MattermostHandler. The server
knows only the handler interface, so the flow is the same for both:
sequenceDiagram
participant Cl as Client (pipeline)
participant S as Bridge server
participant H as Chat handler
participant Ch as Telegram or Mattermost
Cl->>S: ask(q)
S->>H: ask(q, options)
H->>Ch: post (sendMessage / POST /api/v4/posts)
Note over S,H: server records post id → session
Ch-->>H: reply (getUpdates long-poll / WebSocket posted event)
H-->>S: on_reply(text, reply_to_post_id)
S-->>Cl: answer(text)
The handler reports the post the reply points at — Telegram’s
reply_to_message_id, or Mattermost’s thread root_id. Matching reply →
pending question:
- The reply points at a live pending question — route to that question. This is the only route that is right when several are outstanding.
- The reply points at a question that is no longer live (its ask timed
out, or the bridge restarted) — hand it to the orphan router, which
attaches the answer to that session’s
pending_resumesrow. - A fresh message with exactly one question outstanding — route there.
- A fresh message with several outstanding — refuse, and tell the operator to reply to the question they mean. There is no FIFO fallback.
When several machines share one Mattermost bot and channel
(transport.mattermost.shared: true), each machine prefixes its posts with
[<node>] and ignores replies to other machines’ posts (bridge/node_tag.py).
Multiple in-flight asks are supported. The bridge retains the original socket for each ask so the answer routes back to the correct caller.
Bridge restart safety: pending in-memory state is lost on bridge restart.
Pipelines waiting on transport.ask stop waiting and persist the session as
BLOCKED with a pending_resumes row; a later answer still drives the
resume through the sweeper. Restart the bridge before restarting the
poller to minimise this window.
Why this shape
A few decisions worth flagging:
- Local-first, file-based protocol. Everything between ctrlrelay and the agent is a file under the worktree. No long-lived shared memory, no socket contract with the agent. Survives orchestrator restart trivially.
- Subprocess-not-library.
claudeoropencodeis invoked as a subprocess. ctrlrelay has no Python coupling to either agent’s internals — it exchanges a prompt in and a JSON file out. Versioning is decoupled. - Per-repo serial, parallel across repos. Repo locks in SQLite enforce this. A failed lock acquisition fails the session immediately rather than queueing.
- No background event loop. Each CLI command spins up its own
asyncio.run(...). The poller and bridge each own one event loop while they run. Simpler to reason about than a long-lived shared loop. - Bridge as separate process. Restartable independent of pipelines. The cost is the small in-memory pending-asks state that does not survive a bridge restart.
Where to read next
src/ctrlrelay/core/config.py— pydantic schema, the source of truth for the YAML.src/ctrlrelay/pipelines/dev.py— full dev-pipeline orchestration including BLOCKED loop and PR-fix loop.src/ctrlrelay/core/dispatcher.py— the only place we callclaudeoropencode.tests/— covers every public surface; treat tests as executable specifications.