ai-artifacts/projects/dotfiles/024-pi-subagents-lifecycle-supervision-research.md

status, claimed-by, claimed-at, completed-at, parent, blocked-by, tags

status	claimed-by	claimed-at	completed-at	parent	blocked-by	tags
done	019fba91-eabf-76ae-b086-a37ac061d6e8	2026-07-31T21:16:07-04:00	2026-07-31T21:16:34-04:00	002-pi-subagents-map	020-pi-subagents-pi-runtime-research	ticket/research

Subagent lifecycle supervision research

Question

What lifecycle, cancellation, timeout, event delivery, and long-running supervision model should the extension use, including whether Firstmate-style event-driven supervision patterns are relevant and how to avoid brittle sleep-loop orchestration?

Answer

Use an explicit supervisor inside the parent Pi extension and isolate child execution behind the ChildRunner boundary selected in 020-pi-subagents-pi-runtime-research. For version one, the supervisor should run subprocess RPC children and track them as durable child records with event-driven state transitions. Do not supervise by sleeping and polling files. Use RPC events, process lifecycle events, abort signals, and bounded timers as the primary coordination primitives.

A child lifecycle should be a small state machine: queued, starting, running, settling, completed, failed, cancelled, and timed_out. Transitions should be driven by observed events: spawn success, RPC prompt acceptance, agent_start, streaming updates, tool events, agent_settled, process close, abort request, timeout expiry, and parse/protocol errors. The parent should persist status entries in Pi with pi.appendEntry() so status survives reloads without putting full child logs into parent LLM context. Source: Pi extension docs for appendEntry, lifecycle events, session_shutdown, and agent/tool events in /nix/store/rg248h9sz8dylm8p6a9w9fj4zrv4sgm5-pi-coding-agent-0.82.1/lib/node_modules/pi-monorepo/docs/extensions.md.

Cancellation should be cooperative first and forceful second. For RPC subprocesses, send the RPC abort command when the child has reached protocol readiness, close stdin if needed, then signal the child process group with SIGTERM and escalate to SIGKILL after a short grace period. For in-process children later, call AgentSession.abort(), unsubscribe from events, dispose the session, and clean temporary files in finally. Source: Pi RPC abort docs in /nix/store/rg248h9sz8dylm8p6a9w9fj4zrv4sgm5-pi-coding-agent-0.82.1/lib/node_modules/pi-monorepo/docs/rpc.md, Pi extension abort-signal docs in /nix/store/rg248h9sz8dylm8p6a9w9fj4zrv4sgm5-pi-coding-agent-0.82.1/lib/node_modules/pi-monorepo/docs/extensions.md, and the subprocess runner design in 020-pi-subagents-pi-runtime-research.

Timeouts should be wall-clock limits owned by the supervisor, not prompts asking the model to stop. Each spawn should have optional startTimeoutMs, idleTimeoutMs, and runTimeoutMs fields. The first guards child boot and prompt acceptance, the second guards no-progress hangs, and the third guards total runtime. Any timeout should produce a structured timed_out result and then execute the same cancellation path.

Firstmate-style patterns are relevant at the design level, not as code to import. The useful pattern is event-normalized supervision: backend-specific events are converted into a backend-neutral transition model, ambiguous recovery fails closed, locks serialize identity-sensitive spawn and metadata publication, and presentation metadata is never treated as lifecycle authority. The Pi extension should apply those patterns to child-agent supervision while using Pi-native RPC events instead of Herdr panes, tmux windows, or filesystem sentinels. Source: /tmp/firstmate-research/firstmate/bin/fm-spawn.sh and /tmp/firstmate-research/firstmate/bin/backends/herdr.sh.

Findings

Pi event surfaces

Pi extension lifecycle events include session_start, resources_discover, before_agent_start, agent_start, message_start, message_update, message_end, tool_execution_start, tool_execution_update, tool_execution_end, agent_end, agent_settled, and session_shutdown. Source: /nix/store/rg248h9sz8dylm8p6a9w9fj4zrv4sgm5-pi-coding-agent-0.82.1/lib/node_modules/pi-monorepo/docs/extensions.md.

Pi docs say agent_end is not a final idle signal because Pi may still auto-retry, compact and retry, or continue with queued follow-up messages. They recommend agent_settled when status integrations need to know Pi will not continue automatically. Source: /nix/store/rg248h9sz8dylm8p6a9w9fj4zrv4sgm5-pi-coding-agent-0.82.1/lib/node_modules/pi-monorepo/docs/extensions.md#agent_start--agent_end--agent_settled.

Pi tool execution events are ordered only partly. tool_execution_start is emitted in assistant source order during preflight, tool_execution_update events may interleave across tools, and tool_execution_end is emitted in tool completion order. Source: /nix/store/rg248h9sz8dylm8p6a9w9fj4zrv4sgm5-pi-coding-agent-0.82.1/lib/node_modules/pi-monorepo/docs/extensions.md#tool_execution_start--tool_execution_update--tool_execution_end.

Pi custom tool execution receives an AbortSignal. Extension handlers can use ctx.signal for nested async work so Esc cancels abort-aware operations started by the extension. Source: /nix/store/rg248h9sz8dylm8p6a9w9fj4zrv4sgm5-pi-coding-agent-0.82.1/lib/node_modules/pi-monorepo/docs/extensions.md#ctxsignal.

Pi docs instruct extensions not to start background resources from the factory. They should defer resources until session_start or the event/tool/command that needs them and register an idempotent session_shutdown handler to clean session-scoped resources. Source: /nix/store/rg248h9sz8dylm8p6a9w9fj4zrv4sgm5-pi-coding-agent-0.82.1/lib/node_modules/pi-monorepo/docs/extensions.md#long-lived-resources-and-shutdown.

Pi RPC child control

Pi RPC mode is a JSONL protocol over stdin and stdout. It accepts prompt, steer, follow_up, abort, new_session, get_state, and other commands. Source: /nix/store/rg248h9sz8dylm8p6a9w9fj4zrv4sgm5-pi-coding-agent-0.82.1/lib/node_modules/pi-monorepo/docs/rpc.md.

The RPC abort command aborts the current agent operation and returns a success response. Source: /nix/store/rg248h9sz8dylm8p6a9w9fj4zrv4sgm5-pi-coding-agent-0.82.1/lib/node_modules/pi-monorepo/docs/rpc.md#abort.

RPC get_state exposes whether the child is streaming, compacting, retrying, or waiting for bash. Source: /nix/store/rg248h9sz8dylm8p6a9w9fj4zrv4sgm5-pi-coding-agent-0.82.1/lib/node_modules/pi-monorepo/docs/rpc.md#get_state.

RPC streams lifecycle events including agent_settled, message_start, message_update, message_end, tool_execution_start, tool_execution_update, and tool_execution_end. Source: /nix/store/rg248h9sz8dylm8p6a9w9fj4zrv4sgm5-pi-coding-agent-0.82.1/lib/node_modules/pi-monorepo/docs/rpc.md#events.

RPC tool events include toolCallId, which can correlate start, update, and end events. tool_execution_update contains accumulated partial output rather than just a delta. Source: /nix/store/rg248h9sz8dylm8p6a9w9fj4zrv4sgm5-pi-coding-agent-0.82.1/lib/node_modules/pi-monorepo/docs/rpc.md#tool_execution_start--tool_execution_update--tool_execution_end.

Prior Pi subagent runner evidence

The previously inspected mjakl/pi-subagent runner uses pi --mode rpc and sends the child prompt as a JSON prompt command. It watches stdout JSON events, cancels extension UI requests, records streaming updates, watches for agent_settled, asks get_state if the prompt appears handled without an agent start, and normalizes the final result. Source: /tmp/pi-ext-research/pi-subagent/runner.ts.

That runner uses a detached process group on Unix and taskkill /T /F on Windows. It terminates with SIGTERM, escalates to SIGKILL after 500ms on Unix, and has a separate termination settle timeout. Source: /tmp/pi-ext-research/pi-subagent/runner.ts.

That runner enforces optional run timeouts, caps oversized JSON event lines at 25 MiB, truncates stderr, handles unexpected signal exits, and cleans temporary prompt and session snapshot directories in finally. Source: /tmp/pi-ext-research/pi-subagent/runner.ts.

The same runner waits briefly after semantic settlement for transient events, but it does not use unbounded sleep-loop orchestration. It drives completion from RPC events, process close, timeout timers, and abort signals. Source: /tmp/pi-ext-research/pi-subagent/runner.ts.

Firstmate supervision patterns

Firstmate's spawn path is not a simple terminal launcher. It resolves backend, harness, model, effort, task kind, worktree, parent identity, locks, metadata, and recovery conditions before endpoint creation. Source: /tmp/firstmate-research/firstmate/bin/fm-spawn.sh.

Firstmate's Herdr adapter treats presentation workspaces as non-authoritative visual projections. It stores exact endpoint metadata for recovery and refuses ambiguous recovered launches instead of guessing. Source: /tmp/firstmate-research/firstmate/bin/backends/herdr.sh.

Firstmate's Herdr adapter normalizes pane.agent_status_changed edges through a shared transition model and falls back to polling only when the event surface is unavailable. Source: /tmp/firstmate-research/firstmate/bin/backends/herdr.sh.

The transferable Firstmate idea is to separate endpoint authority from presentation, use normalized transitions, and fail closed on ambiguous recovery. The non-transferable parts are tmux/Herdr pane IDs, treehouse worktree allocation, and shell metadata files as primary lifecycle signals.

Lifecycle model

State machine

Use these states:

• queued: accepted by the parent but not yet started.
• starting: process or in-process session is being created.
• running: child prompt was accepted or agent_start was observed.
• settling: agent_settled was observed and the runner is draining final events or waiting for process exit.
• completed: child produced a normal final result.
• failed: child failed due to spawn, protocol, model, tool, parse, or process error.
• cancelled: parent or user cancelled the child.
• timed_out: supervisor timeout fired.

Store timestamps for every state transition. Store the last observed event type and a short status string for the UI. Do not infer long-running state from lack of output alone unless an idle timeout expires.

Event handling

The subprocess RPC runner should parse stdout as strict JSONL, matching Pi RPC framing rules. It should keep stderr as diagnostic output, capped and tail-truncated. It should reject oversized JSON lines with a protocol error. It should handle malformed JSON lines as protocol errors unless Pi documents an out-of-band stdout channel for the launched mode.

Map events as follows:

• RPC prompt response success: starting to running if no agent_start arrives yet.
• agent_start: starting or running to running.
• message_update: update streaming preview and last-progress time.
• tool_execution_start: add or update a tool record and last-progress time.
• tool_execution_update: replace accumulated tool output preview and last-progress time.
• tool_execution_end: mark the tool record complete and last-progress time.
• agent_settled: running to settling.
• Process close with settled result: settling to completed or failed based on normalized result.
• Process close without settlement: failed, unless cancellation or timeout is already active.

Use agent_settled, not agent_end, as the semantic completion signal. Keep a short drain grace period after agent_settled so trailing message and tool events are processed. For persistent child sessions, wait for the child process to exit naturally up to a longer bounded flush timeout before killing it.

Cancellation model

A cancellation request should be idempotent. The first cancellation marks the child as cancellation-pending and records who initiated it. Further cancellation calls should return the same pending or final state.

For subprocess RPC children:

1. If stdin is open and the RPC protocol is alive, send {"type":"abort"}.
2. End stdin when no further command is needed.
3. Send SIGTERM to the process group on Unix or taskkill /T on Windows if the process remains alive.
4. Escalate to SIGKILL or forced kill after a short grace period.
5. Resolve as cancelled with exit code 130 when cancellation was parent-initiated and no better child result exists.

For later in-process children:

1. Call the child AgentSession.abort().
2. Unsubscribe event listeners.
3. Dispose the child session.
4. Clean temporary session snapshots in finally.
5. Resolve as cancelled unless the session had already completed.

Parent Pi session_shutdown must cancel or detach every running child according to a user-configured policy. The safer default is to cancel foreground and supervised background children on parent shutdown. A later persistent-child feature may allow detaching, but detached children must be visible in status and recoverable by session id.

Timeout model

Support three supervisor-owned timers:

• startTimeoutMs: maximum time from process spawn to RPC prompt acceptance or first meaningful child event.
• idleTimeoutMs: maximum time without any message, tool, or state progress after the child starts.
• runTimeoutMs: maximum wall-clock duration from spawn to completion.

Timeouts should never be implemented by prompting the model to stop. A timeout should set a structured error reason, run cancellation, and return a normalized timed_out result. Timers must be cleared exactly once when the child reaches a terminal state.

Persistence and recovery

Use pi.appendEntry() for durable parent-visible child status because custom entries do not participate in LLM context. Source: /nix/store/rg248h9sz8dylm8p6a9w9fj4zrv4sgm5-pi-coding-agent-0.82.1/lib/node_modules/pi-monorepo/docs/extensions.md#piappendentrycustomtype-data.

Persist enough metadata to display and recover status:

• Child id.
• Parent session file path or id.
• Agent name and source path.
• Context mode.
• Working directory.
• Runtime kind.
• Process id when applicable.
• Child session file or id when persistent.
• Start and completion timestamps.
• Current lifecycle state.
• Last event timestamp.
• Stop reason.
• Short final summary.

Do not treat UI entries as the authority for killing a process. Process handles in memory, persistent child session metadata, and explicit child ids are authority. If Pi reload loses process handles, the extension should mark unknown still-running subprocesses as orphaned unless it has a deliberate reattach protocol. Do not guess ownership from a title, status string, or terminal row.

Concurrency model

Support bounded concurrency. A global extension setting should limit concurrently running children. Each tool invocation may additionally limit fan-out. Queue excess children in queued state.

Use a central Supervisor map keyed by child id. Do not let each tool invocation own untracked subprocesses independently. This prevents orphaned children when multiple parent tool calls run concurrently.

Avoiding brittle sleep loops

Avoid unbounded loops like while child_running; sleep 1; poll status. Use event listeners and timers. Polling is acceptable only as a compatibility fallback when a backend lacks events, and it must have an interval, deadline, and state reconciliation rule.

For subprocess RPC, no normal polling is needed. The child already streams events. get_state may be used as a targeted one-shot reconciliation command when prompt acceptance succeeds but no agent_start arrives within a short grace period. Source: Pi RPC state docs and /tmp/pi-ext-research/pi-subagent/runner.ts.

Implementation requirements

• Implement Supervisor as the only owner of child records, process handles, timers, and event subscriptions.
• Implement ChildRunner as a narrow interface with start, abort, and event callbacks.
• Make all cleanup idempotent.
• Clear every timer in the terminal path.
• Remove all event listeners in finally or equivalent disposal paths.
• Cap stdout line size, stderr size, and retained preview size.
• Normalize every terminal path to one result object.
• Expose progress via Pi status UI and durable non-context entries, not parent prompt messages by default.
• Use agent_settled for semantic completion.
• Use process close for final subprocess resource cleanup.
• Treat cancellation, timeout, protocol error, spawn error, and nonzero exit as distinct stop reasons.
• Add tests for cancellation before start, cancellation while running, timeout, malformed JSON, child process spawn failure, child exit before settlement, child settlement before process exit, persistent child flush timeout, and parent shutdown.

Limitations

This ticket did not run a fresh prototype against Pi RPC. It relies on Pi documentation, the previously inspected mjakl/pi-subagent runner, and Firstmate source patterns. The implementation spec should still include a small runner test harness before coding the full extension UI.