chore(vendor): import pi 1.3.0

Imported as a linear vendor commit so forge rebases do not replay the upstream root tree.

git-subtree-dir: third_party/pi

git-subtree-split: 71023aec1546d6d677a65669c5665c0927ce0dd0
This commit is contained in:
2026-10-15 17:58:48 -04:00
parent 719370c8f7
commit 6081e3e325
1743 changed files with 205460 additions and 131705 deletions

View File

@@ -0,0 +1,94 @@
# Changelog
## [1.3.0] - 2026-10-15
### Breaking Changes
- Worker-backed results and `close()` now settle after requesting shutdown, not after worker exit. Hosts that require actual exit must track `onWorkerStarted(exited)` or retain capacity through `acquireWorker()`'s release callback
### Added
- Added `CodemodeSandbox`'s `onWorkerStarted(exited)` callback so hosts can count a worker against concurrency limits until it actually exits, independently of its script result
- Added `CodemodeSandbox`'s `acquireWorker()` hook to reserve shared capacity before starting a worker and release it only on actual exit or constructor failure
### Fixed
- Fixed aborts, timeouts, and `close()` waiting for worker termination on Bun while a non-interruptible built-in runs; shutdown now requests termination without waiting for exit ([#10748](https://github.com/earendil-works/pi/issues/10748))
## [1.2.0] - 2026-10-12
### Breaking Changes
- Removed `MCP_TYPESCRIPT_PREAMBLE` and `mcpStructuredContentSchema()`. `renderToolOutputType()` and the declarations built on it render every output schema as its own type and no longer turn MCP `CallToolResult` schemas into `CallToolResult<T>`; describe MCP results in the output schema instead
### Added
- Added `RemoteCodemodeSandbox`, which runs each script's VM in another isolate while tools run in the caller. `CodemodeSandboxDurableObject` from the new `@earendil-works/pi-codemode/cloudflare` entry is its remote end, deployed as its own Cloudflare Worker. `examples/cloudflare` shows the setup
- Added `InlineCodemodeSandbox`, which runs the VM on the calling thread for runtimes without worker threads. Its `interruptBudget` limits scripts that compute without awaiting a tool
- Added the `@earendil-works/pi-codemode/portable` entry, which has no Node imports, and `CodemodeSandboxBase`, the API all three sandboxes share
- Added `sandbox.start()`, a deterministic run whose calls the host answers itself (`onCall`, `run.deliver()`), so a host can replay a script after a restart: a virtual clock from `startedAt`, randomness from `seed`, UTC, no `WeakRef`/`FinalizationRegistry`, results applied in the order the host numbers them, the VM's fingerprint (`onStarted`), the calls issued after each result (`onIdle`), and whether the run has its outcome (`run.finished`). Works with every sandbox; `execute()` is unchanged
### Changed
- `schemaToType()` and the declarations built on it show property constraints the TypeScript type cannot express as tags on the property's comment line, such as `// @integer @minimum 0 @maximum 30000 @default 0`. Covered keywords: `integer`, numeric bounds, `multipleOf`, string length, `pattern`, `format`, array length, `uniqueItems`, non-null `default`, and `deprecated`; constraints of nullable `anyOf`/`oneOf` fields are read from the non-null branch, and draft-04 boolean `exclusiveMinimum`/`exclusiveMaximum` are shown as exclusive bounds. Property descriptions now also split on U+2028/U+2029. A type over `maxChars` is rendered again without tags before it becomes `unknown` ([#10707](https://github.com/earendil-works/pi/issues/10707))
- `image()` errors describe the accepted image block as `{ type: "image", data, mimeType }` instead of an MCP image block
- The VM now polls for interrupts while it drains promise jobs, so a script such as `while (true) await null` is stopped at the same points as a synchronous loop
### Fixed
- Fixed `parseCodemodeSource()` ignoring a `// @options:` line that is not the first line, so its `timeout_ms` never applied; it now throws `CodemodeSourceError` ([#10631](https://github.com/earendil-works/pi/issues/10631))
- Fixed `CodemodeSandbox` failing with a broken bridge error when the host runs under `node --watch` on Node 24 or 26, which posts `watch:import`/`watch:require` messages on the worker channel ([#10725](https://github.com/earendil-works/pi/issues/10725))
## [1.1.0] - 2026-10-07
### Added
- Added `console: true` to text output items produced by `console.*`, so hosts can tell them apart from `text()` output
## [1.0.4] - 2026-10-05
### Changed
- Built-ins are frozen and built-in globals are read-only before the script runs, so patches such as `Array.prototype.toJSON = ...` have no effect. Instances can still override `constructor`, `name`, `message`, `toString`, `toLocaleString`, `valueOf`, `toJSON`, and `Object.prototype` members ([#10444](https://github.com/earendil-works/pi/issues/10444))
### Fixed
- Fixed a script that patched built-ins crashing the host process and leaving `execute()` unsettled. Malformed payloads from the worker now fail the execution with a `sandbox` error ([#10444](https://github.com/earendil-works/pi/issues/10444))
## [1.0.3] - 2026-10-05
## [1.0.2] - 2026-10-04
## [1.0.1] - 2026-10-03
### Fixed
- Fixed scripts that print in a loop growing the host's memory until it crashes: output is limited to `MAX_OUTPUT_CHARS` (16 Mi) characters and `MAX_OUTPUT_ITEMS` (100000) items, and a script past either limit fails with a `RangeError` ([#10283](https://github.com/earendil-works/pi/issues/10283))
## [1.0.0] - 2026-10-01
### Added
- Added `renderToolOutputType()`, the type a tool call resolves to.
### Changed
- Reading a member of `tools` or of a global namespace that does not exist now throws an error naming the close matches, instead of returning `undefined`. Use `"name" in tools` to check for a tool. `store()` size errors explain what the store is for.
## [0.99.2] - 2026-09-30
### Changed
- Allowed `CodemodeSandbox.workerUrl` to be a string, as required for embedded worker entrypoints in Bun compiled executables ([#10204](https://github.com/earendil-works/pi/issues/10204)).
### Fixed
- Fixed `image()` accepting malformed base64 data and unsupported image types. It now throws a `TypeError` unless the data is valid base64 of a PNG, JPEG, GIF, or WebP image, derives the MIME type from the image signature instead of the declared type, and strips line breaks from wrapped base64 ([#10215](https://github.com/earendil-works/pi/issues/10215)).
## [0.99.1] - 2026-09-29
## [0.99.0] - 2026-09-29
### Added
- Initial spike: `CodemodeSandbox` runs model-written JavaScript in a worker thread and exposes injected tools as `tools.<name>(args)` async functions.

View File

@@ -0,0 +1,374 @@
# @earendil-works/pi-codemode
Runs model-written JavaScript in a QuickJS VM (compiled to WebAssembly) where the only capability is calling injected tools. Nested tool calls never enter the LLM context; only the script's output and return value do.
Scripts use `tools`, `ALL_TOOLS`, `text`, `image`, `exit`, `store`, and `load`, and may start with a `// @options:` line. The coding agent uses it for its built-in `codemode` tool. It has no pi dependencies and can be used on its own to expose any functions (remote APIs, MCP servers, application services) to model-written scripts.
The package runs in two kinds of environments. Each has its own sandbox class, but they share one API, one script environment, and one result format:
| Environment | Sandbox | Import from | Where scripts run |
| --- | --- | --- | --- |
| Node and Bun | `CodemodeSandbox` | `@earendil-works/pi-codemode` | a worker thread per execution |
| Cloudflare Workers and Durable Objects | `RemoteCodemodeSandbox` | `@earendil-works/pi-codemode/portable` | a Durable Object in a separately deployed Worker |
See [Usage on Node and Bun](#usage-on-node-and-bun) and [Usage on Cloudflare Workers](#usage-on-cloudflare-workers). A third sandbox, `InlineCodemodeSandbox`, runs scripts on the calling thread for other runtimes, or on Cloudflare when blocking the caller is acceptable.
## Usage on Node and Bun
```ts
import { CodemodeSandbox } from "@earendil-works/pi-codemode";
const sandbox = new CodemodeSandbox({
timeoutMs: 60_000,
tools: [
{
name: "read",
execute: async (args, { signal }) => {
const { path } = args as { path: string };
return await readFile(path, "utf8");
},
},
],
});
const result = await sandbox.execute(`
const source = await tools.read({ path: "package.json" });
text("bytes " + source.length);
return JSON.parse(source).name;
`);
console.log(result.output); // [{ type: "text", text: "bytes 1234" }]
if (result.ok) console.log(result.value); // "@earendil-works/pi-codemode"
else console.error(result.error.kind, result.error.message);
await sandbox.close();
```
`code` is the body of an async function: `return` and top-level `await` work. Inside the script, with every sandbox:
- `tools.<name>(args)` returns a promise. Arguments and results make a JSON round trip. A tool that throws rejects with an `Error` carrying the same message. Tool names are also exposed as identifiers: characters that are not valid in identifiers become `_` (`toCodemodeIdentifier`), so `my-tool` is `tools.my_tool` as well as `tools["my-tool"]`.
- `ALL_TOOLS` lists `{ name, description }` for every tool, with `name` as the identifier.
- `text(value)` appends a text item to `result.output`; values other than strings are JSON-stringified. `console.log/info/warn/error/debug` append text items with `console: true`.
- `image(urlOrItem)` appends an image item. It accepts a base64 `data:` URL, `{ image_url }`, or an image block (`{ type: "image", data, mimeType }`, as MCP tools and `@earendil-works/pi-ai` produce). Remote URLs are rejected.
- `exit()` ends the script successfully right away, keeping its output and store writes.
- `globals` passed to the sandbox are called as top-level functions, for example a host helper `image(ref)`. They behave like tools but are not recorded in `result.calls`. A name like `models.classify` puts the function on a frozen `models` object. With `spread: true`, `execute` receives all call arguments as an array instead of the first one, and `signature` replaces the declaration generated from the schemas.
- `store(key, value)` and `load(key)` read and write JSON values synchronously. See [Store](#store).
- Nothing else: no timers, `fetch`, `process`, `require`, modules, or `WebAssembly`. `eval` and `Function` work but only produce more code inside the same VM.
`timeoutMs: Infinity` disables the deadline; the script then runs until it settles or `signal` aborts it. A script that waits on a promise nothing can settle (no tool call pending, and the VM has no timers or I/O) fails right away instead of hanging.
`memoryLimitBytes` caps the VM's heap. Allocations beyond it fail inside the script as `InternalError: out of memory`.
Aborts, timeouts, and `close()` request worker termination without waiting for the worker to exit. On Bun, a built-in such as a long `String.prototype.indexOf` may not poll QuickJS's interrupt handler, so its worker can keep computing until the built-in returns. Stopping unrefs the worker after requesting termination, because `terminate()` refs it internally. This does not guarantee computation has stopped. On Bun 1.4.0, a pending `terminate()` can still delay host process exit despite `unref()`; cancellation results and `close()` do not wait for it.
Hosts that limit concurrency can track actual worker lifetime with `onWorkerStarted(exited)`, called for each worker started by `execute()` or `start()`. Its promise resolves when that worker exits, not when the script result settles:
```ts
const workers = new Set<Promise<void>>();
const sandbox = new CodemodeSandbox({
onWorkerStarted(exited) {
workers.add(exited);
void exited.then(() => workers.delete(exited));
},
});
```
Keep a concurrency slot occupied until `exited` resolves. `close()` does not wait for these promises.
To enforce an admission limit, pass `acquireWorker()`. It runs synchronously after WASM loading, immediately before each worker is created. Throw to reject the execution without starting a worker; otherwise return a release callback. The sandbox calls it once on actual worker exit or if the worker constructor fails. Share reservation state across sandbox instances, and do not release it when a script result settles.
## Store
`store`/`load` let scripts keep values across executions. The sandbox does not persist anything itself: pass the current values as `options.store`, and a successful result reports what the script changed as `result.storeWrites` (`{ set, delete }`). Failed executions report no writes.
```ts
const result = await sandbox.execute(`store("runs", (load("runs") ?? 0) + 1)`, { store: saved });
if (result.ok) {
for (const key of result.storeWrites.delete) delete saved[key];
Object.assign(saved, result.storeWrites.set);
}
```
`load` returns a copy, so mutating it does not change the store. Storing `undefined` deletes the key. A value may be at most `MAX_STORE_VALUE_CHARS` (256 Ki) characters of JSON and all values together at most `MAX_STORE_TOTAL_CHARS` (1 Mi); larger writes throw a `RangeError` inside the script.
## Source format
`parseCodemodeSource()` accepts a script whose first line may be an options line:
```js
// @options: {"max_output_tokens": 2000, "timeout_ms": 30000}
const source = await tools.read({ path: "package.json" });
text(JSON.parse(source).name);
```
Supported fields are `max_output_tokens`, a token budget for the output, and `timeout_ms`, a hard deadline. The sandbox does not act on them; the caller decides. The options line is replaced by an empty line, so line numbers in stack traces still match the input. Empty input, invalid JSON, unknown fields, an options line without code, or an options line on any line but the first throw `CodemodeSourceError`. `CODEMODE_SOURCE_GRAMMAR` is a Lark grammar for providers that support grammar-constrained tool input. Both are also available from the lightweight `@earendil-works/pi-codemode/source` entry.
## Bundled Node and Bun hosts
By default `CodemodeSandbox` loads `quickjs-wasi/quickjs.wasm` from the installed package and starts the worker file that sits next to this package's module. Neither exists on disk when the host is bundled, so pass both:
```ts
import { CodemodeSandbox, loadQuickJSWasm } from "@earendil-works/pi-codemode";
const sandbox = new CodemodeSandbox({
tools,
// Compiled once per path and cached.
wasm: loadQuickJSWasm(pathToQuickJSWasm),
// A file of your build containing `import "@earendil-works/pi-codemode/worker";`
workerUrl: new URL("./codemode-worker.js", import.meta.url),
});
```
`workerUrl` accepts a URL or string. For a Bun compiled executable, include the worker as an
additional build entrypoint and pass its relative source path as a string, for example
`"./src/codemode-worker.ts"`; Bun resolves that form from its embedded module graph.
## Usage on Cloudflare Workers
`CodemodeSandbox` needs Node or Bun. It runs each script in a worker thread and reads the QuickJS binary from disk. Cloudflare Workers have neither, and they cannot compile WebAssembly at runtime. Two other sandboxes have the same API, the same script environment, and the same `CodemodeResult`. Only where the VM runs differs:
| Sandbox | Where the VM runs | Use it for |
| --- | --- | --- |
| `CodemodeSandbox` | a worker thread | Node and Bun |
| `RemoteCodemodeSandbox` | another isolate, usually a Durable Object in a separate Worker | a Durable Object that runs an agent session |
| `InlineCodemodeSandbox` | the calling thread | anywhere else with WebAssembly, when blocking the caller is acceptable |
Import the last two from `@earendil-works/pi-codemode/portable`, which has no Node imports. On Cloudflare, `@earendil-works/pi-codemode/cloudflare` adds the deployable Durable Object and the QuickJS module compiled by Wrangler.
### Why a separate Worker
QuickJS runs synchronously, so a script that computes holds its thread until it awaits a tool. Measured on Cloudflare (October 2026), a script that computed for ten seconds delayed every other request to the session object by the full ten seconds in two placements:
- inside the session's Durable Object
- in a Dynamic Worker loaded by that object
In a Durable Object of a separately deployed Worker, the delay was about 50 ms. `RemoteCodemodeSandbox` therefore keeps the tools and call records in the session and sends only the VM to the sandbox Worker. Each tool call travels back to the session over Workers RPC.
### Deploying the sandbox Worker
A complete setup is in [`examples/cloudflare`](https://github.com/earendil-works/pi/tree/main/packages/codemode/examples/cloudflare). The sandbox Worker is one file:
```ts
// sandbox/src/index.ts
export { CodemodeSandboxDurableObject as CodemodeSandbox } from "@earendil-works/pi-codemode/cloudflare";
export default {
fetch: () => new Response("Not found", { status: 404 }),
};
```
```jsonc
// sandbox/wrangler.jsonc
{
"name": "pi-codemode-sandbox",
"main": "src/index.ts",
"compatibility_date": "2026-04-22",
"workers_dev": false,
"preview_urls": false,
"migrations": [{ "tag": "v1", "new_sqlite_classes": ["CodemodeSandbox"] }],
"limits": { "cpu_ms": 60000 }
}
```
Install `@earendil-works/pi-codemode` in the sandbox Worker's project and run `npx wrangler deploy`. Wrangler bundles QuickJS (about 640 KB) as a compiled WebAssembly module. The object has no routes and keeps no state, and it never uses its storage.
### Using it from the agent's Worker
Bind the class with `script_name`. Binding a class exported from the agent's own Worker would bring back the blocking described above.
```jsonc
// agent/wrangler.jsonc
"durable_objects": {
"bindings": [
{ "name": "CODEMODE_SANDBOX", "class_name": "CodemodeSandbox", "script_name": "pi-codemode-sandbox" }
]
}
```
Then replace the sandbox. Tools, options other than the worker-specific `wasm`, `workerUrl`, `acquireWorker`, and `onWorkerStarted`, and results stay the same:
```ts
import type { CodemodeSandboxDurableObject } from "@earendil-works/pi-codemode/cloudflare";
import { RemoteCodemodeSandbox } from "@earendil-works/pi-codemode/portable";
interface Env {
CODEMODE_SANDBOX: DurableObjectNamespace<CodemodeSandboxDurableObject>;
}
// Before: new CodemodeSandbox({ tools, timeoutMs: 60_000 })
const sandbox = new RemoteCodemodeSandbox({
tools,
timeoutMs: 60_000,
remote: () => env.CODEMODE_SANDBOX.getByName(sessionId),
});
const result = await sandbox.execute(code, { signal, store });
```
Import the sandbox from `/portable` in the agent's Worker. Importing `/cloudflare` would also bundle QuickJS there. `remote` is called once per execution and picks the Durable Object that runs it.
- **One object per session**, `getByName(sessionId)`, keeps a session's scripts together. Two scripts of the same session take turns on one thread.
- **One object per execution**, `getByName(crypto.randomUUID())`, runs every script on its own.
Run both Workers locally with `wrangler dev -c agent/wrangler.jsonc -c sandbox/wrangler.jsonc`.
### Limits on Cloudflare
- **Interrupt budget.** While a script computes without calling a tool, no timer can run, and `Date.now()` does not advance. A script that never yields is limited only by the interrupt budget, not by `timeoutMs`. QuickJS polls an interrupt handler while it runs, and the sandbox counts the polls. Past the budget, the execution fails with `kind: "timeout"`.
- The default `DEFAULT_INTERRUPT_BUDGET` of 100,000 polls is about 2 seconds of a tight loop on an Apple M-series laptop and about 10 seconds on Cloudflare.
- Keep the budget below the Worker's `cpu_ms` limit (30 seconds by default). Hitting that limit resets the object instead of failing one script.
- **Memory.** Each VM may use up to 32 MiB by default. All VMs in one isolate share its 128 MB.
- **Changing the limits.** The Durable Object caps every request at its own limits, whatever the client asks for. To change them, subclass it in the sandbox Worker:
```ts
import { CodemodeSandboxDurableObject } from "@earendil-works/pi-codemode/cloudflare";
export class CodemodeSandbox extends CodemodeSandboxDurableObject {
protected maxInterruptBudget = 200_000;
protected maxMemoryLimitBytes = 64 * 1024 * 1024;
}
```
- **Aborts and timeouts.** `timeoutMs` and `signal` end the execution in the caller right away, with `kind: "timeout"` or `kind: "aborted"`. The remote VM stops at its next tool call. A remote VM that is computing at that moment keeps running until it calls a tool or uses up its budget.
- **Billing.** Each execution is one Durable Object request. The sandbox object is billed for wall-clock duration while an execution runs, including time spent waiting on tools.
### On the calling thread
When blocking the caller is acceptable, for example in a stateless Worker that serves one request, run the VM on the caller's thread instead. Nothing extra needs to be deployed:
```ts
import { createInlineCodemodeSandbox } from "@earendil-works/pi-codemode/cloudflare";
const sandbox = createInlineCodemodeSandbox({ tools, timeoutMs: 60_000, interruptBudget: 50_000 });
const result = await sandbox.execute(code);
```
`createInlineCodemodeSandbox()` passes the QuickJS module Wrangler compiled to `InlineCodemodeSandbox`. With another bundler, import `InlineCodemodeSandbox` from `/portable` and pass your own compiled module as `wasm`.
- **Blocking.** A script that computes blocks the caller until it awaits a tool or uses up its `interruptBudget`.
- **Stack.** QuickJS's stack guard is reduced to `INLINE_MAX_STACK_SIZE`, so that it trips before the caller's smaller stack runs out.
## Declarations for the model
Tools and globals can carry `description`, `inputSchema`, and `outputSchema` (JSON Schema). `renderDeclarations()` turns them into TypeScript declarations for a model-facing tool description:
```ts
renderDeclarations({ tools: sandbox.tools, globals: sandbox.globals });
// declare const tools: {
// /** Read a file */
// read(args: {
// path: string;
// }): Promise<string>;
// };
```
Property descriptions become `//` comments. Constraints the type cannot express follow as tags, such as `// @integer @minimum 1 @default 20`; they are dropped first when a type exceeds the size limit. Schemas only shape the declarations; values are not validated against them. Local references (`#/$defs/...`, `#/definitions/...`) are expanded; recursive and remote references render as `unknown`.
## Using with pi-agent-core
To give an `Agent` a codemode tool, expose its other tools to the sandbox and wrap `execute()` as an `AgentTool`:
```ts
import type { AgentTool } from "@earendil-works/pi-agent-core";
import {
type CodemodeJsonSchema,
CodemodeSandbox,
type CodemodeTool,
renderDeclarations,
} from "@earendil-works/pi-codemode";
import { Type } from "typebox";
const sandboxTools: CodemodeTool[] = agentTools.map((tool) => ({
name: tool.name,
description: tool.description,
inputSchema: tool.parameters as CodemodeJsonSchema,
outputSchema: (tool.outputSchema as CodemodeJsonSchema | undefined) ?? { type: "string" },
execute: async (args, { signal }) => {
const result = await tool.execute("nested", args as never, signal);
if (tool.outputSchema && result.structuredContent !== undefined) return result.structuredContent;
return result.content.map((block) => (block.type === "text" ? block.text : "")).join("\n");
},
}));
const codemodeTool: AgentTool = {
name: "codemode",
label: "Codemode",
description: `Run JavaScript that calls tools as \`await tools.<name>(args)\`. Output with text() or return.\n\n${renderDeclarations({ tools: sandboxTools })}`,
parameters: Type.Object({ code: Type.String() }),
execute: async (_toolCallId, { code }, signal) => {
const sandbox = new CodemodeSandbox({ tools: sandboxTools });
try {
const result = await sandbox.execute(code, { signal });
const content = [...result.output];
if (result.ok && result.value !== undefined) content.push({ type: "text", text: JSON.stringify(result.value) });
if (!result.ok) content.push({ type: "text", text: result.error.stack ?? result.error.message });
return { content, details: undefined, isError: !result.ok };
} finally {
await sandbox.close();
}
},
};
```
`result.output` items already have the shape of `@earendil-works/pi-ai`'s `TextContent` and `ImageContent`. Calling `tool.execute()` directly skips the agent's `beforeToolCall` and `afterToolCall` hooks. To apply them to nested calls too, run each call through `runToolCall()` from `@earendil-works/pi-agent-core`, as the [mcp-codemode example](https://github.com/earendil-works/pi/tree/main/packages/agent/examples/mcp-codemode) does. That example also rejects failed nested calls inside the script and combines codemode with MCP tools.
## Deterministic runs
`sandbox.start(code, options)` runs a script so that a host can run it again after a restart and get the same behavior, for example to resume it from a durable log of its calls' results. `execute()` is unaffected.
The VM only runs when the host hands it something, and has no timers or I/O, so it is deterministic except for what it reads from its environment. `start()` fixes that:
- **Time:** `Date.now()` reads `startedAt`, then the `now` of the result being processed, never going backwards. `performance.now()` counts from `startedAt`.
- **Randomness:** `Math.random()` and the VM's other randomness come from `seed`.
- **Timezone:** UTC, whatever the host's.
- **Garbage collection:** `WeakRef` and `FinalizationRegistry` do not exist.
- **Result order:** the host answers each call with a result number (1, 2, ...), and the VM applies results strictly in that order, whatever order a transport delivers them in.
The run declares its own tools and globals, and the host answers every call itself:
```ts
const run = sandbox.start(code, {
startedAt: Date.now(),
seed: 7,
tools: [{ name: "read", description: "Read a file" }],
onStarted: (fingerprint) => {}, // what the VM is: engine versions, prelude hash, limits
onCall: (call) => {
// call.id is the VM's own number for the call, in the order the script made them.
void readFile(call.args).then((text) => run.deliver(nextResult++, Date.now(), call.id, { ok: true, value: text }));
},
onIdle: (seq, issued) => {}, // after result `seq` (0: the first run), `issued` calls in total
});
const result = await run.result;
```
A run has no deadline unless `timeoutMs` is given. Result numbers must run 1, 2, ... without gaps. The same script, `startedAt`, `seed`, tools (with their `jsName`), globals and store, answered with the same results in the same order, makes the same calls with the same arguments and produces the same output on any VM with the same fingerprint. To end a run with an error the script cannot catch, abort its `signal`.
## Results
`execute()` never rejects for script failures. `result.error.kind` is one of:
| kind | meaning |
| --------- | --------------------------------------------------------------------------- |
| `script` | the script threw or failed to parse; `stack` points at `codemode.js:<line>` |
| `timeout` | the deadline expired or the interrupt budget ran out; VM shutdown was requested |
| `aborted` | `options.signal` fired or `close()` was called; VM shutdown was requested |
| `sandbox` | the VM or its transport failed, for example a wasm trap, a missing worker file, or a failed remote |
`result.output` holds the text and image items in the order the script produced them, also for failed executions. The host keeps all of it until the script ends, so output is limited to `MAX_OUTPUT_CHARS` (16 Mi) characters of text and base64 image data and `MAX_OUTPUT_ITEMS` (100000) items. Past either limit the script fails with a `RangeError`, even if it catches the error. `result.calls` lists every tool call with `status: "ok" | "error" | "cancelled"`. A call that is still running when the script returns (not awaited) is aborted through the tool's `signal` and reported as `cancelled`.
## How it works
This section describes `CodemodeSandbox`. The other sandboxes share the same host side, prelude, and VM, and replace only the worker thread. See [Usage on Cloudflare Workers](#usage-on-cloudflare-workers).
Each `execute()` starts a worker thread (about 20 ms including VM creation) that instantiates a fresh QuickJS VM from the compiled wasm module. The VM is a separate wasm instance with its own linear memory. Its only imports are a WASI shim (clock, random, and stdout/stderr writes, which the worker discards) and one host-call entry point, so the script cannot reach the host except through the functions the worker registers.
The worker evaluates a prelude inside the VM that holds the single host bridge in a closure and builds `tools`, `console`, and globals on top of it. The script is compiled as an async function body.
Tool calls are relayed to the host thread as messages; the host runs the tool and posts the JSON result back. The host owns the deadline and the abort signal. When either fires, it sets a shared interrupt flag that the VM polls, then calls `worker.terminate()`. The flag is needed on Bun, where `terminate()` cannot stop a thread that is spinning in wasm (`while (true) {}` or `while (true) await null`).
The worker keeps script execution off the host thread: QuickJS runs synchronously, so a spinning script on the host thread would block its event loop.
## Runtime notes
- Node and Bun support memory limits, interrupts, and a catchable `RangeError` for deep recursion (QuickJS's stack guard is enabled; without it the wasm stack overflows and traps). Bun may delay worker exit during non-interruptible built-ins; results do not wait for exit.
- Stack traces use QuickJS frames (`at f (codemode.js:2:31)`), prefixed with `Name: message` like V8. The wrapper prefix shares line 1 with the script, so line numbers are exact; column numbers on line 1 are shifted.
- QuickJS is an interpreter. Glue code and filtering tool results are fast enough; heavy computation is slower than in V8.

View File

@@ -0,0 +1,31 @@
# pi-codemode on Cloudflare
This example has two Workers:
- **`sandbox/`** deploys `CodemodeSandboxDurableObject` as its own Worker, `pi-codemode-sandbox`.
Scripts run there, in a separate isolate.
- **`agent/`** has a `Session` Durable Object that owns the tools. It runs scripts through
`RemoteCodemodeSandbox`, which keeps the tools and results in `Session` while the sandbox Worker
runs the VM.
The [README of the package](../../README.md#usage-on-cloudflare-workers) explains why the
sandbox must be a separate Worker, and which limits apply.
## Try it locally
Install `@earendil-works/pi-codemode` and `wrangler` in each directory, then start both Workers in
one process:
```sh
npx wrangler dev -c agent/wrangler.jsonc -c sandbox/wrangler.jsonc
curl -X POST http://localhost:8787 --data 'const n = await tools.count({ add: 2 }); text("n=" + n); return n'
```
## Deploy
Deploy the sandbox first, because the agent's binding refers to it by name:
```sh
(cd sandbox && npx wrangler deploy)
(cd agent && npx wrangler deploy)
```

View File

@@ -0,0 +1,50 @@
// A Worker whose Durable Object runs codemode the way an agent session would: tools run here, in the
// session object, and scripts run in the separately deployed pi-codemode-sandbox Worker.
//
// curl -X POST http://localhost:8787 --data 'const n = await tools.count({ add: 2 }); text("n=" + n); return n'
import type { CodemodeSandboxDurableObject } from "@earendil-works/pi-codemode/cloudflare";
// The portable entry has no QuickJS module: the WebAssembly lives only in the sandbox Worker.
import { type CodemodeTool, RemoteCodemodeSandbox } from "@earendil-works/pi-codemode/portable";
import { DurableObject } from "cloudflare:workers";
interface Env {
SESSION: DurableObjectNamespace<Session>;
CODEMODE_SANDBOX: DurableObjectNamespace<CodemodeSandboxDurableObject>;
}
export class Session extends DurableObject<Env> {
private counter = 0;
async runCodemode(code: string) {
const tools: CodemodeTool[] = [
{
name: "count",
description: "Adds to a counter kept in the session and returns the new value",
execute: (args) => {
this.counter += (args as { add: number }).add;
return this.counter;
},
},
];
// Before: new CodemodeSandbox({ tools, timeoutMs: 60_000 })
const sandbox = new RemoteCodemodeSandbox({
tools,
timeoutMs: 60_000,
// One sandbox instance per session keeps a session's scripts together and apart from others.
remote: () => this.env.CODEMODE_SANDBOX.getByName(this.ctx.id.toString()),
});
try {
return await sandbox.execute(code);
} finally {
await sandbox.close();
}
}
}
export default {
async fetch(request, env) {
if (request.method !== "POST") return new Response("POST a script\n", { status: 405 });
const result = await env.SESSION.getByName("demo").runCodemode(await request.text());
return Response.json(result);
},
} satisfies ExportedHandler<Env>;

View File

@@ -0,0 +1,15 @@
{
"$schema": "node_modules/wrangler/config-schema.json",
"name": "pi-codemode-agent-example",
"main": "src/index.ts",
"compatibility_date": "2026-04-22",
"durable_objects": {
"bindings": [
{ "name": "SESSION", "class_name": "Session" },
// The sandbox class lives in its own Worker. `script_name` is what puts scripts in a different
// isolate; binding a class exported from this Worker would run them on the session's thread.
{ "name": "CODEMODE_SANDBOX", "class_name": "CodemodeSandbox", "script_name": "pi-codemode-sandbox" }
]
},
"migrations": [{ "tag": "v1", "new_sqlite_classes": ["Session"] }]
}

View File

@@ -0,0 +1,7 @@
// The whole sandbox Worker: one Durable Object class that runs codemode scripts for callers bound to
// it. Only Workers with a binding can reach the class; HTTP requests get a 404.
export { CodemodeSandboxDurableObject as CodemodeSandbox } from "@earendil-works/pi-codemode/cloudflare";
export default {
fetch: () => new Response("Not found", { status: 404 }),
};

View File

@@ -0,0 +1,13 @@
{
"$schema": "node_modules/wrangler/config-schema.json",
"name": "pi-codemode-sandbox",
"main": "src/index.ts",
"compatibility_date": "2026-04-22",
// Reached only through Durable Object bindings, never over HTTP.
"workers_dev": false,
"preview_urls": false,
"migrations": [{ "tag": "v1", "new_sqlite_classes": ["CodemodeSandbox"] }],
// Optional. The default is 30 seconds of CPU per request; one execution is one request. Keep it above
// the time the interrupt budget allows, so scripts fail with a timeout instead of resetting the object.
"limits": { "cpu_ms": 60000 }
}

View File

@@ -0,0 +1,76 @@
{
"name": "@earendil-works/pi-codemode",
"version": "1.3.0",
"description": "Sandboxed JavaScript execution where the only capability is calling injected tools",
"type": "module",
"main": "./dist/index.js",
"types": "./dist/index.d.ts",
"exports": {
".": {
"source": "./src/index.ts",
"types": "./dist/index.d.ts",
"import": "./dist/index.js"
},
"./declarations": {
"source": "./src/declarations.ts",
"types": "./dist/declarations.d.ts",
"import": "./dist/declarations.js"
},
"./source": {
"source": "./src/source.ts",
"types": "./dist/source.d.ts",
"import": "./dist/source.js"
},
"./portable": {
"source": "./src/portable.ts",
"types": "./dist/portable.d.ts",
"import": "./dist/portable.js"
},
"./cloudflare": {
"source": "./src/cloudflare.ts",
"types": "./dist/cloudflare.d.ts",
"import": "./dist/cloudflare.js"
},
"./worker": {
"source": "./src/runtime/worker.ts",
"types": "./dist/runtime/worker.d.ts",
"import": "./dist/runtime/worker.js"
}
},
"files": [
"dist",
"README.md"
],
"scripts": {
"clean": "shx rm -rf dist",
"build": "tsc -p tsconfig.build.json",
"test": "vitest --run",
"prepublishOnly": "npm run clean && npm run build"
},
"keywords": [
"pi",
"codemode",
"sandbox",
"quickjs",
"wasm",
"cloudflare",
"durable-objects"
],
"author": "Earendil Works",
"license": "MIT",
"repository": {
"type": "git",
"url": "git+https://github.com/earendil-works/pi.git",
"directory": "packages/codemode"
},
"engines": {
"node": ">=22.19.0"
},
"dependencies": {
"quickjs-wasi": "3.6.2"
},
"devDependencies": {
"shx": "0.3.4",
"vitest": "4.1.11"
}
}

View File

@@ -0,0 +1,19 @@
// Modules that only exist when cloudflare.ts is built by Wrangler. Consumers get the real
// declarations from @cloudflare/workers-types or `wrangler types`; these keep this repository's
// type check free of a Workers type dependency and are not published.
declare module "cloudflare:workers" {
export abstract class DurableObject<Env = unknown> {
protected ctx: unknown;
protected env: Env;
constructor(ctx: unknown, env: Env);
}
}
// A pattern, not the exact specifier: coding-agent declares "quickjs-wasi/quickjs.wasm" as a path
// for Bun, and an exact declaration takes precedence where both are visible. cloudflare.ts casts the
// import to the compiled module Wrangler actually produces.
declare module "*/quickjs.wasm" {
const module: unknown;
export default module;
}

View File

@@ -0,0 +1,75 @@
/**
* Cloudflare Workers entry, for Workers built with Wrangler.
*
* Wrangler's default `CompiledWasm` rule turns the `.wasm` import below into a
* `WebAssembly.Module`; Workers cannot compile WebAssembly from bytes at runtime. Other bundlers
* do not support the import: use `@earendil-works/pi-codemode/portable` with your own module there.
*
* Two ways to run codemode on Cloudflare:
* - `RemoteCodemodeSandbox` with a {@link CodemodeSandboxDurableObject} deployed as its own
* Worker: scripts run in a separate isolate and never block the caller. Use this from a Durable
* Object that serves a session.
* - {@link createInlineCodemodeSandbox}: scripts run on the caller's thread. Simplest; a script
* that computes blocks the caller until it awaits a tool or uses up its interrupt budget.
*/
import { DurableObject } from "cloudflare:workers";
import quickjsWasmImport from "quickjs-wasi/quickjs.wasm";
import {
DEFAULT_INTERRUPT_BUDGET,
InlineCodemodeSandbox,
type InlineCodemodeSandboxOptions,
} from "./runtime/inline.ts";
import { type CodemodeRemoteExchange, type CodemodeRemoteRequest, serveCodemodeRemote } from "./runtime/remote.ts";
import type { CodemodeWasmModule } from "./wasm.ts";
export * from "./portable.ts";
// Wrangler's `CompiledWasm` rule makes this a compiled module; the declaration cannot say so (see
// cloudflare-modules.d.ts).
const quickjsWasm = quickjsWasmImport as unknown as CodemodeWasmModule;
/** Default VM memory limit of {@link CodemodeSandboxDurableObject}. Isolates have 128 MB in total. */
export const DEFAULT_REMOTE_MEMORY_LIMIT_BYTES = 32 * 1024 * 1024;
/** `quickjs-wasi/quickjs.wasm`, compiled by Wrangler. */
export const cloudflareQuickJSWasm: CodemodeWasmModule = quickjsWasm;
/** `InlineCodemodeSandbox` with the QuickJS module Wrangler bundled. */
export function createInlineCodemodeSandbox(options: Omit<InlineCodemodeSandboxOptions, "wasm"> = {}) {
return new InlineCodemodeSandbox({ ...options, wasm: quickjsWasm });
}
/**
* The remote end of `RemoteCodemodeSandbox`: a Durable Object whose one RPC method runs a script's
* VM and calls back into the client for every tool call.
*
* Deploy it as its own Worker and bind it from the Worker that runs your agent with `script_name`.
* A separate Worker is what keeps scripts off the agent's thread: a class in the same Worker can
* share an isolate, and therefore a thread, with the objects that call it.
*
* The object keeps no state between executions and never touches storage. Executions that reach
* the same instance share its thread: they interleave while they wait on tools, and a script that
* computes delays the others. Name instances per session (`getByName(sessionId)`) to keep one
* session's scripts together, or per execution for full independence.
*
* Limits apply to every request, whatever the client asks for. Override the fields in a subclass:
*
* export class CodemodeSandbox extends CodemodeSandboxDurableObject {
* protected maxInterruptBudget = 200_000;
* }
*/
export class CodemodeSandboxDurableObject<Env = unknown> extends DurableObject<Env> {
/** Interrupt polls per execution; the client may only ask for less. See `InlineCodemodeSandboxOptions`. */
protected maxInterruptBudget: number = DEFAULT_INTERRUPT_BUDGET;
/** VM memory per execution; the client may only ask for less. */
protected maxMemoryLimitBytes: number = DEFAULT_REMOTE_MEMORY_LIMIT_BYTES;
/** Called by `RemoteCodemodeSandbox` over RPC. Resolves when the script has settled or was stopped. */
execute(request: CodemodeRemoteRequest, exchange: CodemodeRemoteExchange): Promise<void> {
return serveCodemodeRemote(request, exchange, {
wasm: quickjsWasm,
maxInterruptBudget: this.maxInterruptBudget,
maxMemoryLimitBytes: this.maxMemoryLimitBytes,
});
}
}

View File

@@ -0,0 +1,340 @@
import { toCodemodeIdentifier } from "./identifier.ts";
export { toCodemodeIdentifier };
import type { CodemodeJsonSchema, CodemodeTool } from "./types.ts";
const IDENTIFIER = /^[A-Za-z_$][A-Za-z0-9_$]*$/;
const INDENT = " ";
/** Largest rendered input type, in characters, before it becomes `unknown`. */
export const DEFAULT_INPUT_SCHEMA_MAX_CHARS = 16_000;
/** Local `$ref` expansions per rendered schema, so shared definitions cannot blow up the output. */
const MAX_REF_EXPANSIONS = 32;
/** JSON Schema keywords with a number or string value shown as `@keyword value` property tags. */
const TAG_KEYWORDS = [
"minimum",
"exclusiveMinimum",
"maximum",
"exclusiveMaximum",
"multipleOf",
"minLength",
"maxLength",
"pattern",
"format",
"minItems",
"maxItems",
];
/** Tag keywords whose values all hold at once, so only the largest (lower) or smallest (upper) matters. */
const LOWER_BOUNDS = new Set(["minimum", "exclusiveMinimum", "minLength", "minItems"]);
const UPPER_BOUNDS = new Set(["maximum", "exclusiveMaximum", "maxLength", "maxItems"]);
/** Longest property tag; longer ones (large defaults, long patterns) are left out. */
const MAX_TAG_CHARS = 80;
/** Characters that end a `//` comment. */
const LINE_TERMINATORS = /[\r\n\u2028\u2029]/g;
export interface RenderDeclarationsOptions {
tools?: readonly CodemodeTool[];
globals?: readonly CodemodeTool[];
}
/**
* Render TypeScript declarations for the script-visible API. Tools become members of
* `declare const tools`, globals become `declare function` statements, and `ns.member` globals
* members of `declare const ns`. Descriptions become doc comments; schemas become types.
*/
export function renderDeclarations(options: RenderDeclarationsOptions): string {
const sections: string[] = [];
const tools = options.tools ?? [];
if (tools.length > 0) {
const members = tools.map(
(tool) => `${docComment(tool.description, INDENT)}${INDENT}${renderToolSignature(tool)}`,
);
sections.push(`declare const tools: {\n${members.join("\n")}\n};`);
}
const namespaces = new Map<string, string[]>();
for (const global of options.globals ?? []) {
const dot = global.name.indexOf(".");
if (dot === -1) {
sections.push(renderGlobal(`declare function ${global.name}`, global, ""));
continue;
}
const namespace = global.name.slice(0, dot);
const members = namespaces.get(namespace) ?? [];
if (members.length === 0) namespaces.set(namespace, members);
members.push(renderGlobal(global.name.slice(dot + 1), global, INDENT));
}
for (const [namespace, members] of namespaces) {
sections.push(`declare const ${namespace}: {\n${members.join("\n")}\n};`);
}
return sections.join("\n\n");
}
/**
* One tool as a member of the `tools` object: `name(args: T): Promise<R>;` with the
* name as the identifier scripts use. Input types longer than `inputMaxChars` render as `unknown`.
*/
export function renderToolSignature(
tool: Pick<CodemodeTool, "name" | "inputSchema" | "outputSchema">,
options: { inputMaxChars?: number } = {},
): string {
const input =
tool.inputSchema === undefined
? "unknown"
: schemaToType(tool.inputSchema, { maxChars: options.inputMaxChars ?? DEFAULT_INPUT_SCHEMA_MAX_CHARS });
return `${toCodemodeIdentifier(tool.name)}(args: ${input}): Promise<${renderToolOutputType(tool.outputSchema)}>;`;
}
/**
* A tool's sample: the description followed by the tool's declaration. Used for tool
* listings and `ALL_TOOLS` entries.
*/
export function renderToolSample(
tool: Pick<CodemodeTool, "name" | "description" | "inputSchema" | "outputSchema">,
options: { inputMaxChars?: number } = {},
): string {
const declaration = `declare const tools: { ${renderToolSignature(tool, options)} };`;
return `${tool.description?.trim() ?? ""}\n\ncodemode tool declaration:\n\`\`\`ts\n${declaration}\n\`\`\``;
}
/** The type a tool call resolves to: the output schema's type, and `unknown` without a schema. */
export function renderToolOutputType(schema: CodemodeJsonSchema | undefined): string {
return schema === undefined ? "unknown" : schemaToType(schema);
}
function renderGlobal(head: string, global: CodemodeTool, indent: string): string {
if (global.signature !== undefined) {
return `${docComment(global.description, indent)}${indent}${head}${global.signature};`;
}
const input = global.inputSchema === undefined ? "unknown" : schemaToType(global.inputSchema);
const output = global.outputSchema === undefined ? "unknown" : schemaToType(global.outputSchema);
return `${docComment(global.description, indent)}${indent}${head}(args: ${input}): Promise<${output}>;`;
}
function docComment(description: string | undefined, indent: string): string {
const text = description?.trim();
if (!text) return "";
const lines = text.replaceAll("*/", "*\\/").split(/\r?\n/);
if (lines.length === 1) return `${indent}/** ${lines[0]} */\n`;
return `${indent}/**\n${lines.map((line) => `${indent} *${line ? ` ${line}` : ""}`).join("\n")}\n${indent} */\n`;
}
function propertyKey(name: string): string {
return IDENTIFIER.test(name) ? name : JSON.stringify(name);
}
function union(types: string[]): string {
const unique = [...new Set(types)];
if (unique.includes("unknown")) return "unknown";
return unique.length === 0 ? "never" : unique.join(" | ");
}
/**
* Convert a JSON Schema to a TypeScript type expression: objects on one line (`{ a: string; b?: number; }`) with properties sorted by name,
* or one property per line with `//` comments when a property has a description or constraints the
* type cannot express (`// @integer @minimum 1 @default 20`); `Array<T>` for arrays. Local references
* (`#/$defs/...`, `#/definitions/...`) resolve against `schema`; recursive and remote references
* render as `unknown`. A result longer than `maxChars` is rendered again without the constraint tags,
* and renders as `unknown` if it is still too long.
*/
export function schemaToType(schema: CodemodeJsonSchema, options: { maxChars?: number } = {}): string {
const render = (tags: boolean) => toType(schema, { root: schema, resolving: new Set(), expansions: 0, tags });
const type = render(true);
if (options.maxChars === undefined || type.length <= options.maxChars) return type;
const untagged = render(false);
return untagged.length <= options.maxChars ? untagged : "unknown";
}
interface SchemaContext {
root: CodemodeJsonSchema;
/** References being expanded on the current path, to stop at recursive types. */
resolving: Set<string>;
expansions: number;
/** Whether property comments include constraint tags. */
tags: boolean;
}
function resolveRef(ref: string, root: CodemodeJsonSchema): CodemodeJsonSchema | undefined {
if (ref !== "#" && !ref.startsWith("#/")) return undefined;
let current: unknown = root;
for (const segment of ref.slice(2).split("/").filter(Boolean)) {
const key = decodeURIComponent(segment).replaceAll("~1", "/").replaceAll("~0", "~");
if (typeof current !== "object" || current === null || Array.isArray(current) || !(key in current)) {
return undefined;
}
current = (current as Record<string, unknown>)[key];
}
if (typeof current === "boolean") return current;
return typeof current === "object" && current !== null && !Array.isArray(current)
? (current as Record<string, unknown>)
: undefined;
}
function toType(schema: CodemodeJsonSchema, context: SchemaContext): string {
if (schema === true) return "unknown";
if (schema === false) return "never";
if (typeof schema !== "object" || schema === null || Array.isArray(schema)) return "unknown";
if (typeof schema.$ref === "string") {
const ref = schema.$ref;
if (context.resolving.has(ref) || context.expansions >= MAX_REF_EXPANSIONS) return "unknown";
const target = resolveRef(ref, context.root);
if (target === undefined) return "unknown";
context.expansions++;
context.resolving.add(ref);
try {
return toType(target, context);
} finally {
context.resolving.delete(ref);
}
}
if ("const" in schema) return JSON.stringify(schema.const) ?? "unknown";
if (Array.isArray(schema.enum)) return union(schema.enum.map((value) => JSON.stringify(value) ?? "unknown"));
const variants = Array.isArray(schema.anyOf) ? schema.anyOf : Array.isArray(schema.oneOf) ? schema.oneOf : undefined;
if (variants) return union(variants.map((variant) => toType(variant as CodemodeJsonSchema, context)));
if (Array.isArray(schema.allOf)) {
const parts = schema.allOf
.map((part) => toType(part as CodemodeJsonSchema, context))
.filter((part) => part !== "unknown");
return parts.length === 0
? "unknown"
: parts.map((part) => (part.includes(" | ") ? `(${part})` : part)).join(" & ");
}
const type = schema.type;
if (Array.isArray(type)) return union(type.map((entry) => toType({ ...schema, type: entry }, context)));
switch (type) {
case "string":
return "string";
case "number":
case "integer":
return "number";
case "boolean":
return "boolean";
case "null":
return "null";
case "array":
return arrayType(schema, context);
case "object":
return objectType(schema, context);
case undefined:
if ("properties" in schema || "additionalProperties" in schema || "required" in schema) {
return objectType(schema, context);
}
if ("items" in schema || "prefixItems" in schema) return arrayType(schema, context);
return "unknown";
default:
return "unknown";
}
}
function arrayType(schema: Record<string, unknown>, context: SchemaContext): string {
if (schema.items !== undefined && !Array.isArray(schema.items)) {
return `Array<${toType(schema.items as CodemodeJsonSchema, context)}>`;
}
const tuple = Array.isArray(schema.prefixItems)
? schema.prefixItems
: Array.isArray(schema.items)
? schema.items
: [];
if (tuple.length > 0) return `[${tuple.map((item) => toType(item as CodemodeJsonSchema, context)).join(", ")}]`;
return "unknown[]";
}
function descriptionOf(property: unknown): string {
const description = (property as { description?: unknown } | null | undefined)?.description;
return typeof description === "string" ? description.trim() : "";
}
/**
* Constraints of a property that its TypeScript type cannot express, as JSDoc-style tags:
* `@integer @minimum 1 @maximum 100 @default 20`. Empty when there are none.
*/
function constraintTags(schema: unknown): string {
if (typeof schema !== "object" || schema === null || Array.isArray(schema)) return "";
const property = schema as Record<string, unknown>;
// Optional fields such as Pydantic's `int | None` keep their constraints in the non-null branch.
// The branch's constraints apply together with the property's, so both are read.
const variants = Array.isArray(property.anyOf) ? property.anyOf : property.oneOf;
const nonNull: unknown[] = Array.isArray(variants)
? variants.filter((variant) => (variant as { type?: unknown } | null)?.type !== "null")
: [];
const branch = nonNull.length === 1 ? nonNull[0] : undefined;
const branches =
typeof branch === "object" && branch !== null && !Array.isArray(branch)
? [property, branch as Record<string, unknown>]
: [property];
// Draft-04 and OpenAPI 3.0 make `minimum`/`maximum` exclusive with a boolean `exclusiveMinimum`/`exclusiveMaximum`.
const schemas = branches.map(
(schema): Record<string, unknown> => ({
...schema,
...(schema.exclusiveMinimum === true ? { minimum: undefined, exclusiveMinimum: schema.minimum } : {}),
...(schema.exclusiveMaximum === true ? { maximum: undefined, exclusiveMaximum: schema.maximum } : {}),
}),
);
const tags: string[] = [];
const isInteger = (type: unknown) => type === "integer" || (Array.isArray(type) && type.includes("integer"));
if (schemas.some((schema) => isInteger(schema.type))) tags.push("@integer");
for (const keyword of TAG_KEYWORDS) {
const values = schemas.map((schema) => schema[keyword]);
if (LOWER_BOUNDS.has(keyword) || UPPER_BOUNDS.has(keyword)) {
const numbers = values.filter((value) => typeof value === "number");
if (numbers.length === 0) continue;
tags.push(`@${keyword} ${LOWER_BOUNDS.has(keyword) ? Math.max(...numbers) : Math.min(...numbers)}`);
continue;
}
for (const value of new Set(values)) {
if (typeof value === "number" || typeof value === "string") tags.push(`@${keyword} ${value}`);
}
}
if (schemas.some((schema) => schema.uniqueItems === true)) tags.push("@uniqueItems");
// Annotations, unlike constraints, come from the property itself.
if (property.default !== undefined && property.default !== null) {
tags.push(`@default ${JSON.stringify(property.default)}`);
}
if (property.deprecated === true) tags.push("@deprecated");
return tags
.filter((tag) => tag.length <= MAX_TAG_CHARS)
.map((tag) => tag.replace(LINE_TERMINATORS, (char) => `\\u${char.charCodeAt(0).toString(16).padStart(4, "0")}`))
.join(" ");
}
function objectType(schema: Record<string, unknown>, context: SchemaContext): string {
const properties =
typeof schema.properties === "object" && schema.properties !== null && !Array.isArray(schema.properties)
? (schema.properties as Record<string, unknown>)
: {};
const required = new Set(Array.isArray(schema.required) ? schema.required : []);
const names = Object.keys(properties).sort();
const members = names.map((name) => {
const optional = required.has(name) ? "" : "?";
return `${propertyKey(name)}${optional}: ${toType(properties[name] as CodemodeJsonSchema, context)};`;
});
const additional = schema.additionalProperties;
if (additional !== undefined && additional !== false) {
const type = additional === true ? "unknown" : toType(additional as CodemodeJsonSchema, context);
members.push(`[key: string]: ${type};`);
} else if (additional === undefined && names.length === 0) {
members.push("[key: string]: unknown;");
}
if (members.length === 0) return "{}";
const comments = names.map((name) => {
const lines = descriptionOf(properties[name])
.split(LINE_TERMINATORS)
.map((line) => line.trim())
.filter(Boolean);
const tags = context.tags ? constraintTags(properties[name]) : "";
return tags ? [...lines, tags] : lines;
});
if (!comments.some((comment) => comment.length > 0)) return `{ ${members.join(" ")} }`;
const lines = ["{"];
comments.forEach((comment, index) => {
for (const line of comment) lines.push(`${INDENT}// ${line}`);
lines.push(`${INDENT}${members[index].replaceAll("\n", `\n${INDENT}`)}`);
});
for (const member of members.slice(names.length)) lines.push(`${INDENT}${member}`);
lines.push("}");
return lines.join("\n");
}

View File

@@ -0,0 +1,12 @@
/**
* The identifier a script uses for a tool: characters that are not valid in a JavaScript
* identifier become `_`. `mcp__docs__search` stays as is, `my-tool` becomes `my_tool`.
*/
export function toCodemodeIdentifier(name: string): string {
let identifier = "";
for (const char of name) {
const valid = identifier === "" ? /^[A-Za-z_$]$/.test(char) : /^[A-Za-z0-9_$]$/.test(char);
identifier += valid ? char : "_";
}
return identifier === "" ? "_" : identifier;
}

View File

@@ -0,0 +1,65 @@
export {
DEFAULT_INPUT_SCHEMA_MAX_CHARS,
type RenderDeclarationsOptions,
renderDeclarations,
renderToolOutputType,
renderToolSample,
renderToolSignature,
schemaToType,
} from "./declarations.ts";
export { toCodemodeIdentifier } from "./identifier.ts";
export { CodemodeSandboxBase } from "./runtime/execution.ts";
export { CodemodeSandbox } from "./runtime/host.ts";
export {
DEFAULT_INTERRUPT_BUDGET,
InlineCodemodeSandbox,
type InlineCodemodeSandboxOptions,
} from "./runtime/inline.ts";
export {
MAX_OUTPUT_CHARS,
MAX_OUTPUT_ITEMS,
MAX_STORE_TOTAL_CHARS,
MAX_STORE_VALUE_CHARS,
} from "./runtime/prelude-source.ts";
export {
type CodemodeRemote,
type CodemodeRemoteExchange,
type CodemodeRemoteMessage,
type CodemodeRemoteReply,
type CodemodeRemoteRequest,
type CodemodeServeOptions,
RemoteCodemodeSandbox,
type RemoteCodemodeSandboxOptions,
serveCodemodeRemote,
} from "./runtime/remote.ts";
export {
CODEMODE_OPTIONS_PREFIX,
CODEMODE_SOURCE_GRAMMAR,
CodemodeSourceError,
type CodemodeSourceOptions,
type ParsedCodemodeSource,
parseCodemodeSource,
} from "./source.ts";
export type {
CodemodeCall,
CodemodeCallStatus,
CodemodeDeclaredGlobal,
CodemodeDeclaredTool,
CodemodeDuring,
CodemodeError,
CodemodeErrorKind,
CodemodeExecuteOptions,
CodemodeHostCall,
CodemodeJsonSchema,
CodemodeOutcome,
CodemodeOutputItem,
CodemodeResult,
CodemodeRun,
CodemodeRunOptions,
CodemodeSandboxBaseOptions,
CodemodeSandboxOptions,
CodemodeStoreWrites,
CodemodeTool,
CodemodeToolContext,
} from "./types.ts";
export { type CodemodeWasmModule, loadQuickJSWasm } from "./wasm.ts";

View File

@@ -0,0 +1,53 @@
/**
* Sandboxes that need no Node built-ins: `InlineCodemodeSandbox` runs the VM on the calling thread
* and `RemoteCodemodeSandbox` runs it in another isolate. Neither imports `node:*` modules, so this
* entry works in Cloudflare Workers, Durable Objects, and browsers. `CodemodeSandbox`, which uses
* worker threads, stays in the main entry.
*/
export { toCodemodeIdentifier } from "./identifier.ts";
export { CodemodeSandboxBase } from "./runtime/execution.ts";
export {
DEFAULT_INTERRUPT_BUDGET,
InlineCodemodeSandbox,
type InlineCodemodeSandboxOptions,
} from "./runtime/inline.ts";
export {
MAX_OUTPUT_CHARS,
MAX_OUTPUT_ITEMS,
MAX_STORE_TOTAL_CHARS,
MAX_STORE_VALUE_CHARS,
} from "./runtime/prelude-source.ts";
export type { HostToWorkerMessage, VmStartData, WorkerToHostMessage } from "./runtime/protocol.ts";
export {
type CodemodeRemote,
type CodemodeRemoteExchange,
type CodemodeRemoteMessage,
type CodemodeRemoteReply,
type CodemodeRemoteRequest,
type CodemodeServeOptions,
RemoteCodemodeSandbox,
type RemoteCodemodeSandboxOptions,
serveCodemodeRemote,
} from "./runtime/remote.ts";
export type {
CodemodeCall,
CodemodeCallStatus,
CodemodeDeclaredGlobal,
CodemodeDeclaredTool,
CodemodeDuring,
CodemodeError,
CodemodeErrorKind,
CodemodeExecuteOptions,
CodemodeHostCall,
CodemodeJsonSchema,
CodemodeOutcome,
CodemodeOutputItem,
CodemodeResult,
CodemodeRun,
CodemodeRunOptions,
CodemodeSandboxBaseOptions,
CodemodeStoreWrites,
CodemodeTool,
CodemodeToolContext,
} from "./types.ts";
export type { CodemodeWasmModule } from "./wasm.ts";

View File

@@ -0,0 +1,573 @@
/**
* Host side of an execution, shared by every place a VM can run.
*
* The host owns everything the script must not influence: the tool table, call records, output,
* the deadline, the caller's abort signal, and the decoding of what the VM reports. The VM itself
* runs somewhere a {@link VmLauncher} decides: a worker thread (`CodemodeSandbox`), the calling
* thread (`InlineCodemodeSandbox`), or another isolate reached over RPC (`RemoteCodemodeSandbox`).
* The two sides only exchange the messages in `protocol.ts`, so the launchers differ in transport
* and termination, not in behavior.
*
* Lifecycle of one execution:
* 1. The constructor arms the deadline and abort listener, then asks the launcher for a channel.
* 2. Messages from the VM arrive through {@link VmEvents.message}; tool calls run here and their
* results go back through {@link VmChannel.post}. Replies produced before the launcher has
* returned its channel are queued and flushed when it arrives.
* 3. The first terminal event wins: `done`, `crash`, a launcher failure, the deadline, or an abort.
* `finish()` aborts pending tool calls, asks the channel to stop, and resolves the result once
* that request completes. A worker or remote VM may exit later. Later events are ignored.
*
* The launcher, not this class, is responsible for actually stopping a running VM: a worker is
* terminated, an in-thread VM is interrupted and disposed, and a remote VM is told to stop at its
* next exchange.
*/
import { toCodemodeIdentifier } from "../identifier.ts";
import type {
CodemodeCall,
CodemodeCallStatus,
CodemodeDeclaredGlobal,
CodemodeDeclaredTool,
CodemodeDuring,
CodemodeError,
CodemodeExecuteOptions,
CodemodeOutcome,
CodemodeOutputItem,
CodemodeResult,
CodemodeRun,
CodemodeRunOptions,
CodemodeSandboxBaseOptions,
CodemodeStoreWrites,
CodemodeTool,
} from "../types.ts";
import {
type HostToWorkerMessage,
isWorkerToHostMessage,
type VmStartData,
type WorkerToHostMessage,
} from "./protocol.ts";
const DEFAULT_TIMEOUT_MS = 300_000;
const IDENTIFIER = /^[A-Za-z_$][A-Za-z0-9_$]*$/;
const RESERVED_GLOBALS: ReadonlySet<string> = new Set([
"tools",
"ALL_TOOLS",
"console",
"text",
"image",
"exit",
"globalThis",
"store",
"load",
]);
/** Callbacks a launcher uses to report what its VM does. */
export interface VmEvents {
/** A message from the VM. Validated here, so launchers can pass through anything they receive. */
message(message: unknown): void;
/** The VM failed or stopped outside the bridge, for example a worker exit or a remote error. */
failure(error: CodemodeError): void;
}
/** The host's handle on a running VM. */
export interface VmChannel {
post(message: HostToWorkerMessage): void;
/** Requests VM shutdown. Called once; may resolve before a worker or remote VM actually exits. */
stop(): Promise<void>;
}
/**
* Starts a VM for one execution. Rejections become `sandbox` errors with the rejection's message.
* The VM may start reporting messages before the returned promise settles.
*/
export type VmLauncher = (data: VmStartData, events: VmEvents) => VmChannel | Promise<VmChannel>;
export function errorMessage(error: unknown): string {
return error instanceof Error ? error.message : String(error);
}
function serializeStore(store: Readonly<Record<string, unknown>> | undefined): Record<string, string> {
const serialized: Record<string, string> = {};
for (const [key, value] of Object.entries(store ?? {})) {
const json = JSON.stringify(value);
if (json !== undefined) serialized[key] = json;
}
return serialized;
}
/**
* The prelude that serializes values for the host runs in the same VM as the script, so a script
* that patches built-ins (for example `Array.prototype.toJSON`) can make it send malformed data.
* Payloads that do not decode throw this error, and the execution fails as a sandbox error.
*/
class BridgeError extends Error {}
function parseBridgeJson(json: string, what: string): unknown {
try {
return JSON.parse(json);
} catch {
throw new BridgeError(`${what} is not valid JSON`);
}
}
function parseStoreWrites(json: string): CodemodeStoreWrites {
const entries = parseBridgeJson(json, "store writes");
if (!Array.isArray(entries)) throw new BridgeError("store writes are not an array");
const writes: CodemodeStoreWrites = { set: {}, delete: [] };
for (const entry of entries) {
if (
!Array.isArray(entry) ||
typeof entry[0] !== "string" ||
(entry.length === 2 ? typeof entry[1] !== "string" : entry.length !== 1)
) {
throw new BridgeError("store writes contain a malformed entry");
}
const [key, value] = entry as [string, string?];
if (value === undefined) writes.delete.push(key);
else writes.set[key] = parseBridgeJson(value, `store value for ${JSON.stringify(key)}`);
}
return writes;
}
function parseScriptError(json: string): CodemodeError {
const parsed = parseBridgeJson(json, "script error");
if (typeof parsed !== "object" || parsed === null) throw new BridgeError("script error is not an object");
const { name, message, stack } = parsed as Record<string, unknown>;
if (
typeof message !== "string" ||
(name !== undefined && typeof name !== "string") ||
(stack !== undefined && typeof stack !== "string")
) {
throw new BridgeError("script error is malformed");
}
return { kind: "script", name, message, stack };
}
interface PendingCall {
record: CodemodeCall | undefined;
startedAt: number;
controller: AbortController;
}
/** Checks global names: identifiers or `namespace.member`, not shadowing a built-in global or each other. */
function checkGlobals(globals: readonly CodemodeDeclaredGlobal[]): void {
const names = new Set<string>();
const namespaces = new Set<string>();
for (const global of globals) {
const parts = global.name.split(".");
if (parts.length > 2 || !parts.every((part) => IDENTIFIER.test(part)) || RESERVED_GLOBALS.has(parts[0])) {
throw new Error(`Invalid global name "${global.name}"`);
}
if (names.has(global.name)) throw new Error(`Global "${global.name}" is already registered`);
if (parts.length === 2) namespaces.add(parts[0]);
names.add(global.name);
}
for (const name of namespaces) {
if (names.has(name)) throw new Error(`Global "${name}" conflicts with the namespace "${name}"`);
}
}
/** Callbacks of a host-controlled run: its calls go to the caller, which answers them with `deliver()`. */
type Control = Pick<CodemodeRunOptions, "onStarted" | "onCall" | "onIdle">;
interface ExecutionOptions {
code: string;
tools: readonly CodemodeDeclaredTool[];
globals: readonly CodemodeDeclaredGlobal[];
/** Run the script's calls; unused when `control` is set. */
executors: { tools: ReadonlyMap<string, CodemodeTool>; globals: ReadonlyMap<string, CodemodeTool> };
timeoutMs: number;
signal: AbortSignal | undefined;
memoryLimitBytes: number | undefined;
store: Record<string, string>;
deterministic?: { startedAt: number; seed: number };
control?: Control;
launch: VmLauncher;
}
/** One script run in its own VM. */
class Execution {
readonly promise: Promise<CodemodeResult>;
private resolveResult!: (result: CodemodeResult) => void;
private channel: VmChannel | undefined;
/** Replies produced before the launcher returned the channel. */
private unsent: HostToWorkerMessage[] = [];
private readonly executors: ExecutionOptions["executors"];
private readonly control: Control | undefined;
/** Result numbers `deliver()` used. */
private readonly delivered = new Set<number>();
private readonly signal: AbortSignal | undefined;
private readonly timer: ReturnType<typeof setTimeout> | undefined;
private readonly output: CodemodeOutputItem[] = [];
private readonly calls: CodemodeCall[] = [];
private readonly pending = new Map<number, PendingCall>();
/** Whether the execution has its outcome. */
finished = false;
constructor(options: ExecutionOptions) {
this.promise = new Promise<CodemodeResult>((resolve) => {
this.resolveResult = resolve;
});
this.executors = options.executors;
this.control = options.control;
this.signal = options.signal;
if (Number.isFinite(options.timeoutMs)) {
this.timer = setTimeout(() => {
this.finish({ kind: "timeout", message: `Execution timed out after ${options.timeoutMs} ms` });
}, options.timeoutMs);
}
if (options.signal) {
if (options.signal.aborted) {
this.onAbort();
} else {
options.signal.addEventListener("abort", this.onAbort, { once: true });
}
}
if (this.finished) return;
const data: VmStartData = {
code: options.code,
tools: options.tools.map((tool) => ({
name: tool.name,
jsName: tool.jsName ?? toCodemodeIdentifier(tool.name),
description: tool.description ?? "",
})),
globals: options.globals.map((global) => ({ name: global.name, spread: global.spread === true })),
memoryLimitBytes: options.memoryLimitBytes,
store: options.store,
...(options.deterministic ? { deterministic: options.deterministic } : {}),
};
const events: VmEvents = {
message: (message) => this.receive(message),
failure: (error) => this.finish(error),
};
new Promise<VmChannel>((resolve) => resolve(options.launch(data, events))).then(
(channel) => {
if (this.finished) {
void channel.stop().catch(() => undefined);
return;
}
this.channel = channel;
for (const message of this.unsent.splice(0)) channel.post(message);
},
(error: unknown) => {
this.finish({ kind: "sandbox", message: errorMessage(error) });
},
);
}
/** `CodemodeRun.deliver()`. */
deliver(seq: number, now: number, id: number, outcome: CodemodeOutcome): void {
if (this.finished) return;
if (!Number.isFinite(now)) throw new Error(`The clock value of result ${seq} is not a finite number`);
if (!Number.isInteger(seq) || seq < 1 || this.delivered.has(seq)) {
throw new Error(`Result number ${seq} is not a positive integer used once`);
}
const pending = this.pending.get(id);
if (pending === undefined) throw new Error(`Call ${id} is not waiting for a result`);
if (!outcome.ok && outcome.properties !== undefined) {
const { properties } = outcome;
if (typeof properties !== "object" || properties === null || Array.isArray(properties)) {
throw new Error(`The properties of result ${seq} are not an object`);
}
}
// Serialized before anything is recorded, so a value JSON cannot hold leaves the call waiting for another answer.
const payload = outcome.ok
? outcome.value === undefined
? undefined
: JSON.stringify(outcome.value)
: outcome.message;
const properties =
!outcome.ok && outcome.properties !== undefined ? JSON.stringify(outcome.properties) : undefined;
this.delivered.add(seq);
this.pending.delete(id);
if (pending.record) {
pending.record.status = outcome.ok ? "ok" : "error";
pending.record.durationMs = performance.now() - pending.startedAt;
}
this.post({
type: "result",
id,
ok: outcome.ok,
payload,
seq,
now,
...(properties === undefined ? {} : { properties }),
});
}
abort(message: string): Promise<CodemodeResult> {
this.finish({ kind: "aborted", message });
return this.promise;
}
private readonly onAbort = (): void => {
const reason: unknown = this.signal?.reason;
this.finish({ kind: "aborted", message: reason instanceof Error ? reason.message : "Execution aborted" });
};
private post(message: HostToWorkerMessage): void {
if (this.channel) this.channel.post(message);
else this.unsent.push(message);
}
/** An exception here would escape into the launcher's transport and leave the execution unsettled. */
private receive(message: unknown): void {
try {
this.handleMessage(message);
} catch (error) {
this.finish({
kind: "sandbox",
message:
error instanceof BridgeError
? `Sandbox bridge broken: ${error.message}. The script may have modified built-ins such as a prototype's toJSON.`
: `Sandbox host failed: ${errorMessage(error)}`,
});
}
}
private handleMessage(message: unknown): void {
if (this.finished) return;
if (!isWorkerToHostMessage(message)) throw new BridgeError("unknown message from the worker");
switch (message.type) {
case "output":
this.output.push(message.item);
break;
case "call":
if (this.pending.has(message.id)) throw new BridgeError(`duplicate call id ${message.id}`);
if (this.control) this.forwardCall(message, this.control);
else void this.handleCall(message);
break;
case "started":
this.control?.onStarted?.(message.fingerprint);
break;
case "idle":
this.control?.onIdle?.(message.seq, message.issued);
break;
case "done":
this.handleDone(message);
break;
case "crash":
this.finish({ kind: "sandbox", message: message.message });
break;
}
}
private handleDone(message: Extract<WorkerToHostMessage, { type: "done" }>): void {
// Decode everything before finish(), which must not throw once it starts.
if (!message.ok) {
this.finish(parseScriptError(message.error), undefined, undefined, message.during);
return;
}
const value = message.value === undefined ? undefined : parseBridgeJson(message.value, "return value");
this.finish(undefined, value, parseStoreWrites(message.writes), message.during);
}
/** A host-controlled run: record the call and hand it to the caller, which answers with `deliver()`. */
private forwardCall(message: Extract<WorkerToHostMessage, { type: "call" }>, control: Control): void {
const { id, name, target } = message;
const args = message.args === undefined ? undefined : parseBridgeJson(message.args, "call arguments");
const record: CodemodeCall | undefined =
target === "tool" ? { name, status: "cancelled", durationMs: 0 } : undefined;
if (record) this.calls.push(record);
this.pending.set(id, { record, startedAt: performance.now(), controller: new AbortController() });
control.onCall({ id, target, name, args });
}
private async handleCall(message: Extract<WorkerToHostMessage, { type: "call" }>): Promise<void> {
const { id, name } = message;
const isTool = message.target === "tool";
const record: CodemodeCall | undefined = isTool ? { name, status: "cancelled", durationMs: 0 } : undefined;
if (record) this.calls.push(record);
const pending: PendingCall = { record, startedAt: performance.now(), controller: new AbortController() };
this.pending.set(id, pending);
let status: CodemodeCallStatus;
let reply: HostToWorkerMessage;
try {
const tool = (isTool ? this.executors.tools : this.executors.globals).get(name);
if (!tool) throw new Error(`Unknown ${isTool ? "tool" : "global"} "${name}"`);
const args: unknown = message.args === undefined ? undefined : JSON.parse(message.args);
const value = await tool.execute(args, { signal: pending.controller.signal });
reply = { type: "result", id, ok: true, payload: value === undefined ? undefined : JSON.stringify(value) };
status = "ok";
} catch (error) {
reply = { type: "result", id, ok: false, payload: errorMessage(error) };
status = "error";
}
// Already cancelled by finish(): the record keeps "cancelled" and the
// VM is gone or going.
if (!this.pending.delete(id)) return;
if (record) {
record.status = status;
record.durationMs = performance.now() - pending.startedAt;
}
this.post(reply);
}
private finish(
error: CodemodeError | undefined,
value?: unknown,
writes?: CodemodeStoreWrites,
during?: CodemodeDuring,
): void {
if (this.finished) return;
this.finished = true;
clearTimeout(this.timer);
this.signal?.removeEventListener("abort", this.onAbort);
const now = performance.now();
for (const pending of this.pending.values()) {
if (pending.record) pending.record.durationMs = now - pending.startedAt;
pending.controller.abort();
}
this.pending.clear();
this.unsent = [];
const ended = during === undefined ? {} : { during: { seq: during.seq, issued: during.issued } };
const result: CodemodeResult = error
? { ok: false, error, output: this.output, calls: this.calls, ...ended }
: {
ok: true,
value,
output: this.output,
calls: this.calls,
storeWrites: writes ?? { set: {}, delete: [] },
...ended,
};
if (!this.channel) {
this.resolveResult(result);
return;
}
this.channel
.stop()
.catch(() => undefined)
.then(() => this.resolveResult(result));
}
}
/**
* Tool table, defaults, and in-flight executions; the behavior every sandbox shares. Subclasses
* only choose where the VM runs by passing a {@link VmLauncher}.
*
* The script sees `tools.<name>(args)` for every registered tool, `ALL_TOOLS`, the output helpers
* `text`, `image`, `exit`, and `console.*`, `store`/`load`, and the configured globals; nothing
* else (no timers, `fetch`, `process`, `require`, modules). Each `execute()` gets its own VM; the
* sandbox only holds the tool table and defaults. `close()` aborts in-flight executions.
*/
export class CodemodeSandboxBase {
private readonly toolsByName = new Map<string, CodemodeTool>();
private readonly globalsByName = new Map<string, CodemodeTool>();
private readonly timeoutMs: number;
private readonly memoryLimitBytes: number | undefined;
private readonly launch: VmLauncher;
private readonly running = new Set<Execution>();
private closed = false;
constructor(options: CodemodeSandboxBaseOptions, launch: VmLauncher) {
this.timeoutMs = options.timeoutMs ?? DEFAULT_TIMEOUT_MS;
this.memoryLimitBytes = options.memoryLimitBytes;
this.launch = launch;
for (const tool of options.tools ?? []) this.registerTool(tool);
checkGlobals(options.globals ?? []);
for (const global of options.globals ?? []) this.globalsByName.set(global.name, global);
}
/** Throws if a tool with the same name is already registered. */
registerTool(tool: CodemodeTool): void {
if (this.toolsByName.has(tool.name)) throw new Error(`Tool "${tool.name}" is already registered`);
this.toolsByName.set(tool.name, tool);
}
unregisterTool(name: string): boolean {
return this.toolsByName.delete(name);
}
get tools(): CodemodeTool[] {
return [...this.toolsByName.values()];
}
get globals(): CodemodeTool[] {
return [...this.globalsByName.values()];
}
/**
* `code` is an async function body: `return` and top-level `await` work.
* Never rejects for script failures; those come back as `{ ok: false }`.
* The script can use `store(key, value)` and `load(key)` on `options.store`.
*/
execute(code: string, options: CodemodeExecuteOptions = {}): Promise<CodemodeResult> {
if (this.closed) return Promise.reject(new Error("Sandbox is closed"));
const tools = new Map(this.toolsByName);
const execution = new Execution({
code,
tools: [...tools.values()],
globals: [...this.globalsByName.values()],
executors: { tools, globals: this.globalsByName },
timeoutMs: options.timeoutMs ?? this.timeoutMs,
signal: options.signal,
memoryLimitBytes: this.memoryLimitBytes,
store: serializeStore(options.store),
launch: this.launch,
});
this.running.add(execution);
return execution.promise.finally(() => this.running.delete(execution));
}
/**
* Run `code` deterministically, with the caller answering its calls: the same script, start time, seed, tools,
* globals, and store, answered with the same results in the same `seq` order, makes the same calls with the same
* arguments and produces the same output, on any VM with the same fingerprint. A host can therefore replay it
* after a restart. The VM runs in UTC, its clock reads `startedAt` and then the `now` of the result being
* processed, its randomness comes from `seed`, and `WeakRef` and `FinalizationRegistry` do not exist.
*
* The sandbox's own tools and globals are not used: the run declares its own, and every call goes to
* `options.onCall`. Never rejects for script failures; abort through `options.signal` to end the run with an
* error the script cannot catch.
*/
start(code: string, options: CodemodeRunOptions): CodemodeRun {
if (this.closed) throw new Error("Sandbox is closed");
if (!Number.isFinite(options.startedAt) || options.startedAt < 0) {
throw new Error("startedAt must be a non-negative number of milliseconds since the epoch");
}
if (!Number.isInteger(options.seed) || options.seed < 0 || options.seed > 0xffffffff) {
throw new Error("seed must be a 32-bit unsigned integer");
}
const tools = options.tools ?? [];
if (new Set(tools.map((tool) => tool.name)).size !== tools.length) throw new Error("Tool names must be unique");
const scriptNames = tools.map((tool) => tool.jsName ?? toCodemodeIdentifier(tool.name));
if (new Set(scriptNames).size !== tools.length) throw new Error("Tool script names must be unique");
const globals = options.globals ?? [];
checkGlobals(globals);
const execution = new Execution({
code,
tools,
globals,
executors: { tools: new Map(), globals: new Map() },
timeoutMs: options.timeoutMs ?? Number.POSITIVE_INFINITY,
signal: options.signal,
memoryLimitBytes: this.memoryLimitBytes,
store: serializeStore(options.store),
deterministic: { startedAt: options.startedAt, seed: options.seed },
control: { onStarted: options.onStarted, onCall: options.onCall, onIdle: options.onIdle },
launch: this.launch,
});
this.running.add(execution);
const result = execution.promise.finally(() => this.running.delete(execution));
return {
result,
deliver: (seq, now, id, outcome) => execution.deliver(seq, now, id, outcome),
get finished() {
return execution.finished;
},
};
}
/** Aborts in-flight executions (they resolve with `kind: "aborted"`) and rejects new ones. */
async close(): Promise<void> {
this.closed = true;
await Promise.all([...this.running].map((execution) => execution.abort("Sandbox closed")));
}
}

View File

@@ -0,0 +1,88 @@
import { Worker } from "node:worker_threads";
import type { CodemodeSandboxOptions } from "../types.ts";
import { type CodemodeWasmModule, loadQuickJSWasm } from "../wasm.ts";
import { CodemodeSandboxBase, errorMessage, type VmLauncher } from "./execution.ts";
import type { WorkerData } from "./protocol.ts";
function defaultWorkerUrl(): URL {
// `.ts` when running from source (tests, tsx), `.js` from the published dist.
return new URL(import.meta.url.endsWith(".ts") ? "./worker.ts" : "./worker.js", import.meta.url);
}
/**
* Each execution gets its own worker thread and QuickJS VM. A fresh worker per run keeps
* termination simple: a runaway script, including one that only spins the microtask queue, is
* killed with `terminate()` and cannot poison a later run. Before terminating, the host sets a
* shared interrupt flag that the VM polls, because Bun's `terminate()` cannot stop a thread that
* is spinning in wasm. Stopping requests termination without waiting for exit: some built-ins do
* not poll the flag, so on Bun a worker can outlive the script's result.
*/
function workerLauncher(options: CodemodeSandboxOptions): VmLauncher {
return async (data, events) => {
let module: CodemodeWasmModule;
try {
module = await (options.wasm ?? loadQuickJSWasm());
} catch (error) {
throw new Error(`Failed to load QuickJS: ${errorMessage(error)}`);
}
const interrupt = new SharedArrayBuffer(4);
const workerData: WorkerData = { ...data, wasm: module, interrupt };
const releaseWorker = options.acquireWorker?.();
let worker: Worker;
try {
worker = new Worker(options.workerUrl ?? defaultWorkerUrl(), { workerData });
} catch (error) {
releaseWorker?.();
throw new Error(`Failed to start worker: ${errorMessage(error)}`);
}
if (releaseWorker) worker.once("exit", releaseWorker);
worker.on("message", (message: unknown) => {
// Node 24/26 --watch leaks dependency reports onto the worker channel (nodejs/node#65044).
if (
typeof message === "object" &&
message !== null &&
("watch:import" in message || "watch:require" in message)
)
return;
events.message(message);
});
worker.on("error", (error: unknown) => {
events.failure({
kind: "sandbox",
name: error instanceof Error ? error.name : undefined,
message: errorMessage(error),
});
});
worker.on("exit", (code) => {
events.failure({ kind: "sandbox", message: `Worker exited with code ${code} before the script settled` });
});
const stop = async (): Promise<void> => {
Atomics.store(new Int32Array(interrupt), 0, 1);
// A non-interruptible built-in can delay Bun's terminate() until it returns (#10748).
// Do not hold up the result while that worker exits.
void worker.terminate().catch(() => undefined);
// terminate() refs the worker internally; unref it after requesting termination.
worker.unref();
};
try {
options.onWorkerStarted?.(new Promise<void>((resolve) => worker.once("exit", () => resolve())));
} catch (error) {
void stop();
throw error;
}
return { post: (message) => worker.postMessage(message), stop };
};
}
/**
* Runs JavaScript in a QuickJS VM (a separate wasm instance) inside a worker thread. The worker
* keeps script execution off the host thread: QuickJS runs synchronously, so a spinning script on
* the host thread would block its event loop. Requires `node:worker_threads` (Node or Bun); see
* `InlineCodemodeSandbox` and `RemoteCodemodeSandbox` in `@earendil-works/pi-codemode/portable`
* for other runtimes.
*/
export class CodemodeSandbox extends CodemodeSandboxBase {
constructor(options: CodemodeSandboxOptions = {}) {
super(options, workerLauncher(options));
}
}

View File

@@ -0,0 +1,126 @@
/**
* Runs the VM on the calling thread, for runtimes without worker threads (Cloudflare Workers and
* Durable Objects, browsers) and for the server side of `RemoteCodemodeSandbox`.
*
* Problem: QuickJS runs synchronously. While a script computes, the host's event loop does not
* run, so neither the deadline timer nor an abort signal can fire. On Workers even `Date.now()` is
* frozen during synchronous execution, so the VM cannot check a clock either. Example: with a
* worker thread, `while (true) {}` is stopped by the host's 200 ms timer; on the calling thread
* that timer never gets a chance to run.
*
* Solution: an interrupt budget. QuickJS polls its interrupt handler periodically while a script
* runs; this launcher counts the polls across the whole execution and interrupts the script once
* the count passes the budget. The execution then fails with `kind: "timeout"`. The deadline and
* abort signal still work whenever the script waits on a tool, since the event loop runs then.
*
* The rate of polls depends on the code and the machine. Measured with a tight loop: about 50,000
* per second of CPU on an Apple M-series laptop and about 10,000 per second on Cloudflare Durable
* Objects (October 2026). The default of 100,000 allows roughly 2 to 10 seconds of uninterrupted
* computation; keep it well below the platform's own CPU limit, because hitting that limit resets
* a Durable Object instead of failing one script.
*
* Once the VM reports `done` or `crash`, the result is decided, so the VM is interrupted at the
* next poll instead of running on until the host stops it. Example: a script that keeps printing in
* a `for (;;)` loop after its output limit was reached would otherwise run until the budget ran out
* and be reported as a timeout.
*
* Stack: the VM runs on the caller's stack, which is smaller than a worker thread's (about 1 MB
* on Node's main thread), so QuickJS's stack guard is set to {@link INLINE_MAX_STACK_SIZE}.
*
* Reentrancy: VM callbacks run inside QuickJS. Messages from the VM are delivered to the host in
* a microtask, after the VM has returned, so host code (tool calls, `finish()`, disposal) never
* runs inside a VM callback and never re-enters the VM.
*/
import type { CodemodeSandboxBaseOptions } from "../types.ts";
import type { CodemodeWasmModule } from "../wasm.ts";
import { CodemodeSandboxBase, errorMessage, type VmLauncher } from "./execution.ts";
import { startVm } from "./vm.ts";
export const DEFAULT_INTERRUPT_BUDGET = 100_000;
/** Allows recursion about 1,000 calls deep and trips well before a 1 MB host stack overflows. */
export const INLINE_MAX_STACK_SIZE = 256 * 1024;
export interface InlineCodemodeSandboxOptions extends CodemodeSandboxBaseOptions {
/**
* Compiled `quickjs-wasi/quickjs.wasm`. On Cloudflare, import it with Wrangler's `CompiledWasm`
* rule (`@earendil-works/pi-codemode/cloudflare` does this); on Node, `loadQuickJSWasm()`.
*/
wasm: CodemodeWasmModule | Promise<CodemodeWasmModule>;
/**
* Interrupt polls an execution may use before it fails with `kind: "timeout"`. The only limit on
* a script that computes without awaiting a tool. Default: {@link DEFAULT_INTERRUPT_BUDGET}.
*/
interruptBudget?: number;
}
export function inlineLauncher(
wasm: CodemodeWasmModule | Promise<CodemodeWasmModule>,
interruptBudget: number = DEFAULT_INTERRUPT_BUDGET,
): VmLauncher {
return async (data, events) => {
let module: CodemodeWasmModule;
try {
module = await wasm;
} catch (error) {
throw new Error(`Failed to load QuickJS: ${errorMessage(error)}`);
}
let stopped = false;
let settled = false;
let polls = 0;
let exhausted = false;
const session = await startVm({
data,
wasm: module,
maxStackSize: INLINE_MAX_STACK_SIZE,
interruptBudget,
shouldInterrupt: () => {
if (stopped || settled || exhausted) return true;
if (++polls <= interruptBudget) return false;
exhausted = true;
return true;
},
post: (message) => {
const terminal = message.type === "done" || message.type === "crash";
// Only the first terminal message counts: a crash caused by interrupting a settled VM is noise.
if (terminal && settled) return;
if (terminal) settled = true;
queueMicrotask(() => {
if (stopped) return;
// The interrupt surfaces as a crash or a failed done; report what caused it instead.
if (exhausted && (message.type === "done" || message.type === "crash")) {
events.failure({
kind: "timeout",
message: `Execution exceeded its interrupt budget of ${interruptBudget}`,
});
return;
}
events.message(message);
});
},
});
return {
post: (message) => {
if (!stopped) session.deliver(message);
},
stop: async () => {
if (stopped) return;
stopped = true;
session.dispose();
},
};
};
}
/**
* Runs each execution's QuickJS VM on the calling thread. Works in any runtime with WebAssembly
* and microtasks; needs no worker threads, files, or Node built-ins. A script that computes
* blocks the caller's thread until it awaits a tool or uses up its interrupt budget, so prefer
* `CodemodeSandbox` where worker threads exist, and `RemoteCodemodeSandbox` to keep scripts off a
* thread that must stay responsive (such as a Durable Object that serves a session).
*/
export class InlineCodemodeSandbox extends CodemodeSandboxBase {
constructor(options: InlineCodemodeSandboxOptions) {
super(options, inlineLauncher(options.wasm, options.interruptBudget));
}
}

View File

@@ -0,0 +1,522 @@
/**
* JavaScript evaluated inside the QuickJS VM before the script runs.
*
* The VM is its own wasm instance, so nothing here guards a realm boundary.
* The prelude keeps the host bridge in a closure so the script cannot call it
* directly, and builds `tools`, `ALL_TOOLS`, the output helpers (`text`, `image`,
* `exit`, `console`), and globals on top of it. Tool arguments and results cross
* as JSON strings and are parsed on this side.
*
* Output helpers: `text(value)` appends a text item (non-strings are
* JSON-stringified), `image(urlOrItem)` appends an image from a base64 `data:` URL, an
* `{ image_url }` object, or an image block `{ type: "image", data, mimeType }`, and `exit()` ends the script
* successfully. `console.*` appends text items marked as console output.
*
* `store(key, value)` and `load(key)` are synchronous: they work on a snapshot of
* JSON text passed in as `storeJson`, and the keys the script wrote are reported
* with a successful "done".
*
* Evaluates to a function `(bridge, toolsJson, globalsJson, storeJson) => { settle, run, stalled }`.
* `settle(id, ok, payload, propertiesJson)` settles call `id`; a rejection's `Error` gets the own properties of the
* `propertiesJson` object, except `name`, `message`, and `stack`.
* `toolsJson` lists `{ name, jsName, description }`: `tools[jsName]` and `tools[name]` call the
* tool, and `ALL_TOOLS` lists `{ name: jsName, description }`.
* `stalled()` reports a script that has not finished while no host call is pending: with no timers
* or I/O in the VM, nothing can ever resume it.
* `globalsJson` lists `{ name, spread }`; `a.b` names are grouped into a frozen `a` object.
* `bridge(kind, a, b, c)` with kind "call" or "global" (id, name, argsJson),
* "output" ("text" or "console", text) or ("image", data, mimeType), or "done" (ok, valueJsonOrErrorJson, writesJson).
*
* Before anything else the prelude freezes the built-ins and makes the built-in globals read-only.
* The prelude shares them with the script, so a script that patched one (for example
* `Array.prototype.toJSON`) could otherwise corrupt what the prelude reports to the host.
*/
export const MAX_STORE_VALUE_CHARS = 256 * 1024;
export const MAX_STORE_TOTAL_CHARS = 1024 * 1024;
/**
* Output one script may produce with `text()`, `image()`, and `console.*`: characters of text and
* base64 image data, and items. The host keeps all output until the script ends, so without a
* limit a script that prints in a loop grows the host's memory until it crashes. The item limit
* covers loops that print empty strings.
*/
export const MAX_OUTPUT_CHARS = 16 * 1024 * 1024;
export const MAX_OUTPUT_ITEMS = 100_000;
const IMAGE_HELPER_EXPECTS =
'image expects a non-empty image URL string, an object with image_url, or an image block { type: "image", data, mimeType }';
export const PRELUDE_SOURCE: string = `(function (bridge, toolsJson, globalsJson, storeJson) {
"use strict";
// Freeze every object reachable from the built-in globals, plus the intrinsics that are only
// reachable from instances (iterator and generator prototypes, %TypedArray%).
//
// Freezing alone breaks ordinary code through the "override mistake": a data property that is
// read-only on a prototype cannot be assigned on an instance either, so
// this.name = "MyError" in an Error subclass would throw. Commonly overridden properties
// become accessors whose setter defines an own property on the instance instead.
(function lockdown() {
const OVERRIDABLE = new Set(["constructor", "name", "message", "toString", "toLocaleString", "valueOf", "toJSON"]);
const seen = new Set([globalThis]);
const queue = [];
const add = (value) => {
if ((typeof value === "object" && value !== null) || typeof value === "function") {
if (!seen.has(value)) {
seen.add(value);
queue.push(value);
}
}
};
// Accessors from an object literal have no own prototype object, unlike function expressions,
// whose prototype.constructor would be converted again without end.
function allowOverride(object, key, value, enumerable) {
const { get, set } = Object.getOwnPropertyDescriptor(
{
get accessor() {
return value;
},
set accessor(next) {
if (this === object) {
throw new TypeError("Cannot assign to read only property '" + String(key) + "' of a built-in");
}
if ((typeof this !== "object" || this === null) && typeof this !== "function") return;
Object.defineProperty(this, key, { value: next, writable: true, enumerable: true, configurable: true });
},
},
"accessor",
);
Object.defineProperty(object, key, { get, set, enumerable, configurable: false });
add(get);
add(set);
}
for (const key of Reflect.ownKeys(globalThis)) {
const descriptor = Object.getOwnPropertyDescriptor(globalThis, key);
add(descriptor.value);
add(descriptor.get);
add(descriptor.set);
if ("value" in descriptor && descriptor.configurable) {
Object.defineProperty(globalThis, key, { writable: false, configurable: false });
}
}
add(Object.getPrototypeOf(function* () {}));
add(Object.getPrototypeOf(async function () {}));
add(Object.getPrototypeOf(async function* () {}));
add(Object.getPrototypeOf(Int8Array));
add(Object.getPrototypeOf([][Symbol.iterator]()));
add(Object.getPrototypeOf(new Map()[Symbol.iterator]()));
add(Object.getPrototypeOf(new Set()[Symbol.iterator]()));
add(Object.getPrototypeOf(""[Symbol.iterator]()));
add(Object.getPrototypeOf(/a/[Symbol.matchAll]("")));
if (typeof Iterator === "function") {
if (typeof Iterator.prototype.map === "function") add(Object.getPrototypeOf([].values().map((x) => x)));
if (typeof Iterator.from === "function") add(Object.getPrototypeOf(Iterator.from({ next() {} })));
}
while (queue.length > 0) {
const object = queue.pop();
add(Object.getPrototypeOf(object));
const descriptors = Object.getOwnPropertyDescriptors(object);
for (const key of Reflect.ownKeys(descriptors)) {
const descriptor = descriptors[key];
if ("value" in descriptor) {
add(descriptor.value);
if (
descriptor.writable &&
descriptor.configurable &&
(object === Object.prototype || OVERRIDABLE.has(key))
) {
allowOverride(object, key, descriptor.value, descriptor.enumerable);
}
} else {
add(descriptor.get);
add(descriptor.set);
}
}
Object.freeze(object);
}
})();
const stringify = JSON.stringify;
const parse = JSON.parse;
const promiseThen = Promise.prototype.then;
const ErrorCtor = Error;
const TypeErrorCtor = TypeError;
const RangeErrorCtor = RangeError;
const pending = new Map();
let nextId = 1;
let finished = false;
// Thrown by exit() to unwind the script after it already reported success.
const EXIT = Object.freeze({});
function done(ok, payload, writes) {
if (finished) return;
finished = true;
bridge("done", ok, payload, writes);
}
function serialize(value) {
return value === undefined ? undefined : stringify(value);
}
// QuickJS stacks list frames only. Prefix "Name: message" like V8 so the
// text reads the same as a Node error, and drop this prelude's frames.
function errorText(error) {
const head = error.message ? error.name + ": " + error.message : String(error.name);
const frames =
typeof error.stack === "string"
? error.stack.split("\\n").filter((line) => line.trim() && !line.includes("codemode-prelude.js"))
: [];
return [head, ...frames].join("\\n");
}
function format(value) {
if (typeof value === "string") return value;
if (value instanceof ErrorCtor) return errorText(value);
try {
const json = stringify(value);
return json === undefined ? String(value) : json;
} catch {
return String(value);
}
}
// The host requires string fields. A script can set an error's name or message to anything, so
// they are coerced, and a value whose coercion throws gets a placeholder.
function describeError(error) {
try {
if (error instanceof ErrorCtor) {
return stringify({ name: String(error.name), message: String(error.message), stack: errorText(error) });
}
return stringify({ message: String(format(error)) });
} catch {
return stringify({ message: "The script threw a value that cannot be described" });
}
}
function caller(kind, name, spread) {
return (...args) =>
new Promise((resolve, reject) => {
let json;
try {
json = serialize(spread ? args : args[0]);
} catch (error) {
reject(error);
return;
}
const id = nextId++;
pending.set(id, { resolve, reject });
bridge(kind, id, name, json);
});
}
const tools = Object.create(null);
const allTools = [];
for (const { name, jsName, description } of parse(toolsJson)) {
const fn = caller("call", name);
// The first tool wins when two names normalize to the same identifier.
if (!(jsName in tools)) {
tools[jsName] = fn;
allTools.push(Object.freeze({ name: jsName, description }));
}
if (!(name in tools)) tools[name] = fn;
}
Object.freeze(tools);
Object.freeze(allTools);
// Reading a member that does not exist throws an error that names the close matches, instead of
// a later "not a function". \`in\` checks still work.
const comparable = (name) => name.toLowerCase().replace(/[^a-z0-9]/g, "");
function guard(target, label, names, hint) {
return new Proxy(target, {
get(object, property, receiver) {
if (typeof property !== "string" || property in object || property in Object.prototype || property === "then" || property === "toJSON") {
return Reflect.get(object, property, receiver);
}
const wanted = comparable(property);
const exact = names.filter((name) => comparable(name) === wanted);
const close = exact.length > 0 ? exact : names.filter((name) => wanted && (comparable(name).includes(wanted) || wanted.includes(comparable(name))));
let message = label + "." + property + " does not exist.";
if (close.length > 0) message += " Did you mean " + close.slice(0, 5).map((name) => label + "." + name).join(", ") + "?";
else if (names.length <= 20) message += " Available: " + names.join(", ") + ".";
if (hint) message += " " + hint;
message += ' Check for a member with "' + property + '" in ' + label + ".";
throw new TypeErrorCtor(message);
},
});
}
const toolsProxy = guard(
tools,
"tools",
allTools.map((tool) => tool.name),
"ALL_TOOLS lists every tool; searchTools(query) finds tools by topic.",
);
const namespaces = new Map();
for (const { name, spread } of parse(globalsJson)) {
const fn = caller("global", name, spread);
const dot = name.indexOf(".");
if (dot === -1) {
Object.defineProperty(globalThis, name, { value: fn, enumerable: true });
continue;
}
const namespace = name.slice(0, dot);
if (!namespaces.has(namespace)) namespaces.set(namespace, Object.create(null));
namespaces.get(namespace)[name.slice(dot + 1)] = fn;
}
for (const [namespace, members] of namespaces) {
Object.freeze(members);
const value = guard(members, namespace, Object.keys(members));
Object.defineProperty(globalThis, namespace, { value, enumerable: true });
}
// key -> JSON text. Sizes count key and JSON characters.
const stored = new Map(Object.entries(parse(storeJson)));
const writes = new Map();
let storedChars = 0;
for (const [key, json] of stored) storedChars += key.length + json.length;
const STORE_HINT =
"store() is for small state such as IDs or summaries. Show images with image(), keep large data in variables, or write it to a file with a tool.";
function checkKey(name, key) {
if (typeof key !== "string") throw new TypeError(name + "() key must be a string");
}
function store(key, value) {
checkKey("store", key);
const previous = stored.has(key) ? key.length + stored.get(key).length : 0;
if (value === undefined) {
stored.delete(key);
storedChars -= previous;
writes.set(key, undefined);
return;
}
let json;
try {
json = stringify(value);
} catch (error) {
throw new TypeError("store(" + stringify(key) + ") value is not JSON-serializable: " + format(error));
}
if (json === undefined) {
throw new TypeError("store(" + stringify(key) + ") value is not JSON-serializable");
}
if (json.length > ${MAX_STORE_VALUE_CHARS}) {
throw new RangeError(
"store(" + stringify(key) + ") value has " + json.length + " characters of JSON, more than the limit of ${MAX_STORE_VALUE_CHARS}. " +
STORE_HINT,
);
}
const next = storedChars - previous + key.length + json.length;
if (next > ${MAX_STORE_TOTAL_CHARS}) {
throw new RangeError(
"store is full: stored values would exceed ${MAX_STORE_TOTAL_CHARS} characters of JSON. Delete keys with store(key, undefined). " +
STORE_HINT,
);
}
stored.set(key, json);
storedChars = next;
writes.set(key, json);
}
function load(key) {
checkKey("load", key);
const json = stored.get(key);
return json === undefined ? undefined : parse(json);
}
function serializeWrites() {
const entries = [];
for (const [key, json] of writes) entries.push(json === undefined ? [key] : [key, json]);
return stringify(entries);
}
Object.defineProperty(globalThis, "store", { value: store, enumerable: true });
Object.defineProperty(globalThis, "load", { value: load, enumerable: true });
let outputChars = 0;
let outputItems = 0;
// Past the output limits the script fails: done() reports the error, so catching it does not
// resume output, and the host ends the script.
function output(kind, data, mimeType) {
if (finished) return;
outputChars += data.length;
outputItems++;
if (outputChars > ${MAX_OUTPUT_CHARS} || outputItems > ${MAX_OUTPUT_ITEMS}) {
const error = new RangeErrorCtor(
"script output exceeded the limit of ${MAX_OUTPUT_CHARS} characters or ${MAX_OUTPUT_ITEMS} text(), image(), and console calls. " +
"Print a summary instead, or write large data to a file with a tool.",
);
done(false, describeError(error));
throw error;
}
bridge("output", kind, data, mimeType);
}
// Primitives become their string form, everything else JSON.
function outputText(value) {
if (value === undefined || value === null || typeof value !== "object" && typeof value !== "function") {
return String(value);
}
const json = stringify(value);
return json === undefined ? String(value) : json;
}
function text(value) {
let rendered;
try {
rendered = outputText(value);
} catch (error) {
throw new TypeErrorCtor(error instanceof ErrorCtor ? error.message : String(error));
}
output("text", rendered);
}
function imageUrl(value) {
if (typeof value === "string") return value;
if (typeof value !== "object" || value === null || Array.isArray(value)) {
throw new TypeErrorCtor(${JSON.stringify(IMAGE_HELPER_EXPECTS)});
}
if (value.image_url !== undefined) {
if (typeof value.image_url !== "string") throw new TypeErrorCtor(${JSON.stringify(IMAGE_HELPER_EXPECTS)});
return value.image_url;
}
if (typeof value.type !== "string") throw new TypeErrorCtor(${JSON.stringify(IMAGE_HELPER_EXPECTS)});
if (value.type !== "image") {
throw new TypeErrorCtor('image only accepts image blocks, got "' + value.type + '"');
}
if (typeof value.data !== "string" || value.data === "") throw new TypeErrorCtor("image expected image block data");
if (value.data.toLowerCase().startsWith("data:")) return value.data;
return "data:;base64," + value.data;
}
// Base64 of the signatures of the formats providers accept inline (PNG, JPEG except
// JPEG-LS, GIF, "RIFF....WEBP"). Signatures start at byte 0, so their encodings are prefixes.
const IMAGE_SIGNATURES = [
["image/png", /^iVBORw0KGg/],
["image/jpeg", /^[/]9j[/](?!9)/],
["image/gif", /^R0lGOD[dl]h/],
["image/webp", /^UklG.{8}RUJQ/],
];
function image(value) {
const url = imageUrl(value);
if (url === "") throw new TypeErrorCtor(${JSON.stringify(IMAGE_HELPER_EXPECTS)});
const colon = url.indexOf(":");
const scheme = colon === -1 ? "" : url.slice(0, colon).toLowerCase();
if (scheme === "http" || scheme === "https") {
throw new TypeErrorCtor("remote image URLs are not supported in tool outputs. Pass a base64 data URI instead");
}
const comma = url.indexOf(",");
const header = comma === -1 ? [] : url.slice(colon + 1, comma).split(";");
if (scheme !== "data" || comma === -1 || header.slice(1).every((part) => part.toLowerCase() !== "base64")) {
throw new TypeErrorCtor("invalid image output. Pass a base64 data URI instead");
}
// Providers reject the whole request on a bad image, and a persisted image block would be
// resent on every later turn. Line breaks from wrapped base64 are dropped. The declared type
// is ignored in favor of the detected one, as providers also reject mismatches.
const data = url.slice(comma + 1).replace(/\\s+/g, "");
if (data.length % 4 !== 0 || !/^[A-Za-z0-9+/]+={0,2}$/.test(data)) {
throw new TypeErrorCtor("invalid image output. The image data is not valid base64 (truncated or corrupted?)");
}
const head = data.slice(0, 16);
const signature = IMAGE_SIGNATURES.find(([, pattern]) => pattern.test(head));
if (!signature) {
throw new TypeErrorCtor("invalid image output. The image data is not a PNG, JPEG, GIF, or WebP image");
}
output("image", data, signature[0]);
}
function exit() {
let writesJson;
try {
writesJson = serializeWrites();
} catch (error) {
done(false, describeError(error));
throw EXIT;
}
done(true, undefined, writesJson);
throw EXIT;
}
const console = {};
for (const level of ["log", "info", "warn", "error", "debug"]) {
console[level] = (...args) => {
output("console", args.map(format).join(" "));
};
}
Object.freeze(console);
Object.defineProperty(globalThis, "tools", { value: toolsProxy, enumerable: true });
Object.defineProperty(globalThis, "ALL_TOOLS", { value: allTools, enumerable: true });
Object.defineProperty(globalThis, "console", { value: console, enumerable: true });
Object.defineProperty(globalThis, "text", { value: text, enumerable: true });
Object.defineProperty(globalThis, "image", { value: image, enumerable: true });
Object.defineProperty(globalThis, "exit", { value: exit, enumerable: true });
return {
settle(id, ok, payload, propertiesJson) {
const entry = pending.get(id);
if (!entry) return;
pending.delete(id);
if (!ok) {
const error = new ErrorCtor(payload);
if (propertiesJson !== undefined) {
const properties = parse(propertiesJson);
for (const key of Object.keys(properties)) {
if (key === "name" || key === "message" || key === "stack") continue;
Object.defineProperty(error, key, { value: properties[key], writable: true, enumerable: true, configurable: true });
}
}
entry.reject(error);
return;
}
let value;
try {
value = payload === undefined ? undefined : parse(payload);
} catch (error) {
entry.reject(error);
return;
}
entry.resolve(value);
},
run(fn) {
let promise;
try {
promise = fn(toolsProxy, console);
} catch (error) {
done(false, describeError(error));
return;
}
promiseThen.call(
promise,
(value) => {
let json;
try {
json = serialize(value);
} catch (error) {
done(false, describeError(error));
return;
}
done(true, json, serializeWrites());
},
(error) => {
done(false, describeError(error));
},
);
},
stalled() {
if (finished || pending.size > 0) return false;
done(
false,
stringify({
name: "Error",
message:
"The script is waiting on a promise that can never settle: no tool call is pending, and timers do not exist here.",
}),
);
return true;
},
};
})`;

View File

@@ -0,0 +1,93 @@
import type { CodemodeDuring, CodemodeOutputItem } from "../types.ts";
import type { CodemodeWasmModule } from "../wasm.ts";
/**
* Messages between the host (main thread) and the worker. Tool arguments,
* results, and values cross as JSON strings: the worker passes them into and
* out of the QuickJS VM as strings and never builds structured values itself.
*/
/**
* Everything a VM needs to run one script, independent of where it runs. Plain data, so it can
* cross a `postMessage` or a Workers RPC call unchanged.
*/
export interface VmStartData {
code: string;
/** `jsName` is the identifier the script uses; `description` is listed in `ALL_TOOLS`. */
tools: { name: string; jsName: string; description: string }[];
globals: { name: string; spread: boolean }[];
memoryLimitBytes: number | undefined;
/** Snapshot for `load()`: key to JSON text. */
store: Record<string, string>;
/**
* Run deterministically (`CodemodeSandboxBase.start()`): UTC, a virtual clock starting at `startedAt` (ms since
* the epoch) that moves only with delivered results, randomness from `seed`, no `WeakRef`, results applied in
* `seq` order, and `started`/`idle` messages.
*/
deterministic?: { startedAt: number; seed: number };
}
export interface WorkerData extends VmStartData {
/** Compiled `quickjs-wasi` module. Structured clone shares the compiled code with the worker. */
wasm: CodemodeWasmModule;
/**
* One Int32 the host sets to non-zero before terminating the worker. The VM's interrupt handler
* polls it, because Bun's `worker.terminate()` cannot stop a thread that is spinning in wasm.
*/
interrupt: SharedArrayBuffer;
}
/** JSON-encoded `{ name?, message, stack? }` of an error thrown by the script. */
export type ScriptErrorJson = string;
export type WorkerToHostMessage =
| { type: "call"; id: number; target: "tool" | "global"; name: string; args: string | undefined }
| { type: "output"; item: CodemodeOutputItem }
/**
* `writes` is a JSON array of `[key, json]` for `store()` and `[key]` for deletions. Deterministic mode: `during`
* is the result the VM was processing when the script finished (0: its first run) and the calls issued by then.
*/
| { type: "done"; ok: true; value: string | undefined; writes: string; during?: CodemodeDuring }
| { type: "done"; ok: false; error: ScriptErrorJson; during?: CodemodeDuring }
/** The VM failed outside the script's control, for example a wasm trap. */
| { type: "crash"; message: string }
/** Deterministic mode, before the script runs: what the VM is, so a replay can refuse a different one. */
| { type: "started"; fingerprint: string }
/**
* Deterministic mode, after the script waits again: having processed result `seq` (0: the script's first run), it
* has issued `issued` calls in total.
*/
| { type: "idle"; seq: number; issued: number };
export type HostToWorkerMessage =
/**
* `payload` is the JSON result when `ok`, otherwise the error message. Deterministic mode: `seq` numbers the
* results from 1 in the order the VM applies them, and the VM's clock reads `now` while it processes this one.
*/
{
type: "result";
id: number;
ok: boolean;
payload: string | undefined;
seq?: number;
now?: number;
/** Deterministic mode, `ok: false`: a JSON object whose properties the script's `Error` gets. */
properties?: string;
};
export function isWorkerToHostMessage(value: unknown): value is WorkerToHostMessage {
if (typeof value !== "object" || value === null) return false;
const type = (value as { type?: unknown }).type;
return (
type === "call" ||
type === "output" ||
type === "done" ||
type === "crash" ||
type === "started" ||
type === "idle"
);
}
export function isHostToWorkerMessage(value: unknown): value is HostToWorkerMessage {
return typeof value === "object" && value !== null && (value as { type?: unknown }).type === "result";
}

View File

@@ -0,0 +1,277 @@
/**
* Runs the VM in another isolate, reached over an RPC boundary such as a Cloudflare Durable Object
* stub, while tools keep running in the caller.
*
* Problem: a VM on the calling thread blocks that thread while the script computes. In a Durable
* Object that serves an agent session, a CPU-heavy script would stall every other request to the
* session (including the user's abort) for seconds. Measured on Cloudflare (October 2026), a
* script that computed for 10 seconds delayed the session's other requests by 10 seconds when it
* ran in the session object or in a Dynamic Worker, but by about 50 ms when it ran in a Durable
* Object of a separately deployed Worker.
*
* Solution: the caller keeps the host role (`RemoteCodemodeSandbox`: tools, call records,
* deadline, abort) and the remote side only runs the VM (`serveCodemodeRemote`, behind
* `CodemodeSandboxDurableObject` in `@earendil-works/pi-codemode/cloudflare`).
*
* Protocol, one RPC call per execution:
*
* client remote
* remote.execute(request, exchange) ───────▶ starts the VM (in-thread, with interrupt budget)
* ◀─────── exchange([...outputs, call]) one call per tool call
* runs the tool, replies with result ──────▶ VM continues
* ◀─────── exchange([...outputs, done]) terminal, reply ignored
* execute() resolves ◀────────────────────── VM disposed
*
* `exchange` is a function the client passes in; over Workers RPC it becomes a stub that calls back
* into the client. Output is buffered on the remote side and sent with the next call or the
* terminal message, so a script that prints in a loop does not make one RPC per line. Each
* `exchange` carries at most one call, so concurrent tool calls are concurrent RPCs. Calls on one
* stub are delivered in order, which keeps output ordered before the call or result that follows.
*
* Termination: the client cannot interrupt a remote VM that is computing. When the client finishes
* first (deadline, abort, `close()`), it answers every pending and later `exchange` with
* `{ type: "stop" }`, and the remote stops the VM at that point. A script that computes without
* calling a tool runs until its interrupt budget is used up. The remote reports that as a
* `failure` message, which the client turns into the same `kind: "timeout"` error an in-thread
* VM produces.
*
* Failure: if the RPC call rejects (the remote isolate crashed, hit its CPU limit, or was
* redeployed) or returns without a terminal message, the execution fails with `kind: "sandbox"`.
*/
import type { CodemodeError, CodemodeSandboxBaseOptions } from "../types.ts";
import type { CodemodeWasmModule } from "../wasm.ts";
import { CodemodeSandboxBase, errorMessage, type VmChannel, type VmLauncher } from "./execution.ts";
import { DEFAULT_INTERRUPT_BUDGET, inlineLauncher } from "./inline.ts";
import {
type HostToWorkerMessage,
isWorkerToHostMessage,
type VmStartData,
type WorkerToHostMessage,
} from "./protocol.ts";
export interface CodemodeRemoteRequest {
data: VmStartData;
/** Requested interrupt budget. The remote may lower it, see {@link CodemodeServeOptions}. */
interruptBudget?: number;
}
/** What the remote sends to the client: VM messages, or a failure outside the VM bridge. */
export type CodemodeRemoteMessage = WorkerToHostMessage | { type: "failure"; error: CodemodeError };
/**
* The client's answer to an exchange: the result of the call the exchange carried, `null` when it
* carried no call, or `stop` once the client no longer wants the execution.
*/
export type CodemodeRemoteReply = HostToWorkerMessage | { type: "stop" } | null;
export type CodemodeRemoteExchange = (messages: CodemodeRemoteMessage[]) => Promise<CodemodeRemoteReply>;
/**
* The remote end, usually a Durable Object stub for `CodemodeSandboxDurableObject`. Anything with
* this method works, for example an in-process object in tests.
*/
export interface CodemodeRemote {
execute(request: CodemodeRemoteRequest, exchange: CodemodeRemoteExchange): Promise<void>;
}
export interface RemoteCodemodeSandboxOptions extends CodemodeSandboxBaseOptions {
/**
* Returns the remote that runs one execution, called once per `execute()`. For a Durable Object,
* pick the instance here, for example `() => env.CODEMODE_SANDBOX.getByName(sessionId)`.
*/
remote: () => CodemodeRemote;
/** Interrupt budget requested from the remote. Default: the remote's default. */
interruptBudget?: number;
}
/** Output buffered on the remote before it is flushed without waiting for a call. */
const FLUSH_ITEMS = 1_000;
const FLUSH_CHARS = 1 << 20;
export interface CodemodeServeOptions {
wasm: CodemodeWasmModule | Promise<CodemodeWasmModule>;
/** Upper bound for a request's interrupt budget, and the budget when a request names none. */
maxInterruptBudget?: number;
/** Upper bound for a request's VM memory, and the limit when a request names none. */
maxMemoryLimitBytes?: number;
}
/**
* Remote side of {@link RemoteCodemodeSandbox}: runs one execution's VM on this thread and
* forwards everything it reports through `exchange`. Resolves once the VM has stopped and the
* terminal message has been delivered, or the client asked to stop. Never rejects.
*/
export function serveCodemodeRemote(
request: CodemodeRemoteRequest,
exchange: CodemodeRemoteExchange,
options: CodemodeServeOptions,
): Promise<void> {
const maxBudget = options.maxInterruptBudget ?? DEFAULT_INTERRUPT_BUDGET;
const budget = Math.min(request.interruptBudget ?? maxBudget, maxBudget);
const requestedMemory = request.data.memoryLimitBytes;
const maxMemory = options.maxMemoryLimitBytes;
const memoryLimitBytes =
requestedMemory === undefined
? maxMemory
: maxMemory === undefined
? requestedMemory
: Math.min(requestedMemory, maxMemory);
const data: VmStartData = { ...request.data, memoryLimitBytes };
return new Promise<void>((resolve) => {
let channel: VmChannel | undefined;
let ended = false;
let buffered: CodemodeRemoteMessage[] = [];
let bufferedChars = 0;
/** Replies that arrived before the launcher returned the channel. */
const unsent: HostToWorkerMessage[] = [];
const takeBuffer = (): CodemodeRemoteMessage[] => {
const batch = buffered;
buffered = [];
bufferedChars = 0;
return batch;
};
const end = (terminal: CodemodeRemoteMessage | undefined): void => {
if (ended) return;
ended = true;
const batch = takeBuffer();
const delivered = terminal ? exchange([...batch, terminal]).catch(() => null) : Promise.resolve();
void delivered
.then(() => channel?.stop())
.catch(() => undefined)
.then(() => resolve());
};
const reply = (message: CodemodeRemoteReply): void => {
if (ended || message === null) return;
if (message.type === "stop") {
end(undefined);
return;
}
if (channel) channel.post(message);
else unsent.push(message);
};
const onMessage = (message: unknown): void => {
if (ended) return;
const typed = message as WorkerToHostMessage;
switch (typed.type) {
case "output": {
buffered.push(typed);
bufferedChars += typed.item.type === "text" ? typed.item.text.length : typed.item.data.length;
if (buffered.length >= FLUSH_ITEMS || bufferedChars >= FLUSH_CHARS) {
exchange(takeBuffer()).then(reply, () => end(undefined));
}
break;
}
case "started":
buffered.push(typed);
break;
// Sent at once: a host replaying a script waits for it to see that the script issued no further call.
case "call":
case "idle":
exchange([...takeBuffer(), typed]).then(reply, () => end(undefined));
break;
default:
end(typed);
}
};
const launch = inlineLauncher(options.wasm, budget);
Promise.resolve(
launch(data, {
message: onMessage,
failure: (error) => end({ type: "failure", error }),
}),
).then(
(started) => {
channel = started;
if (ended) {
void started.stop().catch(() => undefined);
return;
}
for (const message of unsent.splice(0)) started.post(message);
},
(error: unknown) => end({ type: "failure", error: { kind: "sandbox", message: errorMessage(error) } }),
);
});
}
function remoteLauncher(remote: () => CodemodeRemote, interruptBudget: number | undefined): VmLauncher {
return (data, events) => {
let stopped = false;
let settled = false;
const waiting = new Map<number, (reply: CodemodeRemoteReply) => void>();
const exchange: CodemodeRemoteExchange = (messages) =>
new Promise<CodemodeRemoteReply>((resolve) => {
if (stopped || !Array.isArray(messages)) {
resolve({ type: "stop" });
return;
}
// Register the waiter first: the host may answer a call synchronously, for example when the
// tool throws before returning a promise.
const call = messages.find((message) => isWorkerToHostMessage(message) && message.type === "call");
if (call?.type === "call") waiting.set(call.id, resolve);
else resolve(null);
for (const message of messages) {
if (typeof message === "object" && message !== null && message.type === "failure") {
settled = true;
events.failure(message.error);
continue;
}
// Malformed messages still go to the host, which fails the execution as a broken bridge.
events.message(message);
if (isWorkerToHostMessage(message) && (message.type === "done" || message.type === "crash"))
settled = true;
}
});
let target: CodemodeRemote;
try {
target = remote();
} catch (error) {
throw new Error(`Failed to reach the remote sandbox: ${errorMessage(error)}`);
}
const request: CodemodeRemoteRequest = interruptBudget === undefined ? { data } : { data, interruptBudget };
Promise.resolve()
.then(() => target.execute(request, exchange))
.then(
() => {
if (!settled && !stopped) {
events.failure({ kind: "sandbox", message: "Remote sandbox returned before the script settled" });
}
},
(error: unknown) => {
if (!stopped)
events.failure({ kind: "sandbox", message: `Remote sandbox failed: ${errorMessage(error)}` });
},
);
return {
post: (message) => {
const resolve = waiting.get(message.id);
if (!resolve) return;
waiting.delete(message.id);
resolve(message);
},
stop: async () => {
stopped = true;
for (const resolve of waiting.values()) resolve({ type: "stop" });
waiting.clear();
},
};
};
}
/**
* Runs each execution's VM on a remote, usually a Cloudflare Durable Object deployed with
* `CodemodeSandboxDurableObject`, while tools run here. Same API and results as
* `CodemodeSandbox`, so it can replace it. Differences: a script that computes without calling a
* tool is limited by the remote's interrupt budget rather than `timeoutMs`, and after a deadline or
* abort the remote VM keeps computing until it next calls a tool or uses up that budget.
*/
export class RemoteCodemodeSandbox extends CodemodeSandboxBase {
constructor(options: RemoteCodemodeSandboxOptions) {
super(options, remoteLauncher(options.remote, options.interruptBudget));
}
}

View File

@@ -0,0 +1,341 @@
/**
* Drives one QuickJS VM through one script, independent of the transport that connects it to the
* host. The worker thread, the in-thread launcher, and the remote server all run the VM through
* this module, so the bridge between prelude and host exists once.
*
* Contract with the caller:
* - `post` receives every message for the host, synchronously from inside VM callbacks. The
* caller must not re-enter the VM from `post` (no `deliver`, no `dispose`); a worker posts to
* another thread, and the in-thread launcher defers delivery to a microtask.
* - `deliver` hands the VM a tool result and runs the script until it waits again. It must only be
* called while the VM is idle, which holds for any caller that never re-enters from `post`.
* - `shouldInterrupt` is polled by QuickJS while the script runs, and here between batches of
* promise jobs. Returning true throws an uncatchable error in the script; the VM then reports
* `crash` or a failed `done`. The job batches matter for a script such as `while (true) await
* null`: every iteration is a separate job that executes only a few opcodes, so QuickJS's own
* polling, which counts opcodes, would let it run for a long time between polls.
* - The VM never decides on its own when it is finished: after `done` or `crash` it stays alive
* until the caller disposes it (or terminates the worker that hosts it).
*
* Failures after the VM exists are reported as a `crash` message; `startVm` only rejects when the
* VM cannot be created at all.
*
* Deterministic mode (`data.deterministic`, used by `CodemodeSandboxBase.start()`): a script must make the same
* calls with the same arguments every time it runs with the same results, so a host can replay it after a restart.
* The VM is otherwise deterministic, because it only runs when the host delivers something and has no timers.
* What remains is fixed here:
* - Time: every WASI clock reads virtual time, `startedAt` and then the `now` of the result being processed, never going backwards.
* While the context is created, the realtime clock reads a value derived from the seed instead, because QuickJS
* seeds `Math.random()` from the first clock read, and which read comes first is the engine's business.
* - Randomness: `random_get` draws from a PRNG over the seed.
* - Timezone: UTC, not the host's.
* - Garbage collection: `WeakRef` and `FinalizationRegistry` are not available; collection timing is not.
* - Delivery order: results carry a `seq` and are applied strictly in that order, buffering early ones, because a
* transport (RPC replies) may deliver them in another order than the host sent them.
* The VM reports `started` with its fingerprint before the script runs, and `idle` with the number of calls issued
* after each drain, so the host can check a replay step by step.
*/
import { Intrinsics, JSException, type JSValueHandle, MAX_STACK_SIZE, QuickJS } from "quickjs-wasi";
import type { CodemodeWasmModule } from "../wasm.ts";
import { PRELUDE_SOURCE } from "./prelude-source.ts";
import type { HostToWorkerMessage, VmStartData, WorkerToHostMessage } from "./protocol.ts";
export interface VmSession {
deliver(message: HostToWorkerMessage): void;
/** Releases the VM's wasm instance. The session must not be used afterwards. */
dispose(): void;
}
export interface StartVmOptions {
data: VmStartData;
wasm: CodemodeWasmModule;
/**
* QuickJS's guard on its own stack, which turns deep recursion into a catchable `RangeError`.
* The wasm frames also use the host thread's native stack, so the guard must trip before that
* stack runs out: a host overflow fails the whole execution instead. Default: `MAX_STACK_SIZE`
* (512 KiB), which fits the 4 MB stack of a Node worker thread but not a 1 MB main thread.
*/
maxStackSize?: number;
/** Interrupt polls this VM may use, if the launcher counts them; part of the fingerprint. */
interruptBudget?: number;
post(message: WorkerToHostMessage): void;
shouldInterrupt(): boolean;
}
/**
* Bumped whenever this module or the prelude changes what a deterministic script observes, beyond what the
* fingerprint's other parts (engine versions, prelude hash, limits) already capture.
*/
const DETERMINISTIC_VERSION = 1;
/** FNV-1a over UTF-16 code units, as 8 hex digits: identifies the prelude in the fingerprint. */
function fnv1a(text: string): string {
let hash = 0x811c9dc5;
for (let index = 0; index < text.length; index++) {
hash ^= text.charCodeAt(index);
hash = Math.imul(hash, 0x01000193);
}
return (hash >>> 0).toString(16).padStart(8, "0");
}
const PRELUDE_HASH = fnv1a(PRELUDE_SOURCE);
/** mulberry32: a small, fast PRNG; one 32-bit draw per call. */
function prng(seed: number): () => number {
let state = seed >>> 0;
return () => {
state = (state + 0x6d2b79f5) >>> 0;
let value = Math.imul(state ^ (state >>> 15), 1 | state);
value = (value + Math.imul(value ^ (value >>> 7), 61 | value)) ^ value;
return (value ^ (value >>> 14)) >>> 0;
};
}
/** The deterministic VM's clock: seed-derived while the context is created, then virtual time. */
interface VirtualClock {
creating: boolean;
startedAt: number;
now: number;
seed: number;
}
/** WASI clock and random functions for deterministic mode, on top of {@link discardOutput}. */
function deterministicWasi(clock: VirtualClock) {
const random = prng(clock.seed ^ 0x9e3779b9);
return (memory: { readonly buffer: ArrayBufferLike }) => ({
...discardOutput(memory),
clock_time_get(clockId: number, _precision: bigint, resultPtr: number): number {
let nanoseconds: bigint;
// QuickJS seeds Math.random() from this reading in microseconds and turns 0 into 1, so seed s reads s + 1.
if (clock.creating) nanoseconds = clockId === 0 ? BigInt(clock.seed + 1) * 1000n : 0n;
else if (clockId === 0) nanoseconds = BigInt(Math.round(clock.now)) * 1_000_000n;
else nanoseconds = BigInt(Math.round(clock.now - clock.startedAt)) * 1_000_000n;
new DataView(memory.buffer).setBigUint64(resultPtr, nanoseconds, true);
return 0;
},
random_get(bufPtr: number, bufLen: number): number {
const bytes = new Uint8Array(memory.buffer, bufPtr, bufLen);
for (let index = 0; index < bufLen; index += 4) {
const value = random();
for (let byte = 0; byte < 4 && index + byte < bufLen; byte++) bytes[index + byte] = value >>> (byte * 8);
}
return 0;
},
});
}
/**
* QuickJS writes engine diagnostics to fd 1 and 2, which the default shim
* forwards to the host's stdout and stderr. That output belongs to the host
* application (for example a TUI), so it is discarded. Reporting every byte as
* written keeps libc from retrying.
*/
function discardOutput(memory: { readonly buffer: ArrayBufferLike }) {
return {
fd_write(_fd: number, iovsPtr: number, iovsLen: number, nwrittenPtr: number): number {
const view = new DataView(memory.buffer);
let written = 0;
for (let i = 0; i < iovsLen; i++) {
written += view.getUint32(iovsPtr + i * 8 + 4, true);
}
view.setUint32(nwrittenPtr, written, true);
return 0;
},
};
}
function describeException(error: JSException): string {
const head = error.message ? `${error.name}: ${error.message}` : error.name;
const stack = error.stack?.trimEnd();
return JSON.stringify({ name: error.name, message: error.message, stack: stack ? `${head}\n${stack}` : head });
}
/** Promise jobs run between two `shouldInterrupt` polls while draining the job queue. */
const JOBS_PER_POLL = 50;
function crashMessage(error: unknown): WorkerToHostMessage {
return { type: "crash", message: error instanceof Error ? `${error.name}: ${error.message}` : String(error) };
}
export async function startVm(options: StartVmOptions): Promise<VmSession> {
const { data, post } = options;
const maxStackSize = options.maxStackSize ?? MAX_STACK_SIZE;
const deterministic = data.deterministic;
const clock: VirtualClock | undefined = deterministic && {
creating: true,
startedAt: deterministic.startedAt,
now: deterministic.startedAt,
seed: deterministic.seed >>> 0,
};
const vm = await QuickJS.create({
wasm: options.wasm,
memoryLimit: data.memoryLimitBytes,
// Without a guard, deep recursion overflows the wasm stack and traps instead of throwing a
// catchable RangeError.
maxStackSize,
interruptHandler: options.shouldInterrupt,
wasi: clock ? deterministicWasi(clock) : discardOutput,
...(clock ? { timezoneOffset: 0, intrinsics: Intrinsics.ALL & ~Intrinsics.WEAK_REF } : {}),
});
if (clock) clock.creating = false;
let disposed = false;
/** Calls the script has issued, tools and globals; reported with every `idle`. */
let issued = 0;
/** Deterministic mode: the next result to apply, and results that arrived before it. */
let nextSeq = 1;
/** The result being processed, 0 during the script's first run; reported with `done`. */
let current = 0;
const early = new Map<number, HostToWorkerMessage>();
const apply = (message: HostToWorkerMessage): void => {
vm.withScope(() => {
vm.callFunction(
settle,
api,
vm.newNumber(message.id),
message.ok ? vm.true : vm.false,
message.payload === undefined ? vm.undefined : vm.newString(message.payload),
message.properties === undefined ? vm.undefined : vm.newString(message.properties),
);
});
drain();
};
const session: VmSession = {
deliver(message) {
if (disposed) return;
try {
if (!clock) {
apply(message);
return;
}
const { seq } = message;
if (seq === undefined || !Number.isInteger(seq) || seq < nextSeq || early.has(seq)) {
throw new Error(`Result ${message.id} has sequence number ${seq}, expected ${nextSeq} or later, once`);
}
early.set(seq, message);
for (let next = early.get(nextSeq); next !== undefined; next = early.get(nextSeq)) {
early.delete(nextSeq);
// Never backwards: a resumed host's clock may be behind the one that started the run.
if (next.now !== undefined && Number.isFinite(next.now)) clock.now = Math.max(clock.now, next.now);
current = nextSeq;
apply(next);
post({ type: "idle", seq: nextSeq, issued });
nextSeq++;
}
} catch (error) {
post(crashMessage(error));
}
},
dispose() {
if (disposed) return;
disposed = true;
vm.dispose();
},
};
let api: JSValueHandle;
let settle: JSValueHandle;
let stalled: JSValueHandle;
// `executePendingJobs()` runs the queue to empty without a chance to interrupt, so jobs run one at a
// time through the (internal, but public) exports; quickjs-wasi is pinned to an exact version.
const exports = vm._getExports();
/** Run queued jobs, then fail a script that waits on nothing that can ever resume it. */
const drain = () => {
let jobs = 0;
while (exports.qjs_is_job_pending()) {
// A job only fails for uncatchable errors, such as an interrupt.
if (exports.qjs_execute_pending_job() < 0) throw new Error("Promise job failed");
if (++jobs % JOBS_PER_POLL === 0 && options.shouldInterrupt()) throw new Error("Execution interrupted");
}
vm.callFunction(stalled, api).dispose();
};
try {
// Called from the prelude with primitives only.
const bridge = vm.newFunction("bridge", (kind, a, b, c) => {
switch (kind.toString()) {
case "call":
case "global":
issued++;
post({
type: "call",
id: a.toNumber(),
target: kind.toString() === "call" ? "tool" : "global",
name: b.toString(),
args: c === undefined || c.isUndefined ? undefined : c.toString(),
});
break;
case "output":
post({
type: "output",
item:
a.toString() === "image"
? { type: "image", data: b.toString(), mimeType: c.toString() }
: a.toString() === "console"
? { type: "text", text: b.toString(), console: true }
: { type: "text", text: b.toString() },
});
break;
case "done": {
// The host stops the VM on `done`, so the `idle` of this drain never arrives: report its numbers here.
const during = clock ? { during: { seq: current, issued } } : {};
if (a.toBoolean()) {
post({
type: "done",
ok: true,
value: b === undefined || b.isUndefined ? undefined : b.toString(),
writes: c.toString(),
...during,
});
} else {
post({ type: "done", ok: false, error: b.toString(), ...during });
}
break;
}
}
return vm.undefined;
});
// These handles live as long as the VM, which the caller disposes as a whole.
api = vm.withScope((scope) =>
scope.escape(
vm.callFunction(
vm.evalCode(PRELUDE_SOURCE, "codemode-prelude.js"),
vm.undefined,
bridge,
vm.newString(JSON.stringify(data.tools)),
vm.newString(JSON.stringify(data.globals)),
vm.newString(JSON.stringify(data.store)),
),
),
);
settle = api.getProp("settle");
const run = api.getProp("run");
stalled = api.getProp("stalled");
if (clock) {
const versions = vm.versions;
const budget = options.interruptBudget ?? "none";
post({
type: "started",
fingerprint: `quickjs-wasi ${versions["quickjs-wasi"]}, quickjs ${versions.quickjs}, prelude ${PRELUDE_HASH}, deterministic ${DETERMINISTIC_VERSION}, stack ${maxStackSize}, heap ${data.memoryLimitBytes ?? "none"}, budget ${budget}`,
});
}
// The prefix shares the first line with the script so reported line numbers
// match the script as written.
let fn: JSValueHandle;
try {
fn = vm.evalCode(`(async (tools, console) => {${data.code}\n})`, "codemode.js");
} catch (error) {
if (!(error instanceof JSException)) throw error;
post({ type: "done", ok: false, error: describeException(error) });
return session;
}
vm.callFunction(run, api, fn).dispose();
fn.dispose();
drain();
if (clock) post({ type: "idle", seq: 0, issued });
} catch (error) {
post(crashMessage(error));
}
return session;
}

View File

@@ -0,0 +1,39 @@
/**
* Worker thread entry. One worker runs one script inside a fresh QuickJS VM
* (a separate wasm instance), relays tool calls and output to the host, and
* reports the result. The host terminates the worker when the script settles,
* times out, or is aborted; the worker exists so that a spinning script never
* blocks the host thread.
*
* Importing this module starts the worker. Hosts that bundle their code (for
* example a Bun compiled executable) add a file that imports
* `@earendil-works/pi-codemode/worker` as a separate entrypoint and pass its URL
* or embedded-module string specifier as `workerUrl`.
*/
import { parentPort, workerData } from "node:worker_threads";
import { isHostToWorkerMessage, type WorkerData, type WorkerToHostMessage } from "./protocol.ts";
import { startVm } from "./vm.ts";
function post(message: WorkerToHostMessage): void {
parentPort?.postMessage(message);
}
async function main(data: WorkerData): Promise<void> {
const interrupt = new Int32Array(data.interrupt);
// The VM lives until the host terminates the worker, so the session is never disposed.
const session = await startVm({
data,
wasm: data.wasm,
post,
shouldInterrupt: () => Atomics.load(interrupt, 0) !== 0,
});
parentPort?.on("message", (message: unknown) => {
if (isHostToWorkerMessage(message)) session.deliver(message);
});
}
if (parentPort) {
main(workerData as WorkerData).catch((error: unknown) => {
post({ type: "crash", message: error instanceof Error ? `${error.name}: ${error.message}` : String(error) });
});
}

View File

@@ -0,0 +1,125 @@
/**
* Codemode source format: JavaScript, optionally preceded by one options line.
*
* ```js
* // @options: {"max_output_tokens": 2000, "timeout_ms": 30000}
* const text = await tools.read({ path: "package.json" });
* text(JSON.parse(text).name);
* ```
*/
export const CODEMODE_OPTIONS_PREFIX = "// @options:";
const SUPPORTED_FIELDS = ["max_output_tokens", "timeout_ms"] as const;
const SUPPORTED_FIELDS_TEXT = "`max_output_tokens` and `timeout_ms`";
/** Largest delay `setTimeout` supports, which bounds `timeout_ms`. */
const MAX_TIMEOUT_MS = 2_147_483_647;
/**
* Lark grammar for providers with grammar-constrained tool input. It only fixes the shape of the
* options line; the options JSON and the code are checked by {@link parseCodemodeSource}.
*/
export const CODEMODE_SOURCE_GRAMMAR = String.raw`
start: options_source | plain_source
options_source: OPTIONS_LINE NEWLINE SOURCE
plain_source: SOURCE
OPTIONS_LINE: /[ \t]*\/\/ @options:[^\r\n]*/
NEWLINE: /\r?\n/
SOURCE: /[\s\S]+/
`;
export interface CodemodeSourceOptions {
/** Token budget for the script's output. */
maxOutputTokens?: number;
/** Hard deadline for the whole script in milliseconds, including tool calls. */
timeoutMs?: number;
}
export interface ParsedCodemodeSource {
/** The script with the options line replaced by an empty line, so line numbers are unchanged. */
code: string;
options: CodemodeSourceOptions;
}
export class CodemodeSourceError extends Error {
constructor(message: string) {
super(message);
this.name = "CodemodeSourceError";
}
}
function isSafeInteger(value: unknown): value is number {
return typeof value === "number" && Number.isSafeInteger(value) && value >= 0;
}
function parseOptions(directive: string): CodemodeSourceOptions {
if (directive === "") {
throw new CodemodeSourceError(`@options must be a JSON object with supported fields ${SUPPORTED_FIELDS_TEXT}`);
}
let value: unknown;
try {
value = JSON.parse(directive);
} catch (error) {
throw new CodemodeSourceError(
`@options must be valid JSON with supported fields ${SUPPORTED_FIELDS_TEXT}: ${error instanceof Error ? error.message : String(error)}`,
);
}
if (typeof value !== "object" || value === null || Array.isArray(value)) {
throw new CodemodeSourceError(`@options must be a JSON object with supported fields ${SUPPORTED_FIELDS_TEXT}`);
}
const fields = value as Record<string, unknown>;
for (const key of Object.keys(fields)) {
if (!(SUPPORTED_FIELDS as readonly string[]).includes(key)) {
throw new CodemodeSourceError(`@options only supports ${SUPPORTED_FIELDS_TEXT}; got \`${key}\``);
}
}
const options: CodemodeSourceOptions = {};
const { max_output_tokens, timeout_ms } = fields;
if (max_output_tokens !== undefined) {
if (!isSafeInteger(max_output_tokens)) {
throw new CodemodeSourceError("@options field `max_output_tokens` must be a non-negative safe integer");
}
options.maxOutputTokens = max_output_tokens;
}
if (timeout_ms !== undefined) {
if (!isSafeInteger(timeout_ms) || timeout_ms === 0 || timeout_ms > MAX_TIMEOUT_MS) {
throw new CodemodeSourceError(
`@options field \`timeout_ms\` must be a positive integer up to ${MAX_TIMEOUT_MS}`,
);
}
options.timeoutMs = timeout_ms;
}
return options;
}
/**
* Split an optional first-line `// @options: {...}` from the script. Throws
* {@link CodemodeSourceError} for empty input, invalid options, and an options line that is not the
* first line, which would otherwise be ignored without notice.
*/
export function parseCodemodeSource(input: string): ParsedCodemodeSource {
if (input.trim() === "") {
throw new CodemodeSourceError(
'Expected JavaScript source text (non-empty). Provide JS only, optionally with a first line `// @options: {"max_output_tokens": 1000}`.',
);
}
const lines = input.split("\n");
const misplaced = lines.findIndex(
(line, index) => index > 0 && line.trimStart().startsWith(CODEMODE_OPTIONS_PREFIX),
);
if (misplaced !== -1) {
throw new CodemodeSourceError(
`The ${CODEMODE_OPTIONS_PREFIX} line must be the first line of the input; found one on line ${misplaced + 1}`,
);
}
const newline = input.indexOf("\n");
const firstLine = lines[0].replace(/\r$/, "");
const trimmed = firstLine.trimStart();
if (!trimmed.startsWith(CODEMODE_OPTIONS_PREFIX)) return { code: input, options: {} };
const code = newline === -1 ? "" : input.slice(newline);
if (code.trim() === "") {
throw new CodemodeSourceError("The @options line must be followed by JavaScript source on subsequent lines");
}
return { code, options: parseOptions(trimmed.slice(CODEMODE_OPTIONS_PREFIX.length).trim()) };
}

View File

@@ -0,0 +1,254 @@
import type { CodemodeWasmModule } from "./wasm.ts";
export interface CodemodeToolContext {
/**
* Aborted when the script finishes (including unawaited calls), the
* execution times out, the caller aborts, or the sandbox is closed.
*/
signal: AbortSignal;
}
/** A JSON Schema document. Only used to render declarations; values are not validated against it. */
export type CodemodeJsonSchema = { [key: string]: unknown } | boolean;
export interface CodemodeTool {
/**
* The script calls tools as `tools.<id>(args)`, where `<id>` is the name with characters that
* are not valid in identifiers replaced by `_` (see `toCodemodeIdentifier`), and also as
* `tools["<name>"](args)`. Globals are called as `<name>(args)` and must be identifiers, or
* `<namespace>.<member>`, which groups them into a frozen namespace object.
*/
name: string;
/** Shown as a doc comment in {@link renderDeclarations}, and listed in `ALL_TOOLS` for tools. */
description?: string;
/** Schema of the single argument. Rendered as the parameter type; `unknown` when omitted. */
inputSchema?: CodemodeJsonSchema;
/** Schema of the resolved value. Rendered as the promise type; `unknown` when omitted. */
outputSchema?: CodemodeJsonSchema;
/** Globals only: `execute` receives all call arguments as an array instead of the first one. */
spread?: boolean;
/**
* Globals only: TypeScript parameter list and return type for {@link renderDeclarations}, for
* example `(type: string, id?: string): Promise<Model[]>`. Replaces the rendering from the schemas.
*/
signature?: string;
/**
* `args` is whatever the script passed, after a JSON round trip. The return
* value must be JSON-serializable; a thrown error surfaces in the script as
* an `Error` with the same message.
*/
execute(args: unknown, context: CodemodeToolContext): Promise<unknown> | unknown;
}
/**
* One item of the script's output, in the order the script produced it: `text()` and `console.*`
* produce text items, with `console: true` for `console.*`, and `image()` image items. `data` is base64.
*/
export type CodemodeOutputItem =
| { type: "text"; text: string; console?: true }
| { type: "image"; data: string; mimeType: string };
export type CodemodeCallStatus = "ok" | "error" | "cancelled";
export interface CodemodeCall {
name: string;
status: CodemodeCallStatus;
durationMs: number;
}
export type CodemodeErrorKind =
/** The script threw or failed to parse. `name` and `stack` come from the script's error. */
| "script"
/**
* The overall deadline expired, or a VM on the calling thread used up its interrupt budget.
* The execution ended and the VM was asked to stop; a worker or remote VM may still be computing.
*/
| "timeout"
/** The caller's signal fired or the sandbox was closed. The execution ended and the VM was asked to stop. */
| "aborted"
/**
* The VM or its transport failed outside the script's control (for example a wasm trap, a missing
* worker file, or a remote sandbox that failed).
*/
| "sandbox";
export interface CodemodeError {
kind: CodemodeErrorKind;
name?: string;
message: string;
stack?: string;
}
/** Keys the script changed with `store()`. Only successful executions report writes. */
export interface CodemodeStoreWrites {
set: Record<string, unknown>;
/** Keys stored as `undefined`. */
delete: string[];
}
/**
* `output` is kept for failed executions too, up to the failure. `exit()` completes with `value: undefined`.
* `during`: in a {@link CodemodeSandboxBase.start} run that the script itself ended (it returned, threw, or exited),
* the result being processed when it did.
*/
export type CodemodeResult =
| {
ok: true;
value: unknown;
output: CodemodeOutputItem[];
calls: CodemodeCall[];
storeWrites: CodemodeStoreWrites;
during?: CodemodeDuring;
}
| {
ok: false;
error: CodemodeError;
output: CodemodeOutputItem[];
calls: CodemodeCall[];
during?: CodemodeDuring;
};
/** Options every sandbox accepts, wherever its VM runs. */
export interface CodemodeSandboxBaseOptions {
tools?: CodemodeTool[];
/**
* Functions exposed as top-level identifiers instead of on `tools`, for host helpers such as
* attaching an image to the result. They behave like tools (JSON round trip, promise result)
* but are not recorded in `result.calls`. Names must be identifiers and may not shadow the
* built-in globals (`tools`, `ALL_TOOLS`, `console`, `text`, `image`, `exit`, `store`, `load`).
*/
globals?: CodemodeTool[];
/**
* Overall deadline per execution, including time spent in tools. `Infinity` disables the
* deadline; the execution then only ends when the script settles or is aborted.
* Default: 300000.
*/
timeoutMs?: number;
/**
* Maximum memory the QuickJS VM may allocate. Allocations beyond it fail inside the script as
* `InternalError: out of memory`. Default: no limit beyond wasm32's 4 GiB address space.
*/
memoryLimitBytes?: number;
}
/** Options of `CodemodeSandbox`, which runs each VM in a Node or Bun worker thread. */
export interface CodemodeSandboxOptions extends CodemodeSandboxBaseOptions {
/**
* Compiled `quickjs-wasi/quickjs.wasm`, usually from {@link loadQuickJSWasm}. Default:
* `loadQuickJSWasm()`, the file in the installed `quickjs-wasi` package. Pass it when that file
* is not on disk, for example in a Bun compiled executable.
*/
wasm?: CodemodeWasmModule | Promise<CodemodeWasmModule>;
/**
* Worker entry that imports `@earendil-works/pi-codemode/worker`. Default: this package's own
* worker file. Pass it when this package is bundled, since the default is resolved relative to
* the module that creates the sandbox. Bun compiled executables require the relative string
* specifier of an embedded build entrypoint; other hosts usually use a URL.
*/
workerUrl?: string | URL;
/**
* Synchronously reserves capacity immediately before creating each worker, after loading wasm.
* Throw to deny admission without starting a worker. Return a release callback, called once if
* worker construction fails or when the worker actually exits, not when the result settles.
* The release callback must not throw. Reservations can be shared across sandbox instances.
*/
acquireWorker?(): () => void;
/**
* Called once for each worker started by execute() or start(). `exited` resolves when that worker
* actually exits, which may be after the result and close() settle: on Bun a non-interruptible
* built-in can delay termination. Use it to keep the worker counted against host concurrency
* limits until it exits. A thrown callback error fails the execution and requests worker termination.
*/
onWorkerStarted?(exited: Promise<void>): void;
}
/** A call the script made in a {@link CodemodeSandboxBase.start} run, answered with `CodemodeRun.deliver()`. */
export interface CodemodeHostCall {
/** The VM's own number for the call, `1`, `2`, ... in the order the script made its calls (tools and globals). */
id: number;
target: "tool" | "global";
name: string;
/** What the script passed, after a JSON round trip; for spread globals the argument list. */
args: unknown;
}
/** A tool as a {@link CodemodeSandboxBase.start} run's script sees it: `tools.<name>` and its `ALL_TOOLS` entry. */
export interface CodemodeDeclaredTool {
name: string;
/**
* The identifier the script uses, `tools.<jsName>`. Default: `toCodemodeIdentifier(name)`. A host that replays a run
* passes the one the run started with, so a changed mapping cannot rename a tool under a resumed script.
*/
jsName?: string;
description?: string;
}
/** A global as a {@link CodemodeSandboxBase.start} run's script sees it; the same names as `globals` allow. */
export interface CodemodeDeclaredGlobal {
name: string;
spread?: boolean;
}
/**
* A call's answer. A rejection becomes an `Error` with `message` in the script, carrying the own properties of
* `properties` (JSON values; `name`, `message`, and `stack` are ignored), for example `{ structuredOutput }`.
*/
export type CodemodeOutcome =
| { ok: true; value: unknown }
| { ok: false; message: string; properties?: Record<string, unknown> };
/** In a {@link CodemodeSandboxBase.start} run: the result the VM was processing when the script finished, 0 for its first run, and the calls issued by then. */
export interface CodemodeDuring {
seq: number;
issued: number;
}
export interface CodemodeRunOptions {
/** The VM clock's start, in ms since the epoch: `Date.now()` until the first result arrives. */
startedAt: number;
/** Seeds `Math.random()` and the VM's other randomness. A 32-bit unsigned integer. */
seed: number;
tools?: CodemodeDeclaredTool[];
globals?: CodemodeDeclaredGlobal[];
store?: Readonly<Record<string, unknown>>;
signal?: AbortSignal;
/** Deadline of this run in ms. Default: none (not the sandbox's default). */
timeoutMs?: number;
/** The VM's fingerprint, before the script runs and before any of its calls. */
onStarted?(fingerprint: string): void;
/** The script made a call. Answer it with `CodemodeRun.deliver()`, now or later. */
onCall(call: CodemodeHostCall): void;
/**
* The VM processed result `seq` (`0`: the script's first run, before any result) and waits again, having issued
* `issued` calls in total.
*/
onIdle?(seq: number, issued: number): void;
}
/** A running {@link CodemodeSandboxBase.start} execution. */
export interface CodemodeRun {
readonly result: Promise<CodemodeResult>;
/**
* Whether the run has its outcome: the script ended, timed out, or failed, or the run was aborted. Set when that is
* decided, before `result` settles; calls can no longer be answered. It does not mean the VM has exited.
*/
readonly finished: boolean;
/**
* Answer call `id` as result number `seq`. Results are numbered 1, 2, ... without gaps, and the VM applies them
* strictly in that order, whatever order they arrive in: after a gap it waits for the missing number. While it
* processes this one, its clock reads `now` (ms since the epoch), or the previous value if `now` is earlier.
* Throws for a call that is not waiting for an answer, a `seq` used before, or a `now` that is not finite.
*/
deliver(seq: number, now: number, id: number, outcome: CodemodeOutcome): void;
}
export interface CodemodeExecuteOptions {
signal?: AbortSignal;
/** Overrides the sandbox default for this execution. */
timeoutMs?: number;
/**
* Values the script reads with `load(key)`. Must be JSON-serializable. The script's own
* `store()` calls come back as `result.storeWrites`; persisting them is up to the caller.
*/
store?: Readonly<Record<string, unknown>>;
}

View File

@@ -0,0 +1,35 @@
import { readFile } from "node:fs/promises";
import { createRequire } from "node:module";
/**
* A compiled `WebAssembly.Module` of `quickjs-wasi/quickjs.wasm`. Typed opaquely because the Node
* type definitions do not declare the WebAssembly globals (they live in the DOM lib).
*/
export type CodemodeWasmModule = object;
interface WebAssemblyGlobal {
compile(bytes: Uint8Array): Promise<CodemodeWasmModule>;
}
const modules = new Map<string, Promise<CodemodeWasmModule>>();
/**
* Read and compile the QuickJS wasm once per path. `path` defaults to the file in the installed
* `quickjs-wasi` package; pass it when that file lives elsewhere, for example embedded in a Bun
* compiled executable. A failed load is retried on the next call.
*/
export function loadQuickJSWasm(path?: string): Promise<CodemodeWasmModule> {
const resolved = path ?? createRequire(import.meta.url).resolve("quickjs-wasi/quickjs.wasm");
let module = modules.get(resolved);
if (!module) {
const { WebAssembly } = globalThis as unknown as { WebAssembly: WebAssemblyGlobal };
module = readFile(resolved)
.then((bytes) => WebAssembly.compile(bytes))
.catch((error: unknown) => {
modules.delete(resolved);
throw error;
});
modules.set(resolved, module);
}
return module;
}

View File

@@ -0,0 +1,328 @@
import { describe, expect, it } from "vitest";
import { renderDeclarations, renderToolSample, renderToolSignature, schemaToType } from "../src/index.ts";
const execute = () => undefined;
describe("schemaToType", () => {
it("renders primitives, literals, and unions", () => {
expect(schemaToType({ type: "string" })).toBe("string");
expect(schemaToType({ type: "integer" })).toBe("number");
expect(schemaToType({ type: ["string", "null"] })).toBe("string | null");
expect(schemaToType({ const: "a" })).toBe('"a"');
expect(schemaToType({ enum: ["a", 1, null] })).toBe('"a" | 1 | null');
expect(schemaToType({ anyOf: [{ type: "string" }, { type: "number" }] })).toBe("string | number");
expect(schemaToType({ anyOf: [{ type: "string" }, {}] })).toBe("unknown");
expect(schemaToType({ allOf: [{ anyOf: [{ type: "string" }, { type: "number" }] }, { const: 1 }] })).toBe(
"(string | number) & 1",
);
expect(schemaToType({ $ref: "#/defs/x" })).toBe("unknown");
expect(schemaToType(true)).toBe("unknown");
expect(schemaToType(false)).toBe("never");
});
it("renders objects on one line with sorted properties", () => {
expect(
schemaToType({
type: "object",
properties: { city: { type: "string" }, "max-lines": { type: "number" } },
required: ["city"],
additionalProperties: false,
}),
).toBe('{ city: string; "max-lines"?: number; }');
expect(schemaToType({ type: "object", additionalProperties: { type: "number" } })).toBe(
"{ [key: string]: number; }",
);
expect(schemaToType({ type: "object" })).toBe("{ [key: string]: unknown; }");
expect(schemaToType({ type: "object", properties: {}, additionalProperties: false })).toBe("{}");
});
it("puts property descriptions on comment lines", () => {
expect(
schemaToType({
type: "object",
properties: {
weather: {
type: "array",
description: "look up weather for a given list of locations",
items: { type: "object", properties: { location: { type: "string" } }, required: ["location"] },
},
},
required: ["weather"],
}),
).toBe("{\n // look up weather for a given list of locations\n weather: Array<{ location: string; }>;\n}");
expect(
schemaToType({
type: "object",
properties: {
outer: {
type: "object",
description: "Outer",
properties: { inner: { type: "string", description: "Inner" } },
},
},
}),
).toBe("{\n // Outer\n outer?: {\n // Inner\n inner?: string;\n };\n}");
});
// https://github.com/earendil-works/pi/issues/10707
it("puts constraints the type cannot express on property comment lines", () => {
expect(
schemaToType({
type: "object",
properties: {
timeoutMs: {
type: "integer",
minimum: 0,
maximum: 30000,
default: 0,
description: "Wait in milliseconds.",
},
id: { type: "string", pattern: "^[a-z]+$", maxLength: 8 },
tags: { type: "array", items: { type: "string" }, maxItems: 5, uniqueItems: true, deprecated: true },
name: { type: "string" },
},
required: ["id"],
}),
).toBe(
[
"{",
" // @maxLength 8 @pattern ^[a-z]+$",
" id: string;",
" name?: string;",
" // @maxItems 5 @uniqueItems @deprecated",
" tags?: Array<string>;",
" // Wait in milliseconds.",
" // @integer @minimum 0 @maximum 30000 @default 0",
" timeoutMs?: number;",
"}",
].join("\n"),
);
});
it("reads constraints of optional fields from the non-null branch", () => {
// Pydantic's `limit: int | None = Field(None, ge=1, le=100)`.
expect(
schemaToType({
type: "object",
properties: {
limit: {
anyOf: [{ type: "integer", minimum: 1, maximum: 100 }, { type: "null" }],
default: null,
title: "Limit",
},
},
}),
).toBe("{\n // @integer @minimum 1 @maximum 100\n limit?: number | null;\n}");
// Constraints of the property and the branch both apply.
expect(
schemaToType({
type: "object",
properties: {
n: {
anyOf: [{ type: "integer", minimum: 10, maximum: 50, format: "int32" }, { type: "null" }],
minimum: 0,
maximum: 20,
format: "port",
},
},
}),
).toBe("{\n // @integer @minimum 10 @maximum 20 @format port @format int32\n n?: number | null;\n}");
});
it("reads draft-04 boolean exclusive bounds", () => {
expect(
schemaToType({
type: "object",
properties: {
n: { type: "integer", minimum: 0, exclusiveMinimum: true, maximum: 10, exclusiveMaximum: false },
m: { anyOf: [{ type: "number", maximum: 5, exclusiveMaximum: true }, { type: "null" }], maximum: 8 },
},
}),
).toBe(
[
"{",
" // @maximum 8 @exclusiveMaximum 5",
" m?: number | null;",
" // @integer @exclusiveMinimum 0 @maximum 10",
" n?: number;",
"}",
].join("\n"),
);
});
it("leaves out long tags and escapes line terminators", () => {
expect(
schemaToType({
type: "object",
properties: { config: { type: "object", default: { text: "x".repeat(100) } } },
}),
).toBe("{ config?: { [key: string]: unknown; }; }");
expect(
schemaToType({
type: "object",
properties: {
sep: { type: "string", pattern: "^[\r\n\u2028\u2029]$", default: "\u2028", description: "a\u2028b" },
},
}),
).toBe(
[
"{",
" // a",
" // b",
' // @pattern ^[\\u000d\\u000a\\u2028\\u2029]$ @default "\\u2028"',
" sep?: string;",
"}",
].join("\n"),
);
});
it("drops constraint tags before rendering a type over the budget as unknown", () => {
const schema = { type: "object", properties: { n: { type: "integer", minimum: 1 } } };
expect(schemaToType(schema)).toBe("{\n // @integer @minimum 1\n n?: number;\n}");
expect(schemaToType(schema, { maxChars: 20 })).toBe("{ n?: number; }");
expect(schemaToType(schema, { maxChars: 10 })).toBe("unknown");
});
it("resolves local references and stops at recursive ones", () => {
const schema = {
type: "object",
properties: {
item: { $ref: "#/$defs/Item" },
legacy: { $ref: "#/definitions/Legacy" },
remote: { $ref: "https://example.com/schema.json" },
},
required: ["item"],
$defs: {
Item: {
type: "object",
properties: { id: { type: "string" }, parent: { $ref: "#/$defs/Item" } },
required: ["id"],
},
},
definitions: { Legacy: { enum: ["a", "b"] } },
};
expect(schemaToType(schema)).toBe(
'{ item: { id: string; parent?: unknown; }; legacy?: "a" | "b"; remote?: unknown; }',
);
});
it("renders arrays and tuples", () => {
expect(schemaToType({ type: "array", items: { type: "string" } })).toBe("Array<string>");
expect(schemaToType({ type: "array", prefixItems: [{ type: "string" }, { type: "number" }] })).toBe(
"[string, number]",
);
expect(schemaToType({ type: "array" })).toBe("unknown[]");
});
it("renders types over the budget as unknown", () => {
const properties = Object.fromEntries(Array.from({ length: 50 }, (_, i) => [`field${i}`, { type: "string" }]));
const schema = { type: "object", properties };
expect(schemaToType(schema, { maxChars: 100 })).toBe("unknown");
expect(schemaToType(schema)).toContain("field49?: string;");
});
});
describe("tool declarations", () => {
it("renders signatures with normalized identifiers", () => {
expect(
renderToolSignature({
name: "hidden-dynamic-tool",
inputSchema: {
type: "object",
properties: { city: { type: "string" } },
required: ["city"],
additionalProperties: false,
},
outputSchema: { type: "object", properties: { ok: { type: "boolean" } }, required: ["ok"] },
}),
).toBe("hidden_dynamic_tool(args: { city: string; }): Promise<{ ok: boolean; }>;");
expect(renderToolSignature({ name: "free" })).toBe("free(args: unknown): Promise<unknown>;");
});
it("renders output schemas shaped like an MCP CallToolResult as plain types", () => {
const outputSchema = {
type: "object",
properties: {
content: { type: "array", items: { type: "object" } },
isError: { type: "boolean" },
_meta: { type: "object" },
},
required: ["content"],
};
expect(
renderToolSignature({ name: "plain", inputSchema: { type: "object", properties: {} }, outputSchema }),
).toBe(
"plain(args: { [key: string]: unknown; }): Promise<{ _meta?: { [key: string]: unknown; }; content: Array<{ [key: string]: unknown; }>; isError?: boolean; }>;",
);
});
it("renders the per-tool sample", () => {
expect(renderToolSample({ name: "foo", description: "bar", inputSchema: { type: "string" } })).toBe(
"bar\n\ncodemode tool declaration:\n```ts\ndeclare const tools: { foo(args: string): Promise<unknown>; };\n```",
);
});
});
describe("renderDeclarations", () => {
it("renders tools and globals", () => {
const text = renderDeclarations({
tools: [
{
name: "read",
description: "Read a file.\nSecond line.",
inputSchema: { type: "object", properties: { path: { type: "string" } }, required: ["path"] },
outputSchema: { type: "string" },
execute,
},
{ name: "remote-api", execute },
],
globals: [{ name: "attach", description: "Attach it.", inputSchema: { type: "string" }, execute }],
});
expect(text).toBe(
[
"declare const tools: {",
" /**",
" * Read a file.",
" * Second line.",
" */",
" read(args: { path: string; }): Promise<string>;",
" remote_api(args: unknown): Promise<unknown>;",
"};",
"",
"/** Attach it. */",
"declare function attach(args: string): Promise<unknown>;",
].join("\n"),
);
});
it("renders namespaced globals and explicit signatures", () => {
const text = renderDeclarations({
globals: [
{
name: "models.list",
description: "List models.",
signature: "(type: string): Promise<string[]>",
execute,
},
{ name: "models.get", inputSchema: { type: "string" }, execute },
{ name: "plain", signature: "(): void", execute },
],
});
expect(text).toBe(
[
"declare function plain(): void;",
"",
"declare const models: {",
" /** List models. */",
" list(type: string): Promise<string[]>;",
" get(args: string): Promise<unknown>;",
"};",
].join("\n"),
);
});
it("escapes comment terminators in descriptions", () => {
const text = renderDeclarations({ tools: [{ name: "x", description: "a */ b", execute }] });
expect(text).toContain("/** a *\\/ b */");
});
});

View File

@@ -0,0 +1,410 @@
import { afterEach, describe, expect, it } from "vitest";
import {
type CodemodeHostCall,
type CodemodeOutcome,
type CodemodeRemoteExchange,
type CodemodeRun,
type CodemodeRunOptions,
CodemodeSandbox,
type CodemodeSandboxBase,
type CodemodeSandboxBaseOptions,
loadQuickJSWasm,
RemoteCodemodeSandbox,
serveCodemodeRemote,
} from "../src/index.ts";
/**
* `start()` runs a script deterministically with the caller answering its calls, so a durable host can replay it
* after a restart. Every test runs on the worker sandbox (Node) and the remote sandbox (Cloudflare), whose transport
* clones every message as Workers RPC would.
*/
const BUDGET = 50_000;
/** `delayFirst` holds the reply to the first result back, so later results reach the VM before it. */
function remote(options: CodemodeSandboxBaseOptions, delayFirst = false): RemoteCodemodeSandbox {
return new RemoteCodemodeSandbox({
...options,
remote: () => ({
execute: (request, exchange) => {
const wrapped: CodemodeRemoteExchange = async (messages) => {
const reply = structuredClone(await exchange(structuredClone(messages)));
if (delayFirst && reply?.type === "result" && reply.seq === 1) {
await new Promise((resolve) => setTimeout(resolve, 30));
}
return reply;
};
return serveCodemodeRemote(structuredClone(request), wrapped, {
wasm: loadQuickJSWasm(),
maxInterruptBudget: BUDGET,
});
},
}),
});
}
const BACKENDS: Record<string, (options?: CodemodeSandboxBaseOptions) => CodemodeSandboxBase> = {
worker: (options = {}) => new CodemodeSandbox(options),
remote: (options = {}) => remote(options),
};
const STARTED_AT = Date.UTC(2026, 0, 15, 10, 0, 0);
type Recorded = {
run: CodemodeRun;
calls: CodemodeHostCall[];
idles: [seq: number, issued: number][];
events: string[];
fingerprint: () => string | undefined;
};
/**
* Start `code` with tools `echo` and `wait`. Every call but `wait` is answered at once, by default with its argument,
* as result number 1, 2, ... with the clock at `STARTED_AT + 1000 * seq`; `wait` is left for the test to answer.
*/
function start(
sandbox: CodemodeSandboxBase,
code: string,
options: Partial<CodemodeRunOptions> & { answer?: (call: CodemodeHostCall) => CodemodeOutcome } = {},
): Recorded {
const calls: CodemodeHostCall[] = [];
const idles: [number, number][] = [];
const events: string[] = [];
let fingerprint: string | undefined;
let seq = 0;
const answer = options.answer ?? ((call: CodemodeHostCall) => ({ ok: true, value: call.args }));
const run: CodemodeRun = sandbox.start(code, {
startedAt: STARTED_AT,
seed: 1,
tools: [{ name: "echo" }, { name: "wait" }],
timeoutMs: 20_000,
...options,
onStarted: (value) => {
fingerprint = value;
events.push("started");
},
onCall: (call) => {
calls.push(call);
events.push(`call ${call.id}`);
if (call.name === "wait") return;
queueMicrotask(() => {
seq++;
run.deliver(seq, STARTED_AT + 1000 * seq, call.id, answer(call));
});
},
onIdle: (seq, issued) => {
idles.push([seq, issued]);
events.push(`idle ${seq}`);
},
});
return { run, calls, idles, events, fingerprint: () => fingerprint };
}
const savedTimezone = process.env.TZ;
afterEach(() => {
if (savedTimezone === undefined) delete process.env.TZ;
else process.env.TZ = savedTimezone;
});
for (const [backend, create] of Object.entries(BACKENDS)) {
describe(`deterministic runs (${backend})`, () => {
it("makes the same calls and output for the same start, seed, and results", async () => {
const code = `
const values = [Date.now(), performance.now(), Math.random(), Math.random(), new Date().toISOString()];
const first = await tools.echo(values);
text(JSON.stringify([Date.now(), performance.now(), Math.random()]));
return await tools.echo(first.concat(Math.random()));
`;
const sandbox = create();
const one = start(sandbox, code);
const two = start(sandbox, code);
const [a, b] = await Promise.all([one.run.result, two.run.result]);
// Call durations are wall-clock time; everything else is the same.
const timeless = (result: typeof a) => ({
...result,
calls: result.calls.map(({ name, status }) => ({ name, status })),
});
expect(timeless(a)).toEqual(timeless(b));
expect(one.calls).toEqual(two.calls);
// The clock: startedAt, then the time of the result being processed; performance.now() counts from startedAt.
const [now, elapsed, random] = one.calls[0]!.args as number[];
expect([now, elapsed]).toEqual([STARTED_AT, 0]);
expect((one.calls[0]!.args as unknown[])[4]).toBe("2026-01-15T10:00:00.000Z");
const printed = JSON.parse(a.ok ? (a.output[0] as { text: string }).text : "[]") as number[];
expect(printed.slice(0, 2)).toEqual([STARTED_AT + 1000, 1000]);
// Pinned: a change here changes what replayed scripts see, so it must change the fingerprint too.
expect(random).toBe(PINNED_FIRST_RANDOM);
// The seed decides the random values, not the start time.
const later = start(sandbox, code, { startedAt: STARTED_AT + 5000 });
const reseeded = start(sandbox, code, { seed: 2 });
await Promise.all([later.run.result, reseeded.run.result]);
expect((later.calls[0]!.args as number[])[2]).toBe(random);
expect((reseeded.calls[0]!.args as number[])[2]).not.toBe(random);
// Seed 0 has its own values: QuickJS turns a zero state into 1, which must not make it seed 1's.
const zero = start(sandbox, "return [Math.random(), Math.random()]", { seed: 0 });
const zeroResult = await zero.run.result;
expect(zeroResult).toMatchObject({ ok: true, value: PINNED_SEED_ZERO });
expect((zeroResult.ok ? zeroResult.value : []) as number[]).not.toContain(random);
});
it("runs in UTC whatever the host's timezone, and without WeakRef or FinalizationRegistry", async () => {
process.env.TZ = "Asia/Kolkata";
const code =
"return [new Date(0).getHours(), new Date(0).getTimezoneOffset(), typeof WeakRef, typeof FinalizationRegistry]";
const sandbox = create();
const { run } = start(sandbox, code);
expect(await run.result).toMatchObject({ ok: true, value: [0, 0, "undefined", "undefined"] });
// execute() is unchanged: the host's timezone and every intrinsic.
expect(await sandbox.execute(code)).toMatchObject({ ok: true, value: [5, -330, "function", "function"] });
});
it("applies results in seq order, whatever order they are delivered in", async () => {
const sandbox = create();
const recorded = start(
sandbox,
`const order = [];
await Promise.all([tools.wait("a").then(() => order.push("a")), tools.wait("b").then(() => order.push("b"))]);
return order;`,
);
await waitFor(() => recorded.calls.length === 2 && recorded.idles.length === 1);
const [a, b] = recorded.calls;
// b is result 1 and a result 2, but a arrives first.
recorded.run.deliver(2, STARTED_AT + 2000, a!.id, { ok: true, value: null });
recorded.run.deliver(1, STARTED_AT + 1000, b!.id, { ok: true, value: null });
expect(await recorded.run.result).toMatchObject({ ok: true, value: ["b", "a"] });
// No idle after the last result: the script finished in that drain, and the host stopped the VM.
expect(recorded.idles).toEqual([
[0, 2],
[1, 2],
]);
});
it("reports the fingerprint before any call and the calls issued after each result", async () => {
const sandbox = create({ memoryLimitBytes: 64 * 1024 * 1024 });
const recorded = start(sandbox, "await tools.echo(1); await Promise.all([tools.echo(2), tools.echo(3)]);");
expect(await recorded.run.result).toMatchObject({ ok: true });
expect(recorded.events.slice(0, 3)).toEqual(["started", "call 1", "idle 0"]);
expect(recorded.idles).toEqual([
[0, 1],
[1, 3],
[2, 3],
]);
expect(recorded.calls.map((call) => [call.id, call.target, call.name, call.args])).toEqual([
[1, "tool", "echo", 1],
[2, "tool", "echo", 2],
[3, "tool", "echo", 3],
]);
const fingerprint = recorded.fingerprint()!;
expect(fingerprint).toMatch(
/^quickjs-wasi \S+, quickjs \S+, prelude [0-9a-f]{8}, deterministic \d+, stack \d+, heap 67108864, budget \S+$/,
);
if (backend === "remote") expect(fingerprint).toContain(`budget ${BUDGET}`);
else expect(fingerprint).toContain("budget none");
const same = start(sandbox, "return 1");
const other = start(create({ memoryLimitBytes: 32 * 1024 * 1024 }), "return 1");
await Promise.all([same.run.result, other.run.result]);
expect(same.fingerprint()).toBe(fingerprint);
expect(other.fingerprint()).not.toBe(fingerprint);
});
it("numbers calls to globals with tools, and answers errors as rejections", async () => {
const sandbox = create();
const recorded = start(
sandbox,
`const found = await search("x", 2);
try { await tools.echo("fail"); } catch (error) { text(error.message); }
return found;`,
{
globals: [{ name: "search", spread: true }],
answer: (call) =>
call.args === "fail"
? { ok: false, message: "no" }
: { ok: true, value: call.name === "search" ? ["found"] : call.args },
},
);
const result = await recorded.run.result;
expect(recorded.calls[0]).toEqual({ id: 1, target: "global", name: "search", args: ["x", 2] });
expect(result).toMatchObject({ ok: true, value: ["found"], output: [{ type: "text", text: "no" }] });
expect(recorded.calls[1]).toMatchObject({ id: 2, target: "tool", name: "echo" });
// Calls to globals count as issued calls too.
expect(recorded.idles.slice(0, 2)).toEqual([
[0, 1],
[1, 2],
]);
expect(result.calls).toMatchObject([{ name: "echo", status: "error" }]);
});
it("keeps the clock monotonic and rejects a clock value that is not finite", async () => {
const sandbox = create();
const recorded = start(
sandbox,
"const before = Date.now(); await tools.wait(1); return [before, Date.now(), performance.now()]",
);
await waitFor(() => recorded.calls.length === 1);
expect(() => recorded.run.deliver(1, Number.NaN, 1, { ok: true, value: null })).toThrow("not a finite number");
// A resumed host whose clock is behind the run's start.
recorded.run.deliver(1, STARTED_AT - 2000, 1, { ok: true, value: null });
expect(await recorded.run.result).toMatchObject({ ok: true, value: [STARTED_AT, STARTED_AT, 0] });
});
it("reports the result the VM was processing when the script ended", async () => {
const sandbox = create();
const returned = start(sandbox, "await tools.echo(1); return await tools.echo(2)");
const thrown = start(
sandbox,
"await tools.echo(1); await Promise.all([tools.echo(2), tools.echo(3)]); throw new Error('x')",
);
const first = start(sandbox, "return 1");
expect(await returned.run.result).toMatchObject({ ok: true, value: 2, during: { seq: 2, issued: 2 } });
expect(await thrown.run.result).toMatchObject({ ok: false, during: { seq: 3, issued: 3 } });
expect(await first.run.result).toMatchObject({ ok: true, during: { seq: 0, issued: 0 } });
expect(await sandbox.execute("return 1")).not.toHaveProperty("during");
});
it("leaves a call waiting when its answer cannot be serialized", async () => {
const sandbox = create();
const recorded = start(sandbox, "return await tools.wait(1)");
await waitFor(() => recorded.calls.length === 1);
expect(() => recorded.run.deliver(1, STARTED_AT, 1, { ok: true, value: 1n })).toThrow();
expect(() =>
recorded.run.deliver(1, STARTED_AT, 1, { ok: false, message: "x", properties: { big: 1n } }),
).toThrow();
recorded.run.deliver(1, STARTED_AT, 1, { ok: true, value: "fine" });
expect(await recorded.run.result).toMatchObject({ ok: true, value: "fine" });
});
it("rejects with an Error carrying the answer's properties", async () => {
const sandbox = create();
const recorded = start(
sandbox,
"try { await tools.echo(1); } catch (e) { return [e instanceof Error, e.message, e.name, e.structuredOutput]; }",
{
answer: () => ({
ok: false,
message: "denied",
properties: { structuredOutput: { code: 7 }, name: "Ignored", message: "ignored" },
}),
},
);
expect(await recorded.run.result).toMatchObject({ ok: true, value: [true, "denied", "Error", { code: 7 }] });
});
it("rejects rejection properties that are not an object, and tools sharing a script name", async () => {
const sandbox = create();
const recorded = start(sandbox, "return await tools.wait(1)");
await waitFor(() => recorded.calls.length === 1);
for (const properties of [null, [1]] as unknown as Record<string, unknown>[]) {
expect(() => recorded.run.deliver(1, STARTED_AT, 1, { ok: false, message: "x", properties })).toThrow(
"not an object",
);
}
recorded.run.deliver(1, STARTED_AT, 1, { ok: true, value: 1 });
expect(await recorded.run.result).toMatchObject({ ok: true, value: 1 });
expect(() =>
sandbox.start("", {
startedAt: STARTED_AT,
seed: 1,
tools: [{ name: "a-b" }, { name: "a_b" }],
onCall: () => {},
}),
).toThrow("script names must be unique");
});
it("uses the script names the run passes", async () => {
const sandbox = create();
const recorded = start(
sandbox,
"return [typeof tools.renamed, Object.keys(tools), ALL_TOOLS.map((tool) => tool.name)]",
{
tools: [{ name: "my-tool", jsName: "renamed" }],
},
);
expect(await recorded.run.result).toMatchObject({
ok: true,
value: ["function", ["renamed", "my-tool"], ["renamed"]],
});
});
it("ends with an error the script cannot catch when the caller aborts", async () => {
const sandbox = create();
const controller = new AbortController();
const recorded = start(
sandbox,
`try { await tools.wait(1); } catch { text("caught"); }
text("after");`,
{ signal: controller.signal },
);
await waitFor(() => recorded.calls.length === 1);
controller.abort(new Error("diverged"));
const result = await recorded.run.result;
expect(result).toMatchObject({ ok: false, error: { kind: "aborted", message: "diverged" }, output: [] });
expect(() => recorded.run.deliver(1, STARTED_AT, 1, { ok: true, value: null })).not.toThrow();
});
it("is finished once it has its outcome", async () => {
const sandbox = create();
const recorded = start(sandbox, "await tools.wait(1); return 2");
await waitFor(() => recorded.calls.length === 1);
expect(recorded.run.finished).toBe(false);
recorded.run.deliver(1, STARTED_AT, 1, { ok: true, value: null });
await recorded.run.result;
expect(recorded.run.finished).toBe(true);
});
it("rejects a call that is not waiting and a result number used twice", async () => {
const sandbox = create();
const recorded = start(sandbox, "await Promise.all([tools.wait(1), tools.wait(2)])");
await waitFor(() => recorded.calls.length === 2);
const { run } = recorded;
expect(() => run.deliver(1, STARTED_AT, 99, { ok: true, value: null })).toThrow("Call 99 is not waiting");
run.deliver(1, STARTED_AT, 1, { ok: true, value: null });
expect(() => run.deliver(1, STARTED_AT, 2, { ok: true, value: null })).toThrow("used once");
expect(() => run.deliver(0, STARTED_AT, 2, { ok: true, value: null })).toThrow("used once");
run.deliver(2, STARTED_AT, 2, { ok: true, value: null });
expect(await run.result).toMatchObject({ ok: true });
});
});
}
describe("deterministic runs (remote transport)", () => {
it("applies results in seq order when the transport reorders replies sent in order", async () => {
const sandbox = remote({}, true);
const recorded = start(
sandbox,
`const order = [];
await Promise.all([tools.wait("a").then(() => order.push("a")), tools.wait("b").then(() => order.push("b"))]);
return order;`,
);
await waitFor(() => recorded.calls.length === 2);
const [a, b] = recorded.calls;
recorded.run.deliver(1, STARTED_AT + 1000, a!.id, { ok: true, value: null });
recorded.run.deliver(2, STARTED_AT + 2000, b!.id, { ok: true, value: null });
expect(await recorded.run.result).toMatchObject({ ok: true, value: ["a", "b"] });
});
});
describe("execute() is not deterministic", () => {
it("reads the real clock and real randomness", async () => {
const sandbox = new CodemodeSandbox();
const before = Date.now();
const first = await sandbox.execute("return [Date.now(), Math.random()]");
const second = await sandbox.execute("return [Date.now(), Math.random()]");
const [now, random] = (first.ok ? first.value : []) as number[];
expect(now).toBeGreaterThanOrEqual(before);
expect(now).toBeLessThanOrEqual(Date.now());
expect((second.ok ? second.value : []) as number[]).not.toContain(random);
});
});
async function waitFor(check: () => boolean): Promise<void> {
const deadline = Date.now() + 5000;
while (!check()) {
if (Date.now() > deadline) throw new Error("Condition was not reached");
await new Promise((resolve) => setTimeout(resolve, 5));
}
}
/** The first two `Math.random()` values with seed 0. */
const PINNED_SEED_ZERO = [0.6711372530266764, 0.7258461452833667];
/** The first `Math.random()` with seed 1. */
const PINNED_FIRST_RANDOM = 0.34227450605335275;

View File

@@ -0,0 +1,8 @@
/**
* Stands in for the real worker in host tests: posts the JSON message (or array of messages) passed
* as the script source, so tests can send the host payloads the real prelude never produces.
*/
import { parentPort, workerData } from "node:worker_threads";
const messages: unknown = JSON.parse((workerData as { code: string }).code);
for (const message of Array.isArray(messages) ? messages : [messages]) parentPort?.postMessage(message);

View File

@@ -0,0 +1,51 @@
import { CodemodeSandbox } from "../../src/runtime/host.ts";
const how = process.argv[2];
if (how !== "abort" && how !== "timeout" && how !== "close") throw new Error(`Unknown stop mode: ${how}`);
const started = performance.now();
process.on("exit", () => {
console.log(JSON.stringify({ type: "exit", elapsedMs: performance.now() - started }));
});
const controller = new AbortController();
let timer: ReturnType<typeof setTimeout> | undefined;
let workerExited = false;
const sandbox = new CodemodeSandbox({
memoryLimitBytes: 256 * 1024 * 1024,
onWorkerStarted: (exited) => {
void exited.then(() => {
workerExited = true;
});
},
tools: [
{
name: "ready",
execute: () => {
if (how !== "timeout") {
timer = setTimeout(() => {
if (how === "abort") controller.abort();
else void sandbox.close();
}, 100);
}
},
},
],
});
try {
const result = await sandbox.execute(
"await tools.ready(); return 'a'.repeat(300000).indexOf('a'.repeat(10000) + 'b')",
{ signal: controller.signal, timeoutMs: how === "timeout" ? 300 : Number.POSITIVE_INFINITY },
);
if (result.ok) throw new Error("The script completed instead of being cancelled");
await sandbox.close();
console.log(
JSON.stringify({
type: "result",
kind: result.error.kind,
elapsedMs: performance.now() - started,
workerExited,
}),
);
} finally {
clearTimeout(timer);
await sandbox.close();
}

View File

@@ -0,0 +1,159 @@
import { describe, expect, it } from "vitest";
import {
type CodemodeRemote,
type CodemodeRemoteExchange,
type CodemodeRemoteRequest,
type CodemodeServeOptions,
InlineCodemodeSandbox,
loadQuickJSWasm,
RemoteCodemodeSandbox,
serveCodemodeRemote,
} from "../src/index.ts";
const wasm = loadQuickJSWasm();
/** An in-process remote that records what crosses the boundary, cloned as Workers RPC would. */
function recordingRemote(options: Partial<CodemodeServeOptions> = {}) {
const exchanges: number[] = [];
const served: Promise<void>[] = [];
const remote: CodemodeRemote = {
execute(request: CodemodeRemoteRequest, exchange: CodemodeRemoteExchange) {
const done = serveCodemodeRemote(
structuredClone(request),
async (messages) => {
exchanges.push(messages.length);
return structuredClone(await exchange(structuredClone(messages)));
},
{ wasm, ...options },
);
served.push(done);
return done;
},
};
return { remote, exchanges, served };
}
describe("InlineCodemodeSandbox", () => {
it("fails a script that computes past its interrupt budget as a timeout", async () => {
const sandbox = new InlineCodemodeSandbox({ wasm, interruptBudget: 1_000 });
for (const code of ["while (true) {}", "while (true) await null"]) {
const result = await sandbox.execute(code, { timeoutMs: Number.POSITIVE_INFINITY });
expect(result).toMatchObject({
ok: false,
error: { kind: "timeout", message: "Execution exceeded its interrupt budget of 1000" },
});
}
});
it("counts the budget across waits on tools", async () => {
const sandbox = new InlineCodemodeSandbox({
wasm,
interruptBudget: 1_000,
tools: [{ name: "tick", execute: () => 1 }],
});
const result = await sandbox.execute("for (;;) { for (let i = 0; i < 1e5; i++); await tools.tick(); }");
expect(result).toMatchObject({ ok: false, error: { kind: "timeout" } });
expect(result.calls.length).toBeGreaterThan(1);
});
});
describe("RemoteCodemodeSandbox", () => {
it("runs tools in the caller and the script on the remote", async () => {
const { remote, served } = recordingRemote();
const seen: unknown[] = [];
const sandbox = new RemoteCodemodeSandbox({
remote: () => remote,
tools: [
{
name: "add",
execute: (args) => {
seen.push(args);
return (args as { a: number }).a + 1;
},
},
],
});
const result = await sandbox.execute("text('hi'); return await tools.add({ a: 1 })", { store: { k: 1 } });
expect(result).toMatchObject({ ok: true, value: 2, output: [{ type: "text", text: "hi" }] });
expect(seen).toEqual([{ a: 1 }]);
expect(result.calls).toMatchObject([{ name: "add", status: "ok" }]);
await Promise.all(served);
});
it("batches output instead of making one exchange per line", async () => {
const { remote, exchanges } = recordingRemote();
const sandbox = new RemoteCodemodeSandbox({ remote: () => remote });
const result = await sandbox.execute("for (let i = 0; i < 5000; i++) console.log(i); return 'ok'");
expect(result).toMatchObject({ ok: true, value: "ok" });
expect(result.output).toHaveLength(5000);
expect(result.output[4999]).toEqual({ type: "text", text: "4999", console: true });
expect(exchanges.length).toBeLessThanOrEqual(6);
});
it("caps the interrupt budget and memory at the remote's limits", async () => {
const { remote } = recordingRemote({ maxInterruptBudget: 1_000, maxMemoryLimitBytes: 8 * 1024 * 1024 });
const sandbox = new RemoteCodemodeSandbox({ remote: () => remote, interruptBudget: 1e9 });
expect(await sandbox.execute("while (true) {}")).toMatchObject({
ok: false,
error: { kind: "timeout", message: "Execution exceeded its interrupt budget of 1000" },
});
const memory = await sandbox.execute(
"const parts = []; for (;;) parts.push('x'.repeat(1 << 20) + parts.length);",
);
expect(memory).toMatchObject({ ok: false });
expect(JSON.stringify(memory)).toContain("out of memory");
});
it("tells the remote to stop when the caller aborts", async () => {
const { remote, served } = recordingRemote();
let started: () => void = () => {};
const call = new Promise<void>((resolve) => {
started = resolve;
});
const sandbox = new RemoteCodemodeSandbox({
remote: () => remote,
tools: [
{
name: "hang",
execute: () => {
started();
return new Promise(() => {});
},
},
],
});
const controller = new AbortController();
const pending = sandbox.execute("await tools.hang(); return 'never'", { signal: controller.signal });
await call;
controller.abort();
expect(await pending).toMatchObject({ ok: false, error: { kind: "aborted" } });
// The remote's pending exchange is answered with "stop", so its VM is disposed and execute() returns.
await Promise.all(served);
});
it("reports a failing or vanishing remote as a sandbox error", async () => {
const failing = new RemoteCodemodeSandbox({
remote: () => ({ execute: () => Promise.reject(new Error("object reset")) }),
});
expect(await failing.execute("return 1")).toMatchObject({
ok: false,
error: { kind: "sandbox", message: "Remote sandbox failed: object reset" },
});
const silent = new RemoteCodemodeSandbox({ remote: () => ({ execute: () => Promise.resolve() }) });
expect(await silent.execute("return 1")).toMatchObject({
ok: false,
error: { kind: "sandbox", message: "Remote sandbox returned before the script settled" },
});
const unreachable = new RemoteCodemodeSandbox({
remote: () => {
throw new Error("no binding");
},
});
expect(await unreachable.execute("return 1")).toMatchObject({
ok: false,
error: { kind: "sandbox", message: "Failed to reach the remote sandbox: no binding" },
});
});
});

View File

@@ -0,0 +1,869 @@
import { fileURLToPath } from "node:url";
import vm from "node:vm";
import { afterEach, beforeEach, describe, expect, it } from "vitest";
import {
CodemodeSandbox,
type CodemodeSandboxBase,
type CodemodeSandboxBaseOptions,
type CodemodeTool,
InlineCodemodeSandbox,
loadQuickJSWasm,
MAX_OUTPUT_CHARS,
MAX_OUTPUT_ITEMS,
RemoteCodemodeSandbox,
serveCodemodeRemote,
} from "../src/index.ts";
import { PRELUDE_SOURCE } from "../src/runtime/prelude-source.ts";
/**
* Every behavior test runs against each place a VM can run. The remote backend serves executions in
* process and structured-clones every message, as Workers RPC would. The in-thread backends get a
* small interrupt budget so runaway scripts end quickly.
*/
const INTERRUPT_BUDGET = 20_000;
const BACKENDS: Record<string, (options: CodemodeSandboxBaseOptions) => CodemodeSandboxBase> = {
worker: (options) => new CodemodeSandbox(options),
inline: (options) =>
new InlineCodemodeSandbox({ ...options, wasm: loadQuickJSWasm(), interruptBudget: INTERRUPT_BUDGET }),
remote: (options) =>
new RemoteCodemodeSandbox({
...options,
remote: () => ({
execute: (request, exchange) =>
serveCodemodeRemote(
structuredClone(request),
async (messages) => structuredClone(await exchange(structuredClone(messages))),
{ wasm: loadQuickJSWasm(), maxInterruptBudget: INTERRUPT_BUDGET },
),
}),
}),
};
let backend = "worker";
const sandboxes: CodemodeSandboxBase[] = [];
function newSandbox(options: CodemodeSandboxBaseOptions = {}): CodemodeSandboxBase {
const sandbox = BACKENDS[backend](options);
sandboxes.push(sandbox);
return sandbox;
}
function createSandbox(tools: CodemodeTool[] = [], timeoutMs = 10_000): CodemodeSandboxBase {
return newSandbox({ tools, timeoutMs });
}
afterEach(async () => {
await Promise.all(sandboxes.splice(0).map((sandbox) => sandbox.close()));
});
const echo: CodemodeTool = { name: "echo", execute: (args) => args };
// Base64 of the leading bytes of each format. image() only inspects the signature.
const PNG = "iVBORw0KGgo=";
const JPEG = "/9j/4A==";
const GIF = "R0lGODlh";
const WEBP = "UklGRgAAAABXRUJQ";
describe("embedded sources", () => {
it("parse as JavaScript", () => {
expect(() => new vm.Script(PRELUDE_SOURCE, { filename: "prelude.js" })).not.toThrow();
});
});
describe.each(Object.keys(BACKENDS))("%s backend", (name) => {
beforeEach(() => {
backend = name;
});
describe("script execution", () => {
it("returns the script's return value after a JSON round trip", async () => {
const sandbox = createSandbox();
expect(await sandbox.execute("return { a: 1, b: [true, 'x'] }")).toMatchObject({
ok: true,
value: { a: 1, b: [true, "x"] },
output: [],
calls: [],
});
expect(await sandbox.execute("return 'plain'")).toMatchObject({ ok: true, value: "plain" });
expect(await sandbox.execute("")).toMatchObject({ ok: true, value: undefined });
});
it("supports top-level await", async () => {
const sandbox = createSandbox();
const result = await sandbox.execute("const x = await Promise.resolve(41); return x + 1");
expect(result).toMatchObject({ ok: true, value: 42 });
});
it("collects text(), image(), and console output in order", async () => {
const sandbox = createSandbox();
const result = await sandbox.execute(`
console.log("hello", 1, { a: 1 });
text({ json: true });
text(undefined);
text(7);
image("data:image/png;base64,${PNG}");
image({ image_url: "data:image/jpeg;base64,${JPEG}" });
image({ type: "image", data: "${GIF}", mimeType: "image/gif" });
image("data:image/png;base64,${WEBP}");
image({ type: "image", data: "${PNG}" });
console.error(new Error("bad"));
return null;
`);
expect(result.ok).toBe(true);
expect(result.output.slice(0, -1)).toEqual([
{ type: "text", text: 'hello 1 {"a":1}', console: true },
{ type: "text", text: '{"json":true}' },
{ type: "text", text: "undefined" },
{ type: "text", text: "7" },
{ type: "image", data: PNG, mimeType: "image/png" },
{ type: "image", data: JPEG, mimeType: "image/jpeg" },
{ type: "image", data: GIF, mimeType: "image/gif" },
// The MIME type comes from the data, not from the declared type.
{ type: "image", data: WEBP, mimeType: "image/webp" },
{ type: "image", data: PNG, mimeType: "image/png" },
]);
expect(result.output.at(-1)).toMatchObject({
type: "text",
text: expect.stringMatching(/^Error: bad/),
console: true,
});
});
it("rejects invalid text() and image() arguments", async () => {
const sandbox = createSandbox();
const result = await sandbox.execute(`
const errors = [];
const circular = {};
circular.self = circular;
for (const run of [
() => text(circular),
() => image(""),
() => image("https://example.com/a.png"),
() => image("data:image/png,raw"),
() => image({ type: "text", text: "x" }),
() => image({ type: "image", data: "" }),
() => image(42),
() => image("data:image/png;base64,AAAA!"),
() => image("data:image/png;base64,AAAAA"),
() => image("data:image/png;base64,AA=A"),
() => image("data:image/png;base64,"),
() => image("data:image/png;base64,AAAA\\n[Output truncated]"),
() => image({ type: "image", data: "AAAA!", mimeType: "image/png" }),
() => image("data:image/png;base64,AAAA"),
() => image("data:image/png;base64,QUJD"),
() => image("data:image/jpeg;base64,/9j/9w=="),
]) {
try { run(); errors.push("no error"); } catch (error) { errors.push(error.name + ": " + error.message); }
}
return errors;
`);
expect(result.ok).toBe(true);
if (!result.ok) return;
expect(result.output).toEqual([]);
const errors = result.value as string[];
expect(errors[0]).toMatch(/^TypeError: .*circular/);
expect(errors.slice(1)).toEqual([
'TypeError: image expects a non-empty image URL string, an object with image_url, or an image block { type: "image", data, mimeType }',
"TypeError: remote image URLs are not supported in tool outputs. Pass a base64 data URI instead",
"TypeError: invalid image output. Pass a base64 data URI instead",
'TypeError: image only accepts image blocks, got "text"',
"TypeError: image expected image block data",
'TypeError: image expects a non-empty image URL string, an object with image_url, or an image block { type: "image", data, mimeType }',
...Array(6).fill(
"TypeError: invalid image output. The image data is not valid base64 (truncated or corrupted?)",
),
...Array(3).fill("TypeError: invalid image output. The image data is not a PNG, JPEG, GIF, or WebP image"),
]);
});
// https://github.com/earendil-works/pi/issues/10215
it("accepts wrapped and large base64 image data", async () => {
const sandbox = createSandbox();
const large = `iVBORw0KGgoA${"QUJD".repeat(256 * 1024)}`;
const result = await sandbox.execute(`
image("data:image/png;base64,iVBORw0K\\r\\nGgo=\\n");
image("data:image/png;base64,${large}");
`);
expect(result.ok).toBe(true);
expect(result.output).toEqual([
{ type: "image", data: PNG, mimeType: "image/png" },
{ type: "image", data: large, mimeType: "image/png" },
]);
});
it("ends the script successfully on exit(), keeping output and store writes", async () => {
const sandbox = createSandbox([echo]);
const result = await sandbox.execute(`
text("before");
store("k", 1);
await tools.echo(1);
try { exit(); } catch {}
text("after");
return "unreachable";
`);
expect(result).toMatchObject({
ok: true,
value: undefined,
output: [{ type: "text", text: "before" }],
storeWrites: { set: { k: 1 }, delete: [] },
});
});
it("keeps output produced before a failure", async () => {
const sandbox = createSandbox();
const result = await sandbox.execute('text("partial");\nthrow new Error("boom")');
expect(result).toMatchObject({ ok: false, output: [{ type: "text", text: "partial" }] });
});
it("reports syntax errors with the script's line number", async () => {
const sandbox = createSandbox();
const result = await sandbox.execute("const a = 1;\nconst b = ;\nreturn a");
expect(result.ok).toBe(false);
if (result.ok) return;
expect(result.error.kind).toBe("script");
expect(result.error.name).toBe("SyntaxError");
expect(result.error.stack).toMatch(/codemode\.js:2/);
});
it("reports thrown errors with the script's line number", async () => {
const sandbox = createSandbox();
const result = await sandbox.execute("const a = 1;\nthrow new TypeError('boom ' + a)");
expect(result.ok).toBe(false);
if (result.ok) return;
expect(result.error).toMatchObject({ kind: "script", name: "TypeError", message: "boom 1" });
expect(result.error.stack).toMatch(/codemode\.js:2/);
});
it("formats stacks like V8 without prelude frames", async () => {
const sandbox = createSandbox();
const result = await sandbox.execute("console.log(new Error('inner'));\nthrow new RangeError('outer')");
expect(result.ok).toBe(false);
if (result.ok) return;
expect(result.error.stack).toMatch(/^RangeError: outer\n {4}at .*codemode\.js:2/);
expect(result.error.stack).not.toContain("codemode-prelude.js");
expect(result.output[0]).toMatchObject({
type: "text",
text: expect.stringMatching(/^Error: inner\n {4}at .*codemode\.js:1/),
});
expect(JSON.stringify(result.output)).not.toContain("codemode-prelude.js");
});
it("reports non-Error throws", async () => {
const sandbox = createSandbox();
const result = await sandbox.execute("throw { code: 7 }");
expect(result).toMatchObject({ ok: false, error: { kind: "script", message: '{"code":7}' } });
});
it("reports a non-serializable return value as a script error", async () => {
const sandbox = createSandbox();
const result = await sandbox.execute("return 10n");
expect(result).toMatchObject({ ok: false, error: { kind: "script", name: "TypeError" } });
});
});
describe("tools", () => {
it("exposes tools as async functions and records calls", async () => {
const seen: unknown[] = [];
const sandbox = createSandbox([
{
name: "add",
execute: (args) => {
seen.push(args);
const { a, b } = args as { a: number; b: number };
return { sum: a + b };
},
},
]);
const result = await sandbox.execute(`
const first = await tools.add({ a: 1, b: 2 });
const second = await tools.add({ a: first.sum, b: 10 });
return second.sum;
`);
expect(result).toMatchObject({ ok: true, value: 13 });
expect(seen).toEqual([
{ a: 1, b: 2 },
{ a: 3, b: 10 },
]);
expect(result.calls.map((call) => [call.name, call.status])).toEqual([
["add", "ok"],
["add", "ok"],
]);
expect(result.calls.every((call) => call.durationMs >= 0)).toBe(true);
});
it("runs concurrent calls and lists tool names", async () => {
const sandbox = createSandbox([
echo,
{ name: "delay", execute: (args) => new Promise((resolve) => setTimeout(() => resolve(args), 20)) },
]);
const result = await sandbox.execute(`
const [a, b, c] = await Promise.all([tools.delay(1), tools.delay(2), tools.echo(3)]);
return { values: [a, b, c], names: Object.keys(tools) };
`);
expect(result).toMatchObject({ ok: true, value: { values: [1, 2, 3], names: ["echo", "delay"] } });
});
it("exposes tools under normalized identifiers and lists them in ALL_TOOLS", async () => {
const sandbox = createSandbox([
{ name: "my-tool", description: "Dashes", execute: () => "dash" },
{ name: "my_tool", description: "Shadowed", execute: () => "underscore" },
{ name: "mcp__docs__search", execute: () => "mcp" },
]);
const result = await sandbox.execute(`
try { ALL_TOOLS.push({}); } catch {}
return {
all: ALL_TOOLS,
calls: [await tools.my_tool(), await tools["my-tool"](), await tools.mcp__docs__search()],
};
`);
expect(result).toMatchObject({
ok: true,
value: {
all: [
{ name: "my_tool", description: "Dashes" },
{ name: "mcp__docs__search", description: "" },
],
calls: ["dash", "dash", "mcp"],
},
});
});
it("passes undefined arguments and results through", async () => {
const sandbox = createSandbox([{ name: "noop", execute: (args) => args }]);
const result = await sandbox.execute("return [await tools.noop(), await tools.noop(null)]");
expect(result).toMatchObject({ ok: true, value: [null, null] });
});
it("turns tool errors into catchable Errors in the script", async () => {
const sandbox = createSandbox([
{
name: "fail",
execute: () => {
throw new Error("tool exploded");
},
},
]);
const result = await sandbox.execute(`
try {
await tools.fail();
return "no error";
} catch (error) {
return { isError: error instanceof Error, message: error.message };
}
`);
expect(result).toMatchObject({ ok: true, value: { isError: true, message: "tool exploded" } });
expect(result.calls).toMatchObject([{ name: "fail", status: "error" }]);
});
it("rejects calls to unknown tools", async () => {
const sandbox = createSandbox();
const result = await sandbox.execute("return await tools.missing()");
expect(result).toMatchObject({ ok: false, error: { kind: "script", name: "TypeError" } });
});
it("aborts unawaited calls when the script returns", async () => {
let aborted = false;
const sandbox = createSandbox([
{
name: "slow",
execute: (_args, { signal }) =>
new Promise((_resolve, reject) => {
signal.addEventListener("abort", () => {
aborted = true;
reject(new Error("aborted"));
});
}),
},
]);
const result = await sandbox.execute("tools.slow(); return 'early'");
expect(result).toMatchObject({ ok: true, value: "early" });
expect(result.calls).toMatchObject([{ name: "slow", status: "cancelled" }]);
expect(aborted).toBe(true);
});
it("names close matches when a script reads a tool that does not exist", async () => {
const sandbox = createSandbox([echo, { name: "web-search", execute: () => "" }]);
const attempt = async (expression: string) => {
const result = await sandbox.execute(`return ${expression};`);
return result.ok ? result.value : result.error.message;
};
expect(await attempt("tools.Echo")).toBe(
'tools.Echo does not exist. Did you mean tools.echo? ALL_TOOLS lists every tool; searchTools(query) finds tools by topic. Check for a member with "Echo" in tools.',
);
expect(await attempt("tools.websearch")).toContain("Did you mean tools.web_search?");
expect(await attempt("tools.nothing")).toContain("Available: echo, web_search.");
expect(
await attempt("['echo' in tools, 'nothing' in tools, String(tools.toString), JSON.stringify(tools)]"),
).toEqual([true, false, "undefined", "{}"]);
});
it("supports register and unregister between executions", async () => {
const sandbox = createSandbox();
sandbox.registerTool(echo);
expect(() => sandbox.registerTool(echo)).toThrow(/already registered/);
expect(sandbox.tools.map((tool) => tool.name)).toEqual(["echo"]);
expect(await sandbox.execute("return await tools.echo('a')")).toMatchObject({ ok: true, value: "a" });
expect(sandbox.unregisterTool("echo")).toBe(true);
expect(await sandbox.execute("return 'echo' in tools")).toMatchObject({ ok: true, value: false });
});
});
describe("store and load", () => {
it("reads the snapshot and reports writes", async () => {
const sandbox = createSandbox();
const result = await sandbox.execute(
`
const seen = load("counter");
store("counter", seen + 1);
store("list", [1, { a: null }]);
store("old", undefined);
return [seen, load("counter"), load("missing"), load("old")];
`,
{ store: { counter: 41, old: "x" } },
);
expect(result).toMatchObject({
ok: true,
// undefined array elements become null in the JSON round trip of the return value.
value: [41, 42, null, null],
storeWrites: { set: { counter: 42, list: [1, { a: null }] }, delete: ["old"] },
});
});
it("returns copies, so mutating a loaded value does not change the store", async () => {
const sandbox = createSandbox();
const result = await sandbox.execute(
`const value = load("obj"); value.a = 2; const kept = { b: 1 }; store("kept", kept); kept.b = 2;
return [load("obj").a, load("kept").b];`,
{ store: { obj: { a: 1 } } },
);
expect(result).toMatchObject({ ok: true, value: [1, 1], storeWrites: { set: { kept: { b: 1 } } } });
});
it("rejects invalid keys, values, and oversized writes inside the script", async () => {
const sandbox = createSandbox();
const result = await sandbox.execute(`
const attempt = (fn) => { try { fn(); return "ok"; } catch (error) { return error.name; } };
return [
attempt(() => store(1, "x")),
attempt(() => load({})),
attempt(() => store("fn", () => 1)),
attempt(() => store("big", "x".repeat(300 * 1024))),
attempt(() => { for (let i = 0; i < 8; i++) store("k" + i, "x".repeat(200 * 1024)); }),
];
`);
expect(result).toMatchObject({
ok: true,
value: ["TypeError", "TypeError", "TypeError", "RangeError", "RangeError"],
});
});
it("explains oversized writes", async () => {
const sandbox = createSandbox();
const result = await sandbox.execute(`store("img", "x".repeat(300 * 1024));`);
expect(result.ok).toBe(false);
if (result.ok) return;
expect(result.error.message).toContain('store("img") value has 307202 characters of JSON');
expect(result.error.message).toContain("Show images with image()");
});
it("reserves the store and load names", () => {
const execute = () => undefined;
expect(() => newSandbox({ globals: [{ name: "store", execute }] })).toThrow(/Invalid global/);
expect(() => newSandbox({ globals: [{ name: "load", execute }] })).toThrow(/Invalid global/);
});
});
describe("globals", () => {
it("exposes globals as top-level functions without recording them as calls", async () => {
const seen: unknown[] = [];
const sandbox = newSandbox({
tools: [echo],
globals: [{ name: "attach", execute: (args) => void seen.push(args) }],
});
const result = await sandbox.execute(`
await attach({ ref: 1 });
attach("not awaited");
return [typeof attach, typeof globalThis.attach, await tools.echo(2)];
`);
expect(result).toMatchObject({ ok: true, value: ["function", "function", 2] });
expect(result.calls.map((call) => call.name)).toEqual(["echo"]);
// Messages are handled in order, so an unawaited global still runs before the script settles.
expect(seen).toEqual([{ ref: 1 }, "not awaited"]);
});
it("groups namespaced globals and spreads arguments on request", async () => {
const seen: unknown[] = [];
const sandbox = newSandbox({
globals: [
{ name: "models.list", spread: true, execute: (args) => void seen.push(args) },
{ name: "models.first", execute: (args) => args },
],
});
const result = await sandbox.execute(`
await models.list("classifier", undefined, 3);
await models.list();
try { models.extra = 1; } catch {}
return [Object.keys(models), await models.first("a", "ignored"), "extra" in models];
`);
expect(result).toMatchObject({ ok: true, value: [["list", "first"], "a", false] });
// undefined array elements become null in the JSON round trip.
expect(seen).toEqual([["classifier", null, 3], []]);
});
it("names the members of a namespace when a script reads one that does not exist", async () => {
const execute = () => undefined;
const sandbox = newSandbox({
globals: [
{ name: "models.classify", execute },
{ name: "models.generateImages", execute },
],
});
const result = await sandbox.execute("await models.generateImage();");
expect(result.ok).toBe(false);
if (result.ok) return;
expect(result.error.message).toBe(
'models.generateImage does not exist. Did you mean models.generateImages? Check for a member with "generateImage" in models.',
);
});
it("rejects invalid and reserved global names", () => {
const execute = () => undefined;
for (const name of ["a.b.c", "a.", ".a", "tools.x", "store.x", "a.not-valid"]) {
expect(() => newSandbox({ globals: [{ name, execute }] }), name).toThrow(/Invalid global/);
}
expect(() =>
newSandbox({
globals: [
{ name: "models", execute },
{ name: "models.list", execute },
],
}),
).toThrow(/conflicts with the namespace/);
expect(() => newSandbox({ globals: [{ name: "not-valid", execute }] })).toThrow(/Invalid global/);
expect(() => newSandbox({ globals: [{ name: "tools", execute }] })).toThrow(/Invalid global/);
expect(() => newSandbox({ globals: [{ name: "console", execute }] })).toThrow(/Invalid global/);
});
});
describe("limits and lifetime", () => {
it("terminates a synchronous infinite loop on timeout", async () => {
const sandbox = createSandbox();
const started = performance.now();
const result = await sandbox.execute("while (true) {}", { timeoutMs: 200 });
expect(result).toMatchObject({ ok: false, error: { kind: "timeout" } });
expect(performance.now() - started).toBeLessThan(5_000);
});
it("runs without a deadline when timeoutMs is Infinity", async () => {
const sandbox = createSandbox([
{ name: "wait", execute: () => new Promise((r) => setTimeout(() => r("late"), 50)) },
]);
const result = await sandbox.execute("return await tools.wait()", { timeoutMs: Number.POSITIVE_INFINITY });
expect(result).toMatchObject({ ok: true, value: "late" });
});
it("fails a script that waits on a promise nothing can settle", async () => {
const sandbox = createSandbox([{ name: "echo", execute: (args) => args }]);
const result = await sandbox.execute("await tools.echo(1); await new Promise(() => {}); return 'never'", {
timeoutMs: Number.POSITIVE_INFINITY,
});
expect(result).toMatchObject({
ok: false,
error: { kind: "script", message: expect.stringContaining("can never settle") },
});
// Returning while a call is still pending is not a stall.
const returned = await sandbox.execute("tools.echo(2); return 'early'", {
timeoutMs: Number.POSITIVE_INFINITY,
});
expect(returned).toMatchObject({ ok: true, value: "early" });
});
it("terminates a microtask-spinning loop on timeout", async () => {
const sandbox = createSandbox();
const result = await sandbox.execute("while (true) await null", { timeoutMs: 200 });
expect(result).toMatchObject({ ok: false, error: { kind: "timeout" } });
});
it("aborts via signal and cancels in-flight calls", async () => {
const controller = new AbortController();
let toolSignal: AbortSignal | undefined;
// Resolves on each call, so the test waits for the worker instead of a fixed delay.
let called: () => void = () => {};
const nextCall = () =>
new Promise<void>((resolve) => {
called = resolve;
});
const sandbox = createSandbox([
{
name: "hang",
execute: (_args, { signal }) => {
toolSignal = signal;
called();
return new Promise(() => {});
},
},
]);
const firstCall = nextCall();
const promise = sandbox.execute("await tools.hang(); return 'never'");
await firstCall;
controller.abort(new Error("user cancelled"));
expect(toolSignal?.aborted).toBe(false);
const result = await sandbox.execute("await tools.hang()", { signal: controller.signal });
expect(result).toMatchObject({ ok: false, error: { kind: "aborted", message: "user cancelled" } });
const second = new AbortController();
const secondCall = nextCall();
const pending = sandbox.execute("await tools.hang(); return 'never'", { signal: second.signal });
await secondCall;
second.abort();
const aborted = await pending;
expect(aborted).toMatchObject({ ok: false, error: { kind: "aborted" } });
expect(aborted.calls).toMatchObject([{ name: "hang", status: "cancelled" }]);
expect(toolSignal?.aborted).toBe(true);
await sandbox.close();
expect(await promise).toMatchObject({ ok: false, error: { kind: "aborted", message: "Sandbox closed" } });
});
// #10283: the host keeps all output, so a script that prints in a loop must not grow it without bound.
it("fails a script whose output passes the limits, even if it catches the error", async () => {
const sandbox = createSandbox();
for (const print of ["text(s)", "console.log(s)", 'image("data:image/png;base64," + p)']) {
const result = await sandbox.execute(`
const s = "x".repeat(1 << 20);
const p = "iVBORw0KGgoA" + "A".repeat(1 << 20);
for (;;) { try { ${print}; } catch {} }
`);
expect(result).toMatchObject({
ok: false,
error: {
kind: "script",
name: "RangeError",
message: expect.stringContaining("script output exceeded"),
},
});
const chars = result.output.reduce(
(sum, item) => sum + (item.type === "text" ? item.text.length : item.data.length),
0,
);
expect(chars).toBeLessThanOrEqual(MAX_OUTPUT_CHARS);
expect(chars).toBeGreaterThan(MAX_OUTPUT_CHARS - (2 << 20));
}
const empty = await sandbox.execute(`for (;;) text("");`);
expect(empty).toMatchObject({ ok: false, error: { name: "RangeError" } });
expect(empty.output).toHaveLength(MAX_OUTPUT_ITEMS);
});
it("rejects execute after close", async () => {
const sandbox = createSandbox();
await sandbox.close();
await expect(sandbox.execute("return 1")).rejects.toThrow(/closed/);
});
it("runs executions in parallel without sharing state", async () => {
const sandbox = createSandbox();
const results = await Promise.all([
sandbox.execute("globalThis.shared = 'a'; await null; return globalThis.shared"),
sandbox.execute("globalThis.shared = 'b'; await null; return globalThis.shared"),
sandbox.execute("return typeof globalThis.shared"),
]);
expect(results.map((result) => (result.ok ? result.value : result.error))).toEqual(["a", "b", "undefined"]);
});
it("turns deep recursion into a catchable RangeError", async () => {
const sandbox = createSandbox();
const result = await sandbox.execute(`
let depth = 0;
function dive() { depth++; dive(); }
try { dive(); } catch (error) { return [error.name, depth > 1000]; }
`);
expect(result).toMatchObject({ ok: true, value: ["RangeError", true] });
});
});
describe("escape hatches", () => {
it("has no host globals", async () => {
const sandbox = createSandbox();
const result = await sandbox.execute(`
return [
typeof process, typeof require, typeof module, typeof setTimeout, typeof fetch,
typeof WebAssembly, typeof std, typeof os, typeof globalThis.constructor,
]
`);
expect(result).toMatchObject({
ok: true,
value: [
"undefined",
"undefined",
"undefined",
"undefined",
"undefined",
"undefined",
"undefined",
"undefined",
"function",
],
});
});
it("keeps eval and Function inside the VM", async () => {
// Code generation is allowed: it can only produce more code in the same wasm instance.
const sandbox = createSandbox([echo]);
const result = await sandbox.execute(`
return [
eval("typeof process"),
new Function("return typeof process")(),
tools.echo.constructor("return typeof require")(),
(async () => {}).constructor("return typeof setTimeout")() instanceof Promise,
];
`);
expect(result).toMatchObject({ ok: true, value: ["undefined", "undefined", "undefined", true] });
});
it("rejects dynamic import", async () => {
const sandbox = createSandbox();
const result = await sandbox.execute(`
try { await import("node:fs"); return "imported"; } catch (error) { return error.constructor.name; }
`);
expect(result.ok).toBe(true);
if (!result.ok) return;
expect(result.value).not.toBe("imported");
});
// #10444: the prelude shares the built-ins with the script, so patching them could corrupt what
// the prelude sends to the host.
it("ignores patches to built-ins and built-in globals", async () => {
const sandbox = createSandbox([echo]);
const result = await sandbox.execute(`
Array.prototype.toJSON = () => null;
Object.prototype.toJSON = () => 5;
Promise.prototype.then = () => {};
Map.prototype.get = () => undefined;
globalThis.JSON = { stringify: () => "x", parse: () => "x" };
store("k", [1]);
return [await tools.echo([2]), JSON.stringify({ a: 1 })];
`);
expect(result).toMatchObject({ ok: true, value: [[2], '{"a":1}'], storeWrites: { set: { k: [1] } } });
});
it("freezes intrinsics that are only reachable from instances", async () => {
const sandbox = createSandbox();
const result = await sandbox.execute(`
return [
Object.getPrototypeOf(function* () {}).prototype,
Object.getPrototypeOf(async function () {}),
Object.getPrototypeOf(Int8Array).prototype,
Object.getPrototypeOf([][Symbol.iterator]()),
Object.getPrototypeOf(Object.getPrototypeOf([][Symbol.iterator]())),
Object.getPrototypeOf(new Map()[Symbol.iterator]()),
Object.getPrototypeOf(/a/[Symbol.matchAll]("")),
].every((object) => Object.isFrozen(object));
`);
expect(result).toMatchObject({ ok: true, value: true });
});
it("still lets instances override properties of frozen prototypes", async () => {
const sandbox = createSandbox();
const result = await sandbox.execute(`
const object = {};
object.toString = () => "custom";
function Legacy() {}
Legacy.prototype = Object.create(Error.prototype);
Legacy.prototype.constructor = Legacy;
const bare = new Error();
bare.message = "set later";
class MyError extends Error {
constructor(message) {
super(message);
this.name = "MyError";
}
}
let patched = "silent";
try { Error.prototype.name = "Patched"; } catch (error) { patched = error.constructor.name; }
return [String(object), new Legacy().constructor === Legacy, bare.message, new MyError("x").name, Error.prototype.name, patched];
`);
expect(result).toMatchObject({
ok: true,
value: ["custom", true, "set later", "MyError", "Error", "TypeError"],
});
});
it("reports errors whose name or message is not a string", async () => {
const sandbox = createSandbox();
const result = await sandbox.execute("const error = new Error('x'); error.message = 42; throw error;");
expect(result).toMatchObject({ ok: false, error: { kind: "script", name: "Error", message: "42" } });
});
it("keeps tools and console frozen", async () => {
const sandbox = createSandbox([echo]);
const result = await sandbox.execute(`
try { tools.echo = () => 'nope'; } catch {}
try { tools.extra = () => 'nope'; } catch {}
try { globalThis.tools = null; } catch {}
return ["extra" in tools, await tools.echo('still')];
`);
expect(result).toMatchObject({ ok: true, value: [false, "still"] });
});
});
});
describe("worker host", () => {
beforeEach(() => {
backend = "worker";
});
it("accepts a worker path string", async () => {
const workerPath = fileURLToPath(new URL("../src/runtime/worker.ts", import.meta.url));
const sandbox = new CodemodeSandbox({ workerUrl: workerPath });
sandboxes.push(sandbox);
expect(await sandbox.execute("return 1")).toMatchObject({ ok: true, value: 1 });
});
it("reports a missing worker file as a sandbox error", async () => {
const sandbox = new CodemodeSandbox({ workerUrl: new URL("./does-not-exist.js", import.meta.url) });
sandboxes.push(sandbox);
expect(await sandbox.execute("return 1")).toMatchObject({ ok: false, error: { kind: "sandbox" } });
});
it("reports a failing wasm module as a sandbox error", async () => {
const sandbox = new CodemodeSandbox({ wasm: Promise.reject(new Error("no wasm")) });
sandboxes.push(sandbox);
expect(await sandbox.execute("return 1")).toMatchObject({
ok: false,
error: { kind: "sandbox", message: "Failed to load QuickJS: no wasm" },
});
});
// #10725: Node 24/26 `--watch` posts dependency reports on the worker channel.
it("ignores Node --watch notifications from the worker", async () => {
const sandbox = new CodemodeSandbox({ workerUrl: new URL("./fixtures/raw-worker.ts", import.meta.url) });
sandboxes.push(sandbox);
const result = await sandbox.execute(
JSON.stringify([
{ "watch:require": ["/worker.js"] },
{ "watch:import": ["file:///worker.js"] },
{ type: "done", ok: true, value: "42", writes: "[]" },
]),
);
expect(result).toMatchObject({ ok: true, value: 42 });
});
// #10444: a malformed payload from the worker must fail the run as a sandbox error instead of
// throwing in the host's message listener and never settling.
it.each([
[{ type: "done", ok: true, value: "1", writes: "null" }, "store writes are not an array"],
[{ type: "done", ok: true, value: "1", writes: "[1]" }, "store writes contain a malformed entry"],
[{ type: "done", ok: true, value: "1", writes: '[["k", "{"]]' }, 'store value for "k" is not valid JSON'],
[{ type: "done", ok: true, value: "{", writes: "[]" }, "return value is not valid JSON"],
[{ type: "done", ok: false, error: "5" }, "script error is not an object"],
[{ type: "done", ok: false, error: "{}" }, "script error is malformed"],
[{ type: "nonsense" }, "unknown message from the worker"],
])("reports a broken bridge as a sandbox error: %j", async (message, reason) => {
const sandbox = new CodemodeSandbox({ workerUrl: new URL("./fixtures/raw-worker.ts", import.meta.url) });
sandboxes.push(sandbox);
const result = await sandbox.execute(JSON.stringify(message));
expect(result).toMatchObject({ ok: false, error: { kind: "sandbox" } });
expect(result.ok ? undefined : result.error.message).toContain(`Sandbox bridge broken: ${reason}`);
});
});

View File

@@ -0,0 +1,78 @@
import { describe, expect, it } from "vitest";
import { CodemodeSourceError, parseCodemodeSource } from "../src/source.ts";
describe("parseCodemodeSource", () => {
it("returns plain code unchanged", () => {
expect(parseCodemodeSource("text('hi')")).toEqual({ code: "text('hi')", options: {} });
expect(parseCodemodeSource("// just a comment\nreturn 1")).toEqual({
code: "// just a comment\nreturn 1",
options: {},
});
});
it("parses the options line and keeps line numbers", () => {
expect(parseCodemodeSource('// @options: {"timeout_ms": 10}\nconst a = 1;\ntext(a)')).toEqual({
code: "\nconst a = 1;\ntext(a)",
options: { timeoutMs: 10 },
});
expect(parseCodemodeSource(' // @options:{"max_output_tokens":0,"timeout_ms":1500}\r\ntext(1)').options).toEqual(
{
maxOutputTokens: 0,
timeoutMs: 1500,
},
);
expect(parseCodemodeSource("// @options: {}\ntext(1)")).toEqual({ code: "\ntext(1)", options: {} });
});
it("only treats the first line as an options line", () => {
expect(parseCodemodeSource("// @optionsx {}\ntext(1)").options).toEqual({});
expect(parseCodemodeSource('text("// @options: {}")').options).toEqual({});
});
// #10631: an options line after the first line used to be ignored, so its timeout never applied.
it("rejects an options line that is not the first line", () => {
const cases: [string, number][] = [
['// count things\n// @options: {"timeout_ms": 1000}\nwhile (true) {}', 2],
['\n // @options: {"timeout_ms": 1000}\nwhile (true) {}', 2],
['text(1)\r\n// @options: {"timeout_ms": 1}', 2],
['// @options: {"timeout_ms": 1}\ntext(1)\n// @options: {"timeout_ms": 2}', 3],
];
for (const [input, line] of cases) {
expect(() => parseCodemodeSource(input), input).toThrow(CodemodeSourceError);
expect(() => parseCodemodeSource(input), input).toThrow(
`The // @options: line must be the first line of the input; found one on line ${line}`,
);
}
});
it("rejects empty input and invalid options", () => {
const cases: [string, string | RegExp][] = [
["", /Expected JavaScript source text \(non-empty\)/],
[" \n", /Expected JavaScript source text \(non-empty\)/],
["// @options:\ntext(1)", /@options must be a JSON object with supported fields/],
["// @options: {timeout_ms: 1}\ntext(1)", /@options must be valid JSON with supported fields/],
["// @options: [1]\ntext(1)", /@options must be a JSON object with supported fields/],
[
'// @options: {"yield": 1}\ntext(1)',
"@options only supports `max_output_tokens` and `timeout_ms`; got `yield`",
],
[
'// @options: {"max_output_tokens": 1.5}\ntext(1)',
"@options field `max_output_tokens` must be a non-negative safe integer",
],
['// @options: {"timeout_ms": 0}\ntext(1)', /@options field `timeout_ms` must be a positive integer/],
[
'// @options: {"timeout_ms": 1}',
"The @options line must be followed by JavaScript source on subsequent lines",
],
[
'// @options: {"timeout_ms": 1}\n \n',
"The @options line must be followed by JavaScript source on subsequent lines",
],
];
for (const [input, message] of cases) {
expect(() => parseCodemodeSource(input), input).toThrow(CodemodeSourceError);
expect(() => parseCodemodeSource(input), input).toThrow(message);
}
});
});

View File

@@ -0,0 +1,229 @@
import { execFile, spawnSync } from "node:child_process";
import { fileURLToPath } from "node:url";
import { promisify } from "node:util";
import { Worker } from "node:worker_threads";
import { afterEach, describe, expect, it, vi } from "vitest";
import { type CodemodeResult, CodemodeSandbox } from "../src/index.ts";
afterEach(() => {
vi.useRealTimers();
vi.restoreAllMocks();
});
describe("worker lifetime", () => {
// #10748: reject admission before creating a worker, not by terminating an excess worker.
it("does not start a worker when admission is denied", async () => {
const onWorkerStarted = vi.fn();
const acquireWorker = vi.fn(() => {
throw new Error("worker capacity exhausted");
});
const sandbox = new CodemodeSandbox({ acquireWorker, onWorkerStarted });
try {
expect(await sandbox.execute("while (true) {}")).toMatchObject({
ok: false,
error: { kind: "sandbox", message: "worker capacity exhausted" },
});
expect(acquireWorker).toHaveBeenCalledTimes(1);
expect(onWorkerStarted).not.toHaveBeenCalled();
} finally {
await sandbox.close();
}
});
// #10748: a failed Worker constructor must return its reserved capacity.
it("releases admission when worker construction fails", async () => {
const releaseWorker = vi.fn();
const acquireWorker = vi.fn(() => releaseWorker);
const sandbox = new CodemodeSandbox({
acquireWorker,
workerUrl: new URL("https://invalid.example/worker.js"),
});
try {
expect(await sandbox.execute("return 1")).toMatchObject({
ok: false,
error: { kind: "sandbox", message: expect.stringContaining("Failed to start worker:") },
});
expect(acquireWorker).toHaveBeenCalledTimes(1);
expect(releaseWorker).toHaveBeenCalledTimes(1);
} finally {
await sandbox.close();
}
});
// #10748: loading failures must not reserve worker capacity.
it("does not acquire admission when wasm loading fails", async () => {
const acquireWorker = vi.fn(() => vi.fn());
const sandbox = new CodemodeSandbox({
acquireWorker,
wasm: Promise.reject(new Error("wasm unavailable")),
});
try {
expect(await sandbox.execute("return 1")).toMatchObject({
ok: false,
error: { kind: "sandbox", message: "Failed to load QuickJS: wasm unavailable" },
});
expect(acquireWorker).not.toHaveBeenCalled();
} finally {
await sandbox.close();
}
});
// #10748: Bun can leave terminate() pending while a non-interruptible built-in runs.
it.each(["abort", "timeout", "close"] as const)("settles %s without waiting for worker exit", async (how) => {
vi.useFakeTimers();
const terminate = Worker.prototype.terminate;
let worker: Worker | undefined;
let releaseTermination: (code: number) => void = () => {};
const termination = new Promise<number>((resolve) => {
releaseTermination = resolve;
});
const terminateSpy = vi.spyOn(Worker.prototype, "terminate").mockImplementation(function (this: Worker) {
worker = this;
// Like the real Node/Bun implementation, termination refs the worker while waiting for exit.
this.ref();
return termination;
});
const refSpy = vi.spyOn(Worker.prototype, "ref");
const unrefSpy = vi.spyOn(Worker.prototype, "unref");
let ready: () => void = () => {};
const called = new Promise<void>((resolve) => {
ready = resolve;
});
let toolSignal: AbortSignal | undefined;
let exited: Promise<void> | undefined;
let didExit = false;
const sandbox = new CodemodeSandbox({
timeoutMs: 100,
onWorkerStarted: (promise) => {
exited = promise;
void promise.then(() => {
didExit = true;
});
},
tools: [
{
name: "hang",
execute: (_args, { signal }) => {
toolSignal = signal;
ready();
return new Promise(() => {});
},
},
],
});
const controller = new AbortController();
let result: CodemodeResult | undefined;
let didClose = false;
const pending = sandbox.execute('text("partial"); await tools.hang()', { signal: controller.signal });
void pending.then((value) => {
result = value;
});
try {
await called;
if (how === "abort") controller.abort(new Error("cancelled"));
if (how === "timeout") await vi.advanceTimersByTimeAsync(100);
if (how === "close") {
void sandbox.close().then(() => {
didClose = true;
});
}
await vi.advanceTimersByTimeAsync(0);
expect(result).toMatchObject({
ok: false,
error: { kind: how === "timeout" ? "timeout" : "aborted" },
output: [{ type: "text", text: "partial" }],
calls: [{ name: "hang", status: "cancelled" }],
});
expect(toolSignal?.aborted).toBe(true);
expect(terminateSpy).toHaveBeenCalledTimes(1);
expect(unrefSpy).toHaveBeenCalledTimes(1);
expect(refSpy).toHaveBeenCalledTimes(1);
expect(unrefSpy.mock.invocationCallOrder[0]).toBeGreaterThan(refSpy.mock.invocationCallOrder[0]);
expect(exited).toBeDefined();
expect(didExit).toBe(false);
if (how === "close") expect(didClose).toBe(true);
// Neither close() nor late worker messages may replace the decided outcome or wait for exit.
await sandbox.close();
worker?.emit("message", { type: "done", ok: true, value: "42", writes: "[]" });
expect(await pending).toBe(result);
expect(terminateSpy).toHaveBeenCalledTimes(1);
} finally {
vi.useRealTimers();
terminateSpy.mockRestore();
if (worker) releaseTermination(await terminate.call(worker));
await sandbox.close();
}
await exited;
expect(didExit).toBe(true);
});
// #10748: exit accounting applies to successful executions and host-controlled runs too.
it("reports the actual exit of execute() and start() workers", async () => {
const exits: Promise<void>[] = [];
const releaseWorker = vi.fn();
const acquireWorker = vi.fn(() => releaseWorker);
const sandbox = new CodemodeSandbox({ acquireWorker, onWorkerStarted: (exited) => void exits.push(exited) });
try {
expect(await sandbox.execute("return 1")).toMatchObject({ ok: true, value: 1 });
const run = sandbox.start("return 2", { startedAt: 0, seed: 0, onCall: () => {} });
expect(await run.result).toMatchObject({ ok: true, value: 2 });
expect(run.finished).toBe(true);
expect(exits).toHaveLength(2);
await Promise.all(exits);
expect(acquireWorker).toHaveBeenCalledTimes(2);
expect(releaseWorker).toHaveBeenCalledTimes(2);
} finally {
await sandbox.close();
}
});
// #10748: a failing lifetime hook must not leak the worker it was handed.
it("requests termination when onWorkerStarted throws", async () => {
let exited: Promise<void> | undefined;
const releaseWorker = vi.fn();
const sandbox = new CodemodeSandbox({
acquireWorker: () => releaseWorker,
onWorkerStarted: (promise) => {
exited = promise;
throw new Error("lifetime hook failed");
},
});
try {
expect(await sandbox.execute("while (true) {}")).toMatchObject({
ok: false,
error: { kind: "sandbox", message: "lifetime hook failed" },
});
expect(exited).toBeDefined();
await exited;
expect(releaseWorker).toHaveBeenCalledTimes(1);
} finally {
await sandbox.close();
}
});
});
const runtimes = [
{ name: "Node", command: process.execPath, available: true },
{ name: "Bun", command: "bun", available: spawnSync("bun", ["--version"], { timeout: 5000 }).status === 0 },
];
describe.each(runtimes)("$name process lifetime", ({ name, command, available }) => {
// #10748: verify both prompt cancellation and, on Node, prompt process exit with a real worker.
// Bun 1.4.0's pending terminate() keeps the process alive even after unref(), independently of codemode.
it.skipIf(!available).each(["abort", "timeout", "close"] as const)(
name === "Node" ? "exits promptly after %s with a real worker" : "settles %s promptly with a real worker",
async (how) => {
const fixture = fileURLToPath(new URL("./fixtures/worker-process-exit.ts", import.meta.url));
const { stdout } = await promisify(execFile)(command, [fixture, how], { timeout: 20000 });
const lines = stdout.trim().split(/\r?\n/);
expect(lines).toHaveLength(2);
const result = JSON.parse(lines[0]) as { type: string; kind: string; elapsedMs: number };
const exit = JSON.parse(lines[1]) as { type: string; elapsedMs: number };
expect(result).toMatchObject({ type: "result", kind: how === "timeout" ? "timeout" : "aborted" });
expect(exit.type).toBe("exit");
expect(result.elapsedMs).toBeLessThan(1000);
if (name === "Node") expect(exit.elapsedMs - result.elapsedMs).toBeLessThan(1000);
},
25000,
);
});

View File

@@ -0,0 +1,11 @@
{
"extends": "../../tsconfig.base.json",
"compilerOptions": {
"outDir": "./dist",
"rootDir": "./src"
},
"include": ["src/**/*.ts"],
// Ambient declarations the build needs; `exclude` below does not apply to `files`.
"files": ["src/cloudflare-modules.d.ts"],
"exclude": ["node_modules", "dist", "**/*.d.ts", "src/**/*.d.ts"]
}

View File

@@ -0,0 +1,11 @@
import { defineConfig } from "vitest/config";
export default defineConfig({
test: {
globals: true,
environment: "node",
reporters: process.env.GITHUB_ACTIONS ? ["dot", "github-actions"] : ["dot"],
},
resolve: { conditions: ["source"] },
ssr: { resolve: { conditions: ["source"] } },
});