chore(vendor): import pi 1.3.0
Imported as a linear vendor commit so forge rebases do not replay the upstream root tree. git-subtree-dir: third_party/pi git-subtree-split: 71023aec1546d6d677a65669c5665c0927ce0dd0
This commit is contained in:
alexion committed
1 parent
719370c8f7
commit
6081e3e325
1743 files changed
+205459
-131704
No files matched your search
Vendored
+278
-71
@@ -2,7 +2,7 @@
|
||||
|
||||
Unified LLM API with provider collections, automatic auth resolution, token and cost tracking, and simple context persistence and hand-off to other models mid-session.
|
||||
|
||||
**Note**: This library only includes models that support tool calling (function calling), as this is essential for agentic workflows.
|
||||
**Note**: The chat catalog only includes models that support tool calling (function calling), as this is essential for agentic workflows. Image and classifier catalogs use their operation-specific capabilities.
|
||||
|
||||
## Table of Contents
|
||||
|
||||
@@ -26,9 +26,9 @@ Unified LLM API with provider collections, automatic auth resolution, token and
|
||||
- [Streaming Tool Calls with Partial JSON](#streaming-tool-calls-with-partial-json)
|
||||
- [Validating Tool Arguments](#validating-tool-arguments)
|
||||
- [Complete Event Reference](#complete-event-reference)
|
||||
- [Compact Assistant Message Frames](#compact-assistant-message-frames)
|
||||
- [Image Input](#image-input)
|
||||
- [Image Generation](#image-generation)
|
||||
- [Classification](#classification)
|
||||
- [Thinking/Reasoning](#thinkingreasoning)
|
||||
- [Unified Interface](#unified-interface-streamsimplecompletesimple)
|
||||
- [Provider-Specific Options](#provider-specific-options-streamcomplete)
|
||||
@@ -38,12 +38,14 @@ Unified LLM API with provider collections, automatic auth resolution, token and
|
||||
- [Aborting Requests](#aborting-requests)
|
||||
- [Continuing After Abort](#continuing-after-abort)
|
||||
- [Debugging Provider Payloads](#debugging-provider-payloads)
|
||||
- [Observing Provider Stream Events](#observing-provider-stream-events)
|
||||
- [Custom Providers](#custom-providers)
|
||||
- [createProvider()](#createprovider)
|
||||
- [Calling API Implementations Directly](#calling-api-implementations-directly)
|
||||
- [OpenAI Compatibility Settings](#openai-compatibility-settings)
|
||||
- [Faux Provider for Tests](#faux-provider-for-tests)
|
||||
- [Cross-Provider Handoffs](#cross-provider-handoffs)
|
||||
- [System Messages](#system-messages)
|
||||
- [Context Serialization](#context-serialization)
|
||||
- [Browser Usage](#browser-usage)
|
||||
- [Bundling and Tree Shaking](#bundling-and-tree-shaking)
|
||||
@@ -57,10 +59,12 @@ Unified LLM API with provider collections, automatic auth resolution, token and
|
||||
|
||||
## Supported Providers
|
||||
|
||||
- **OpenAI**
|
||||
- **OpenAI** (including the Decisions classifier API)
|
||||
- **Ant Ling**
|
||||
- **Azure OpenAI (Responses)**
|
||||
- **OpenAI Codex** (ChatGPT Plus/Pro subscription, requires OAuth, see below)
|
||||
- **OpenAI Codex (legacy)** (ChatGPT Plus/Pro subscription, requires OAuth, see below)
|
||||
- **Radius** (API key or OAuth, with a dynamically refreshed gateway catalog)
|
||||
- **TypeSafe** (System One classifier API)
|
||||
- **DeepSeek**
|
||||
- **NVIDIA NIM**
|
||||
- **Anthropic**
|
||||
@@ -86,6 +90,7 @@ Unified LLM API with provider collections, automatic auth resolution, token and
|
||||
- **OpenCode Go**
|
||||
- **Fireworks** (uses OpenAI- and Anthropic-compatible APIs)
|
||||
- **Kimi For Coding** (Moonshot AI subscription endpoint, uses Anthropic-compatible API)
|
||||
- **Meta** (Model API, uses OpenAI Responses-compatible API)
|
||||
- **Qwen Token Plan** (separate Individual and existing catalogs, with a separate China provider)
|
||||
- **Xiaomi MiMo** (defaults to API billing endpoint, with separate Token Plan providers for `cn`/`ams`/`sgp` regions)
|
||||
- **Any OpenAI-compatible API**: Ollama, vLLM, LM Studio, etc.
|
||||
@@ -272,7 +277,7 @@ Reads are synchronous and return the last-known lists:
|
||||
const providers = models.getProviders(); // registered Provider objects
|
||||
const provider = models.getProvider('anthropic'); // one provider
|
||||
|
||||
const all = models.getModels(); // every model across providers
|
||||
const all = models.getModels(); // every chat model across providers
|
||||
const anthropicModels = models.getModels('anthropic');
|
||||
const model = models.getModel('anthropic', 'claude-sonnet-4-5');
|
||||
|
||||
@@ -285,7 +290,31 @@ for (const m of anthropicModels) {
|
||||
}
|
||||
```
|
||||
|
||||
Dynamically listed models are typed `Model<Api>`. Narrow with the `hasApi()` guard when you need API-specific option typing:
|
||||
The unqualified reads `getModels()`/`getModel()`/`getAvailable()` return chat models (`Model<Api>`) usable with `stream()`. The `*OfType` reads return one model type, and `getAllModels()`/`getAllAvailable()` return every type as `AnyModel`:
|
||||
|
||||
```typescript
|
||||
const images = models.getModelsOfType('image', 'openrouter'); // ImageModel[]
|
||||
const flux = models.getModelOfType('image', 'openrouter', 'black-forest-labs/flux.2-pro');
|
||||
const jev = models.getModelOfType('classifier', 'typesafe', 'jev-latest');
|
||||
const availableImages = await models.getAvailableOfType('image');
|
||||
const everything = models.getAllModels(); // AnyModel[]
|
||||
```
|
||||
|
||||
The model's `type` decides which operation accepts it: chat models stream, `type: "image"` models generate images, and `type: "classifier"` models classify structured state. `type` is optional on chat models, so a model without `type` is a chat model. Do not compare `type` directly; narrow mixed lists with `isModelType()` or read the effective type with `getModelType()`:
|
||||
|
||||
```typescript
|
||||
import { isModelType } from '@earendil-works/pi-ai';
|
||||
|
||||
for (const model of models.getAllModels()) {
|
||||
if (isModelType(model, 'image')) {
|
||||
// model: ImageModel<ImageApi>
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
IDs are unique within each provider and type; one upstream model may have separate entries for different operations. On a provider, `getModels()` returns chat models and the optional `getAllModels()` returns every type; providers with only chat models can omit it.
|
||||
|
||||
Dynamically listed chat models are typed `Model<Api>`. Narrow with the `hasApi()` guard when you need API-specific option typing:
|
||||
|
||||
```typescript
|
||||
import { hasApi } from '@earendil-works/pi-ai';
|
||||
@@ -302,11 +331,26 @@ if (m && hasApi(m, 'anthropic-messages')) {
|
||||
For tooling that wants the generated built-in catalog with full literal typing (provider and model IDs auto-complete), independent of any collection:
|
||||
|
||||
```typescript
|
||||
import { getBuiltinModel, getBuiltinModels, getBuiltinProviders } from '@earendil-works/pi-ai/providers/all';
|
||||
import {
|
||||
getAllBuiltinModels,
|
||||
getBuiltinClassifierModel,
|
||||
getBuiltinClassifierModels,
|
||||
getBuiltinImageModel,
|
||||
getBuiltinImageModels,
|
||||
getBuiltinModel,
|
||||
getBuiltinModels,
|
||||
getBuiltinProviders,
|
||||
} from '@earendil-works/pi-ai/providers/all';
|
||||
|
||||
const model = getBuiltinModel('openai', 'gpt-4o-mini'); // typed Model<'openai-responses'>
|
||||
const radius = getBuiltinModel('radius', 'balanced'); // typed Model<'pi-messages'>
|
||||
const flux = getBuiltinImageModel('openrouter', 'black-forest-labs/flux.2-pro');
|
||||
const jev = getBuiltinClassifierModel('typesafe', 'jev-latest');
|
||||
const providers = getBuiltinProviders();
|
||||
const anthropic = getBuiltinModels('anthropic');
|
||||
const openrouterChat = getBuiltinModels('openrouter'); // Model[]
|
||||
const openrouterImages = getBuiltinImageModels('openrouter'); // ImageModel[]
|
||||
const typesafeClassifiers = getBuiltinClassifierModels('typesafe'); // ClassifierModel[]
|
||||
const openrouterAll = getAllBuiltinModels('openrouter'); // AnyModel[]
|
||||
```
|
||||
|
||||
### Dynamic Providers
|
||||
@@ -320,7 +364,7 @@ await models.refresh(); // refresh all providers conc
|
||||
const fresh = models.getModel('llamacpp', 'qwen3-30b');
|
||||
```
|
||||
|
||||
Static built-in providers are no-ops for `refresh()`. See [createProvider()](#createprovider) for building a dynamic provider.
|
||||
Static built-in providers are no-ops for `refresh()`. Radius is both static and dynamic: it ships the public `radius.pi.dev` catalog for synchronous API lookup, then overlays cached and freshly fetched `/v1/config` models when refreshed with configured auth. See [createProvider()](#createprovider) for building a dynamic provider.
|
||||
|
||||
## Auth
|
||||
|
||||
@@ -417,6 +461,8 @@ Built-in providers resolve these env vars (Node.js; in browsers pass `apiKey` ex
|
||||
| Ant Ling | `ANT_LING_API_KEY` |
|
||||
| Azure OpenAI | `AZURE_OPENAI_API_KEY` + `AZURE_OPENAI_BASE_URL` (e.g. `https://{resource}.ai.azure.com`) or `AZURE_OPENAI_RESOURCE_NAME`. Supports `*.openai.azure.com`, `*.cognitiveservices.azure.com` and `*.ai.azure.com`; root endpoints auto-normalize to `/openai/v1`. Optional: `AZURE_OPENAI_API_VERSION` (default `v1`), `AZURE_OPENAI_DEPLOYMENT_NAME_MAP`. |
|
||||
| Anthropic | `ANTHROPIC_API_KEY` or `ANTHROPIC_OAUTH_TOKEN` |
|
||||
| Radius | `RADIUS_API_KEY` |
|
||||
| TypeSafe | `TYPESAFE_API_KEY` |
|
||||
| DeepSeek | `DEEPSEEK_API_KEY` |
|
||||
| NVIDIA NIM | `NVIDIA_API_KEY` |
|
||||
| Google | `GEMINI_API_KEY` |
|
||||
@@ -440,6 +486,7 @@ Built-in providers resolve these env vars (Node.js; in browsers pass `apiKey` ex
|
||||
| Hugging Face | `HF_TOKEN` |
|
||||
| OpenCode Zen / OpenCode Go | `OPENCODE_API_KEY` |
|
||||
| Kimi For Coding | `KIMI_API_KEY` |
|
||||
| Meta | `META_API_KEY` |
|
||||
| Qwen Token Plan (existing catalog) | `QWEN_TOKEN_PLAN_API_KEY` |
|
||||
| Qwen Token Plan (Individual) | `QWEN_TOKEN_PLAN_API_KEY` |
|
||||
| Qwen Token Plan (China) | `QWEN_TOKEN_PLAN_CN_API_KEY` |
|
||||
@@ -675,34 +722,6 @@ All streaming events emitted during assistant message generation:
|
||||
|
||||
Streaming events for different content blocks are not guaranteed to be contiguous. Providers may emit deltas for text, thinking, and tool calls in the same upstream chunk, and pi may surface corresponding events interleaved, for example `text_start`, `text_delta`, `toolcall_start`, `text_delta`, `toolcall_delta`. Consumers must use `contentIndex` to associate each delta/end event with its block and must not assume that a block's `*_start`/`*_delta`/`*_end` sequence is uninterrupted by events for other blocks.
|
||||
|
||||
### Compact Assistant Message Frames
|
||||
|
||||
`AssistantMessageFrameEncoder` converts one stream into compact, persistable `AssistantMessageFrame` values. Create one encoder per stream and feed it every event in order. The encoder understands that `partial` is live: a block-start event consumed after the provider has already queued later deltas snapshots the current block once, and covered queued text/thinking deltas produce no duplicate frame. It retains only per-open-block counters plus, temporarily, the raw prefix needed to synchronize an already-advanced tool call. It never clones the growing full partial per token.
|
||||
|
||||
The start frame contains message metadata with empty content. Text and thinking frames store each generated character at most once before the authoritative end frame. Tool calls that were already advanced when their start event was consumed use one compact JSON checkpoint before ordinary deltas resume. Terminal `done` and `error` events produce no frame because final message settlement is separate. A pre-generation `error` therefore produces no frames.
|
||||
|
||||
`reduceAssistantMessageFrames()` is the canonical pure reducer. It reconstructs text, thinking, and tool-call arguments, including interleaved blocks identified by `contentIndex`, and rejects malformed sequences. It performs a single pass over the iterable and returns `undefined` when there is no start frame. End frames replace blocks with the provider's authoritative completed content and metadata. The reducer does not validate tool arguments against a TypeBox schema; call `validateToolCall` before execution.
|
||||
|
||||
```typescript
|
||||
import {
|
||||
AssistantMessageFrameEncoder,
|
||||
reduceAssistantMessageFrames,
|
||||
type AssistantMessageFrame,
|
||||
} from '@earendil-works/pi-ai';
|
||||
|
||||
const encoder = new AssistantMessageFrameEncoder();
|
||||
const frames: AssistantMessageFrame[] = [];
|
||||
for await (const event of s) {
|
||||
const frame = encoder.encode(event);
|
||||
if (frame) frames.push(frame);
|
||||
}
|
||||
|
||||
const reconstructedPartial = reduceAssistantMessageFrames(frames);
|
||||
const finalMessage = await s.result(); // Persist terminal settlement separately.
|
||||
```
|
||||
|
||||
An encoder rejects duplicate starts, updates before start, `done` before start, events after a terminal event, duplicate block starts, and block-kind mismatches. An `error` before start is valid and returns no frame.
|
||||
|
||||
## Image Input
|
||||
|
||||
Models with vision capabilities can process images. You can check if a model supports images via the `input` property. If you pass images to a non-vision model, they are silently ignored.
|
||||
@@ -741,20 +760,19 @@ for (const block of response.content) {
|
||||
|
||||
## Image Generation
|
||||
|
||||
Image generation uses a separate API surface from text/chat generation, mirroring the chat-side design: an `ImagesModels` collection holds `ImagesProvider`s, reads are sync, and auth resolves through the owning provider. Image generation is a one-shot API: `generateImages()` waits for the provider response and returns the final `AssistantImages` result — do not use the chat/stream APIs for it.
|
||||
Image models live in the same `Models` collection and on the same `Provider` as chat models, so one credential per provider covers both. They are typed `ImageModel` with `type: "image"` and are used through `generateImages()`, a one-shot API that waits for the provider response and returns the final `AssistantImages` result. Do not use the chat/stream APIs for them; `stream()` rejects image models.
|
||||
|
||||
### Basic Image Generation
|
||||
|
||||
```typescript
|
||||
import { builtinImagesModels } from '@earendil-works/pi-ai/providers/all';
|
||||
import { builtinModels } from '@earendil-works/pi-ai/providers/all';
|
||||
|
||||
// Every built-in image-generation provider; accepts the same options as createModels()
|
||||
const imagesModels = builtinImagesModels();
|
||||
const models = builtinModels();
|
||||
|
||||
const model = imagesModels.getModel('openrouter', 'google/gemini-2.5-flash-image')!;
|
||||
const model = models.getModelOfType('image', 'openrouter', 'google/gemini-2.5-flash-image')!;
|
||||
|
||||
// Auth resolves through the provider (OPENROUTER_API_KEY here); explicit apiKey wins
|
||||
const result = await imagesModels.generateImages(model, {
|
||||
const result = await models.generateImages(model, {
|
||||
input: [{ type: 'text', text: 'Generate a red circle on a plain white background.' }]
|
||||
});
|
||||
|
||||
@@ -768,7 +786,33 @@ for (const block of result.output) {
|
||||
}
|
||||
```
|
||||
|
||||
Like the chat side, you can build the collection from parts: `createImagesModels({ credentials?, authContext? })`, the `openrouterImagesProvider()` factory from `@earendil-works/pi-ai/providers/openrouter-images`, and `createImagesProvider({ id, auth, models, refreshModels?, api })` for custom image providers (with `imagesModels.refresh(provider?)` for dynamic lists). Failures never reject — they return an `AssistantImages` with `stopReason: "error"`. The collection's provider-scoped `getAuth(providerId)` works exactly like the chat-side one.
|
||||
`generateImages()` accepts only `ImageModel` values. If an upstream model supports both chat and image generation, the catalog contains separate entries with the same provider and ID: `getModel()` returns its chat operation and `getModelOfType('image', ...)` returns its image operation. Failures never reject; they return an `AssistantImages` with `stopReason: "error"`, including unknown providers, unconfigured auth, and providers without an image implementation.
|
||||
|
||||
A provider declares image support with the `images` option of [`createProvider()`](#createprovider): a map from `model.api` to an implementation with `generateImages()`. Image models go into the same `models` list as chat models. `api` becomes optional when `images` is present, so an image-only provider is just a provider without chat models:
|
||||
|
||||
```typescript
|
||||
import { createProvider, envApiKeyAuth } from '@earendil-works/pi-ai';
|
||||
|
||||
const pixels = createProvider({
|
||||
id: 'pixels',
|
||||
auth: { apiKey: envApiKeyAuth('Pixels API key', ['PIXELS_API_KEY']) },
|
||||
models: [{
|
||||
type: 'image',
|
||||
id: 'flux-pro',
|
||||
name: 'FLUX Pro',
|
||||
api: 'pixels-images',
|
||||
provider: 'pixels',
|
||||
baseUrl: 'https://api.pixels.test/v1',
|
||||
input: ['text'],
|
||||
output: ['image'],
|
||||
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
|
||||
}],
|
||||
images: {
|
||||
'pixels-images': { generateImages: async (model, context, options) => { /* ... */ } },
|
||||
},
|
||||
});
|
||||
models.setProvider(pixels);
|
||||
```
|
||||
|
||||
The old global API (`getImageModel()` / `getImageModels()` / `getImageProviders()` / `generateImages()`) remains available on the [compat entrypoint](#migrating-from-the-old-global-api):
|
||||
|
||||
@@ -789,7 +833,7 @@ Some models also support image input:
|
||||
import { readFileSync } from 'fs';
|
||||
|
||||
const imageBuffer = readFileSync('input.png');
|
||||
const result = await imagesModels.generateImages(model, {
|
||||
const result = await models.generateImages(model, {
|
||||
input: [
|
||||
{ type: 'text', text: 'Create a variation of this image with a blue background.' },
|
||||
{ type: 'image', data: imageBuffer.toString('base64'), mimeType: 'image/png' }
|
||||
@@ -800,14 +844,13 @@ const result = await imagesModels.generateImages(model, {
|
||||
Check capabilities on the model metadata:
|
||||
|
||||
```typescript
|
||||
console.log(model.input); // ['text', 'image']
|
||||
console.log(model.output); // ['image'] or ['image', 'text']
|
||||
console.log(model.input); // ['text'] or ['text', 'image']
|
||||
console.log(model.output); // ['image'] or ['image', 'text']
|
||||
```
|
||||
|
||||
### Notes and Limitations
|
||||
|
||||
- Image models live in `ImagesModels` collections, chat models in `Models` collections; the two are separate surfaces.
|
||||
- Use `generateImages()`, not the chat/stream APIs.
|
||||
- Image models and chat models share `Models` and `Provider`; list them with `getModelsOfType('image')` and run them with `generateImages()`, never the chat/stream APIs.
|
||||
- Image-generation models do not participate in tool calling.
|
||||
- Outputs are returned in `AssistantImages.output` and can include both base64-encoded `ImageContent` blocks and `TextContent` blocks.
|
||||
- Some models return only images, others return images plus text. Check `model.output`.
|
||||
@@ -816,6 +859,108 @@ console.log(model.output); // ['image'] or ['image', 'text']
|
||||
- If you want a model to analyze images in a conversation or call tools, use the regular chat APIs with a model that supports image input.
|
||||
- At the moment, image generation is available through only one provider, OpenRouter.
|
||||
|
||||
## Classification
|
||||
|
||||
Classifier models consume structured JSON state, and optionally images, and answer one or more typed questions. They do not use chat or image-generation APIs. TypeSafe's Jev model, Cloudflare's Clef models, and OpenAI's GPT-6 Luna are available from these built-in providers:
|
||||
|
||||
| Provider | Model IDs | Auth |
|
||||
| --- | --- | --- |
|
||||
| `typesafe` | `jev-latest` | `TYPESAFE_API_KEY` |
|
||||
| `openrouter` | `typesafe/jev-1.13`, `~typesafe/jev-latest` | `OPENROUTER_API_KEY` or OpenRouter OAuth |
|
||||
| `cloudflare-workers-ai` | `typesafe/jev`, `@cf/cloudflare/clef`, `@cf/cloudflare/clef-flash` | `CLOUDFLARE_API_KEY` and `CLOUDFLARE_ACCOUNT_ID` |
|
||||
| `vercel-ai-gateway` | `typesafe-ai/jev` | `AI_GATEWAY_API_KEY` |
|
||||
| `opencode` | `jev-1.13`, `jev-1.13-free` | `OPENCODE_API_KEY` |
|
||||
| `openai` | `gpt-6-luna` | `OPENAI_API_KEY` |
|
||||
|
||||
```typescript
|
||||
import { builtinModels } from '@earendil-works/pi-ai/providers/all';
|
||||
|
||||
const models = builtinModels();
|
||||
const model = models.getModelOfType('classifier', 'typesafe', 'jev-latest')!;
|
||||
const result = await models.classify(model, {
|
||||
state: { message: 'The change works perfectly, thanks.' },
|
||||
questions: {
|
||||
category: {
|
||||
type: 'choice',
|
||||
instructions: 'Classify the message.',
|
||||
criteria: {
|
||||
approval: 'The user approves of the result',
|
||||
correction: 'The user requests a correction'
|
||||
}
|
||||
},
|
||||
satisfaction: {
|
||||
type: 'score',
|
||||
instructions: 'Score user satisfaction.',
|
||||
criteria: ['dissatisfied', 'neutral', 'satisfied']
|
||||
},
|
||||
approved: {
|
||||
type: 'bool',
|
||||
instructions: 'Does the user approve?',
|
||||
criteria: { true: 'Approval', false: 'No approval' }
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
console.log(result.answers);
|
||||
```
|
||||
|
||||
The public contract uses `bool` questions and `{ type: "bool", probability }` answers. The TypeSafe adapter translates those to and from its `noul` wire representation. Like image generation, `classify()` resolves to a result with `stopReason: "error"` instead of rejecting for provider, authentication, or response errors.
|
||||
|
||||
When the service reports token counts, `result.usage` carries them with their cost at the model's catalog price, the same `Usage` shape as chat messages. All System One services report token counts; a request that was answered with malformed answers keeps its usage. Local classifiers such as `llama-cpp-classify` report no usage.
|
||||
|
||||
`ClassifierOptions.temperature` divides the answer logits by the given value before they are normalized; values above 1 soften the distribution. APIs that cannot apply it, such as System One and OpenAI Decisions, ignore it.
|
||||
|
||||
`ClassifierContext.images` holds image blocks (`{ type: "image", data, mimeType }`) judged together with the state. Only models whose `input` includes `"image"` accept them; `classify()` returns an error result for other models.
|
||||
|
||||
### OpenAI Decisions
|
||||
|
||||
The `openai-decisions` API calls OpenAI's [Decisions API](https://developers.openai.com/api/docs/guides/decisions) (`POST /v1/decisions`). GPT-6 Luna is its only model. The state is sent as JSON text; with images, the input is one user message with the state followed by the images as data URLs (at most 128). `choice` and `score` questions map to the Decisions types of the same name. `bool` questions become `predicate` questions; predicates have no criteria field, so the meanings of true and false are appended to the instructions. If OpenAI refuses to answer a question, the whole result is an error. Usage reports input tokens only, priced at the catalog input rate with OpenAI's long-context multiplier.
|
||||
|
||||
The endpoint needs an OpenAI API key; Sign in with ChatGPT credentials are rejected, so the `openai` provider hides `gpt-6-luna` from `getAvailableOfType('classifier')` while it uses OAuth. The API rejects inputs above 922K tokens, but requests that run longer than about five seconds, currently above roughly 600K input tokens, fail with a gateway timeout (504).
|
||||
|
||||
### Chat models on llama.cpp
|
||||
|
||||
llama.cpp also serves decision models such as Julia-1 or Kev natively through `/v1/systemone`. Since llama.cpp 0.6.0, `GET /models` lists `decisions` in a model's `architecture.output_modalities` for these models. They use the `typesafe-system-one` API with the server's `/v1` URL as `baseUrl` (for example `http://127.0.0.1:8080/v1`); it needs an `apiKey`, which llama.cpp ignores unless it was started with `--api-key`. The `llama-cpp-classify` API is the fallback for chat models.
|
||||
|
||||
The `llama-cpp-classify` API turns a chat model served by llama.cpp's `llama-server` into a classifier. Each question becomes one chat prompt: the state, every question of the request, the state again, and the question with its answers under single-token labels (letters for a choice, `Yes`/`No` for a bool, digits for a score). The prompt up to the final question is shared by all questions of a request, so the server's prompt cache evaluates the state once per request. The server returns the log-probabilities of the next token, and the answer is the softmax over the label tokens. Choices support up to 62 options and scores up to 10 levels. The model's `baseUrl` is the server URL; a trailing `/v1` is ignored. In router mode, the model ID selects the model.
|
||||
|
||||
```typescript
|
||||
import { createProvider } from '@earendil-works/pi-ai';
|
||||
import { llamaCppClassifyApi } from '@earendil-works/pi-ai/api/llama-cpp-classify.lazy';
|
||||
|
||||
const provider = createProvider({
|
||||
id: 'local-llama',
|
||||
auth: { apiKey: { name: 'llama.cpp', resolve: async () => ({ auth: {} }) } },
|
||||
models: [{
|
||||
type: 'classifier',
|
||||
id: 'qwen3-4b',
|
||||
name: 'Qwen3 4B',
|
||||
api: 'llama-cpp-classify',
|
||||
provider: 'local-llama',
|
||||
baseUrl: 'http://127.0.0.1:8080',
|
||||
input: ['text'],
|
||||
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
|
||||
contextWindow: 32768
|
||||
}],
|
||||
classifiers: { 'llama-cpp-classify': llamaCppClassifyApi() }
|
||||
});
|
||||
```
|
||||
|
||||
Raw label probabilities are usually overconfident; pass `temperature` above 1 to soften them.
|
||||
|
||||
Custom providers register classifier models and implementations by API ID:
|
||||
|
||||
```typescript
|
||||
createProvider({
|
||||
id: 'classifier-service',
|
||||
auth,
|
||||
models: [model],
|
||||
classifiers: {
|
||||
'classifier-api': { classify: async (model, context, options) => result }
|
||||
}
|
||||
});
|
||||
```
|
||||
|
||||
## Thinking/Reasoning
|
||||
|
||||
Many models support thinking/reasoning capabilities where they can show their internal thought process. You can check if a model supports reasoning via the `reasoning` property. If you pass reasoning options to a non-reasoning model, they are silently ignored.
|
||||
@@ -827,7 +972,7 @@ Many models support thinking/reasoning capabilities where they can show their in
|
||||
const model = models.getModel('anthropic', 'claude-sonnet-4-5')!;
|
||||
// or models.getModel('openai', 'gpt-5-mini');
|
||||
// or models.getModel('google', 'gemini-2.5-flash');
|
||||
// or models.getModel('xai', 'grok-4.6');
|
||||
// or models.getModel('xai', 'grok-4.7');
|
||||
|
||||
// Check if model supports reasoning
|
||||
if (model.reasoning) {
|
||||
@@ -1024,11 +1169,30 @@ const response = await models.complete(model, context, {
|
||||
|
||||
The callback is supported by `stream`, `complete`, `streamSimple`, and `completeSimple`.
|
||||
|
||||
### Observing Provider Stream Events
|
||||
|
||||
Use `onProviderStreamEvent` to inspect provider-specific fields that Pi does not include in `AssistantMessage`. The callback receives the parsed event available to the adapter before Pi normalizes it. Treat the event as read-only because mutations can affect normalization. This is not guaranteed to be the original HTTP bytes or SSE frame.
|
||||
|
||||
```typescript
|
||||
const openRouterModel = models.getModel('openrouter', 'openrouter/auto')!;
|
||||
const response = await models.complete(openRouterModel, context, {
|
||||
headers: { "X-OpenRouter-Metadata": "enabled" },
|
||||
onProviderStreamEvent: (data) => {
|
||||
const chunk = data as Record<string, unknown>;
|
||||
if (chunk.openrouter_metadata) {
|
||||
console.log(chunk.openrouter_metadata);
|
||||
}
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
Callbacks are awaited in stream order, so slow callbacks delay stream consumption and thrown errors fail the request. SDK-backed adapters can expose only fields retained by their SDK.
|
||||
|
||||
## Custom Providers
|
||||
|
||||
### createProvider()
|
||||
|
||||
`createProvider()` builds a provider from parts: identity, auth, a model list, and an API implementation. Use it for local inference servers, proxies, or any OpenAI/Anthropic-compatible endpoint:
|
||||
`createProvider()` builds a provider from parts: identity, auth, a model list, and an API implementation (`api` for chat models, `images` for image generation, `classifiers` for classification; at least one is required, see [Image Generation](#image-generation)). Use it for local inference servers, proxies, or any OpenAI/Anthropic-compatible endpoint:
|
||||
|
||||
```typescript
|
||||
import { createModels, createProvider, envApiKeyAuth, type Model } from '@earendil-works/pi-ai';
|
||||
@@ -1110,7 +1274,7 @@ const tenantGateway = createProvider({
|
||||
});
|
||||
```
|
||||
|
||||
Dynamic model lists use `fetchModels`. `Models.refresh()` refreshes every configured dynamic provider, passing its effective API-key or refreshed OAuth credential. A `ModelsStore` persists dynamic catalogs; both stores default to in-memory implementations. Its `read`, `write`, and `delete` operations accept optional cancellation, and `Models` binds those waits to the provider refresh signal.
|
||||
Dynamic model lists use `fetchModels`, which can return models of every type. `Models.refresh()` refreshes every configured dynamic provider, passing its effective API-key or refreshed OAuth credential. A `ModelsStore` persists dynamic catalogs; both stores default to in-memory implementations. Its `read`, `write`, and `delete` operations accept optional cancellation, and `Models` binds those waits to the provider refresh signal.
|
||||
|
||||
```typescript
|
||||
const models = createModels({ credentials, modelsStore });
|
||||
@@ -1132,7 +1296,7 @@ for (const [provider, error] of result.errors) console.error(provider, error);
|
||||
|
||||
Use `models.refresh({ providers: ['openrouter'] })` to restrict work to selected providers, `models.refresh({ allowNetwork: false })` to restore persisted catalogs without network access, or `models.refresh({ force: true })` to bypass provider freshness checks. Model reads stay synchronous and return the last restored or refreshed list.
|
||||
|
||||
`createProvider()` handles dynamic publication and persistence automatically. Handwritten `Provider.refreshModels()` implementations receive the read-only `context.stored` snapshot and publish through `context.publish({ persist?, update? })`. Omit `persist` to leave storage unchanged, pass a `ModelsStoreEntry` to write it, or pass `persist: null` to delete it. Publication is generation-checked; put synchronous in-memory catalog changes in `update` rather than mutating state before publication.
|
||||
`createProvider()` handles dynamic publication and persistence automatically. Handwritten `Provider.refreshModels()` implementations receive the read-only `context.stored` snapshot and publish through `context.publish({ persist?, update? })`. Omit `persist` to leave storage unchanged, pass a `ModelsStoreEntry` to write it, or pass `persist: null` to delete it. `ModelsStoreEntry.models` contains models of every type. Publication is generation-checked; put synchronous in-memory catalog changes in `update` rather than mutating state before publication.
|
||||
|
||||
Custom models can carry `headers` (e.g. proxies behind bot detection) and `compat` flags. `Models.getAuth(model)` includes those model headers, and stream methods merge them before explicit request headers and `transformHeaders`. See [OpenAI Compatibility Settings](#openai-compatibility-settings).
|
||||
|
||||
@@ -1168,12 +1332,13 @@ const ollamaReasoningModel: Model<'openai-completions'> = {
|
||||
|
||||
### Calling API Implementations Directly
|
||||
|
||||
The API implementations are importable on their own. Each module exports exactly `stream` and `streamSimple` with that API's full option typing. Direct calls bypass provider auth — pass `apiKey` explicitly:
|
||||
The API implementations are importable on their own. Each module exports exactly `stream` and `streamSimple` with that API's full option typing. Direct calls bypass provider auth and context normalization — pass `apiKey` explicitly and wrap the context in `normalizeContext()`:
|
||||
|
||||
```typescript
|
||||
import { normalizeContext } from '@earendil-works/pi-ai';
|
||||
import { stream } from '@earendil-works/pi-ai/api/anthropic-messages';
|
||||
|
||||
const s = stream(claudeModel, context, {
|
||||
const s = stream(claudeModel, normalizeContext(context), {
|
||||
apiKey: process.env.ANTHROPIC_API_KEY,
|
||||
thinkingEnabled: true,
|
||||
thinkingBudgetTokens: 2048,
|
||||
@@ -1206,9 +1371,11 @@ interface OpenAICompletionsCompat {
|
||||
supportsDeveloperRole?: boolean; // Whether provider supports `developer` role vs `system` (default: true)
|
||||
supportsReasoningEffort?: boolean; // Whether provider supports `reasoning_effort` (default: true)
|
||||
supportsUsageInStreaming?: boolean; // Whether provider supports `stream_options: { include_usage: true }` (default: true)
|
||||
supportsStrictMode?: boolean; // Whether provider supports `strict` in tool definitions (default: true)
|
||||
supportsStrictMode?: boolean; // Whether provider supports `strict` in tool definitions (default: false; enabled in metadata for capable built-in models)
|
||||
supportsOpenAIGrammarTools?: boolean; // Whether to emit OpenAI custom Lark/regex grammar tools; false falls back to normal function tools (default: false; the generated catalog enables it for capable models)
|
||||
sendSessionAffinityHeaders?: boolean; // Send session-affinity data from `sessionId` (default: false)
|
||||
supportsMidConvoSystemMessages?: boolean; // Whether the model accepts system messages after the conversation started; false folds them into the leading prompt (default: false; the generated catalog enables it for verified models)
|
||||
supportsMidConvoToolAdditions?: boolean; // Whether system messages can add tools mid-conversation via Kimi-style `tools` system messages; requires supportsMidConvoSystemMessages (default: false)
|
||||
sendSessionAffinityHeaders?: boolean; // Send session-affinity data from `sessionId` (default: true for OpenRouter, false otherwise)
|
||||
sessionAffinityFormat?: 'openai' | 'openai-nosession' | 'openrouter'; // Format for session affinity: 'openai' uses `prompt_cache_key`, `session_id`, `x-client-request-id`, and `x-session-affinity`; 'openai-nosession' uses `prompt_cache_key`, `x-client-request-id`, and `x-session-affinity`; 'openrouter' uses `x-session-id` (default: auto-detected)
|
||||
maxTokensField?: 'max_completion_tokens' | 'max_tokens'; // Which field name to use (default: max_completion_tokens)
|
||||
requiresToolResultName?: boolean; // Whether tool results require the `name` field (default: false)
|
||||
@@ -1234,6 +1401,8 @@ interface OpenAIResponsesCompat {
|
||||
}
|
||||
```
|
||||
|
||||
OpenRouter requests send `x-session-id` from `sessionId` when prompt caching is enabled. Chat Completions and Anthropic Messages both auto-detect OpenRouter endpoints unless `sendSessionAffinityHeaders` is explicitly false. On Anthropic-compatible models, `sessionAffinityFormat: "openrouter"` selects `x-session-id`; when unset, the existing `x-session-affinity` format is used. Explicit request headers take precedence over generated headers.
|
||||
|
||||
If `compat` is not set, the library falls back to URL-based detection. If `compat` is partially set, unspecified fields use the detected defaults. This is useful for:
|
||||
|
||||
- **LiteLLM proxies**: May not support `store` field
|
||||
@@ -1329,13 +1498,13 @@ Notes:
|
||||
|
||||
## Cross-Provider Handoffs
|
||||
|
||||
The library supports seamless handoffs between different LLM providers within the same conversation. This allows you to switch models mid-conversation while preserving context, including thinking blocks, tool calls, and tool results.
|
||||
The library supports seamless handoffs between different LLM providers within the same conversation. This allows you to switch models mid-conversation while preserving context, including text, tool calls, and tool results.
|
||||
|
||||
When messages from one provider are sent to a different provider, the library automatically transforms them for compatibility:
|
||||
|
||||
- **User and tool result messages** are passed through unchanged
|
||||
- **Assistant messages from the same provider/API** are preserved as-is
|
||||
- **Assistant messages from different providers** have their thinking blocks converted to text with `<thinking>` tags
|
||||
- **Assistant messages from the same provider, API, and model** are preserved as-is, including thinking blocks
|
||||
- **Assistant messages from a different model** have their thinking blocks dropped. Signatures and encrypted reasoning only work with the model that produced them, and readable thinking sent as plain text makes the new model treat it as part of the reply
|
||||
- **Tool calls and regular text** are preserved unchanged
|
||||
|
||||
```typescript
|
||||
@@ -1356,7 +1525,7 @@ const claude = models.getModel('anthropic', 'claude-sonnet-4-5')!;
|
||||
context.messages.push({ role: 'user', content: 'What is 25 * 18?', timestamp: Date.now() });
|
||||
context.messages.push(await models.completeSimple(claude, context, { reasoning: 'medium' }));
|
||||
|
||||
// Switch to GPT-5 - it will see Claude's thinking as <thinking> tagged text
|
||||
// Switch to GPT-5 - it sees Claude's answer, but not Claude's thinking
|
||||
const gpt5 = models.getModel('openai', 'gpt-5-mini')!;
|
||||
context.messages.push({ role: 'user', content: 'Is that calculation correct?', timestamp: Date.now() });
|
||||
context.messages.push(await models.complete(gpt5, context));
|
||||
@@ -1367,7 +1536,40 @@ context.messages.push({ role: 'user', content: 'What was the original question?'
|
||||
const geminiResponse = await models.complete(gemini, context);
|
||||
```
|
||||
|
||||
All providers can handle messages from other providers — text, tool calls and results (including images), thinking blocks (transformed to tagged text), and aborted messages with partial content. This enables flexible workflows: start with a fast model, switch to a more capable one for complex reasoning, or maintain continuity across provider outages.
|
||||
All providers can handle messages from other providers — text, tool calls and results (including images), thinking blocks (dropped when the model changes), and aborted messages with partial content. This enables flexible workflows: start with a fast model, switch to a more capable one for complex reasoning, or maintain continuity across provider outages.
|
||||
|
||||
## System Messages
|
||||
|
||||
`Context.systemPrompt` and `Context.tools` are shorthand for a leading system message. The public entry points (`Models.stream()`, `streamSimple()`, `complete()`, `completeSimple()`) accept a `Context` and call `normalizeContext()` once; everything below them, including `Provider.stream()`, `ProviderStreams`, and the API implementation modules, receives the resulting `TranscriptContext`, which only has `messages`. The transcript can also carry system messages later in the conversation to change the prompt or the tool set without rewriting the history:
|
||||
|
||||
```typescript
|
||||
interface SystemMessage {
|
||||
role: "system";
|
||||
content: string | TextContent[]; // leading: base prompt; later: added instructions
|
||||
sections?: Record<string, string | null>; // named prompt sections; later messages patch by name, null removes
|
||||
toolsAdded?: Tool[]; // tools that become available here
|
||||
toolsRemoved?: ToolReference[]; // tools that stop being available here
|
||||
timestamp: number;
|
||||
}
|
||||
```
|
||||
|
||||
Sections are opaque text rendered verbatim after `content`, joined by blank lines. Keep each one self-delimiting (a tag, a heading) so the model can relate an update to the original. Replaying every system message in order yields the current prompt and tools; the replay helpers take the message list:
|
||||
|
||||
```typescript
|
||||
import { getCurrentSystemPrompt, getCurrentTools } from "@earendil-works/pi-ai";
|
||||
|
||||
const messages: Message[] = [
|
||||
{ role: "system", content: "You are helpful.", sections: { rules: "<rules>Be brief.</rules>" }, toolsAdded: [readTool], timestamp: 1 },
|
||||
{ role: "user", content: "hi", timestamp: 2 },
|
||||
{ role: "system", content: "", sections: { rules: "<rules>Be thorough.</rules>" }, toolsRemoved: [{ name: "read" }], timestamp: 3 },
|
||||
];
|
||||
getCurrentSystemPrompt(messages); // "You are helpful.\n\n<rules>Be thorough.</rules>"
|
||||
getCurrentTools(messages); // []
|
||||
```
|
||||
|
||||
A custom `Provider` or `ProviderStreams` implementation reads the prompt and tools the same way from `context.messages`; `context.systemPrompt` and `context.tools` do not exist at that layer.
|
||||
|
||||
Models that accept system messages mid-conversation (`supportsMidConvoSystemMessages` in the model's compat settings, set by the generated catalog for verified models) receive each later system message in place, so the cached prefix stays intact; section changes are framed by name for the model. Every other model receives `collapseSystemMessages(transcript)`: the replayed prompt and current tools as the leading system message, with later system messages dropped. Anthropic models that also set `supportsMidConvoToolChanges` send tool changes as native blocks (`inline-tools-2026-09-15` beta): the top-level tool list holds the initial tools plus a stable deferred placeholder and never changes, later tools are defined by value in `tool_addition` blocks (a new definition under an existing name replaces the old one), and removals are `tool_removal` references, so tool changes do not invalidate the prompt cache. That needs at least one initial tool; otherwise the current tool list is sent at the top level with the system text only. OpenAI Responses models with `supportsAdditionalTools` or `supportsToolSearch` anchor additive tool changes at their message; everything else sends the current tool list at the top level.
|
||||
|
||||
## Context Serialization
|
||||
|
||||
@@ -1433,10 +1635,10 @@ Browser compatibility notes:
|
||||
|
||||
## Bundling and Tree Shaking
|
||||
|
||||
For small bundles, import only the providers you need:
|
||||
For small bundles and low-overhead unbundled scripts, import the model runtime and only the providers you need:
|
||||
|
||||
```typescript
|
||||
import { createModels } from '@earendil-works/pi-ai';
|
||||
import { createModels } from '@earendil-works/pi-ai/models';
|
||||
import { openaiProvider } from '@earendil-works/pi-ai/providers/openai';
|
||||
|
||||
const models = createModels();
|
||||
@@ -1445,7 +1647,8 @@ models.setProvider(openaiProvider());
|
||||
|
||||
Rules:
|
||||
|
||||
- `@earendil-works/pi-ai` is the core entrypoint and does not import built-in catalogs, provider factories, or SDK implementations.
|
||||
- `@earendil-works/pi-ai/models` exports the model runtime (`createModels`, `createProvider`, model helpers, and their types) without TypeBox, built-in catalogs, or SDK implementations. Other types can still use `import type` from the root.
|
||||
- `@earendil-works/pi-ai` is the core entrypoint and does not import built-in catalogs, real provider factories, or SDK implementations, but it eagerly imports TypeBox and schema validation. Unbundled Node scripts do not tree-shake its unused exports; prefer `./models`, `./providers/faux`, and specific `./utils/*` subpaths when those are all you need.
|
||||
- `@earendil-works/pi-ai/providers/<provider>` imports that provider's catalog and lazy API wrapper only.
|
||||
- `@earendil-works/pi-ai/providers/all` imports every built-in provider factory and all catalogs. Use it only when you want the full built-in set.
|
||||
- With code splitting, provider SDKs stay in lazy chunks and load on first request.
|
||||
@@ -1509,7 +1712,8 @@ Use this when one process needs different provider settings per request, or when
|
||||
Several providers support OAuth authentication instead of static API keys:
|
||||
|
||||
- **Anthropic** (Claude Pro/Max subscription)
|
||||
- **OpenAI Codex** (ChatGPT Plus/Pro subscription, access to GPT-5.x Codex models)
|
||||
- **OpenAI** (Sign in with ChatGPT: uses the ChatGPT subscription with the OpenAI API)
|
||||
- **OpenAI Codex (legacy)** (ChatGPT Plus/Pro subscription, access to GPT-5.x Codex models)
|
||||
- **GitHub Copilot** (Copilot subscription)
|
||||
- **OpenRouter** (OAuth PKCE that mints a user-controlled API key)
|
||||
|
||||
@@ -1589,7 +1793,7 @@ Built-in login and refresh flows are private provider implementations. Use provi
|
||||
|
||||
Provider notes:
|
||||
|
||||
**OpenAI Codex**: Requires a ChatGPT Plus or Pro subscription. Provides access to GPT-5.x Codex models with extended context windows and reasoning capabilities. The library automatically handles session-based prompt caching when `sessionId` is provided in stream options unless `cacheRetention` is `"none"`. You can set `transport` in stream options to `"sse"`, `"websocket"`, or `"auto"` for Codex Responses transport selection. When using WebSocket with a `sessionId` and cache retention enabled, connections are reused per session and expire after 5 minutes of inactivity.
|
||||
**OpenAI Codex (legacy)**: Superseded by Sign in with ChatGPT on the OpenAI provider. Requires a ChatGPT Plus or Pro subscription. Provides access to GPT-5.x Codex models with extended context windows and reasoning capabilities. The library automatically handles session-based prompt caching when `sessionId` is provided in stream options unless `cacheRetention` is `"none"`. You can set `transport` in stream options to `"sse"`, `"websocket"`, or `"auto"` for Codex Responses transport selection. When using WebSocket with a `sessionId` and cache retention enabled, connections are reused per session and expire after 5 minutes of inactivity. Call `cleanupSessionResources(sessionId)` when finished so the pooled connection does not keep the process alive.
|
||||
|
||||
**Azure OpenAI (Responses)**: Uses the Responses API only. Set `AZURE_OPENAI_API_KEY` and either `AZURE_OPENAI_BASE_URL` or `AZURE_OPENAI_RESOURCE_NAME`. `AZURE_OPENAI_BASE_URL` supports both `https://<resource>.openai.azure.com` and `https://<resource>.cognitiveservices.azure.com`; root endpoints are normalized to `.../openai/v1` automatically. Use `AZURE_OPENAI_API_VERSION` (defaults to `v1`) to override the API version if needed. Deployment names are treated as model IDs by default, override with `azureDeploymentName` or `AZURE_OPENAI_DEPLOYMENT_NAME_MAP` using comma-separated `model-id=deployment` pairs (for example `gpt-4o-mini=my-deployment,gpt-4o=prod`). Legacy deployment-based URLs are intentionally unsupported.
|
||||
|
||||
@@ -1618,6 +1822,9 @@ Compat is a strict superset of the root entrypoint, so a file can switch its imp
|
||||
| `getEnvApiKey('openai')` | `await models.getAuth(model.provider)` |
|
||||
| `streamAnthropic(model, ctx, opts)` | `stream` from `@earendil-works/pi-ai/api/anthropic-messages`, or a provider in a collection |
|
||||
| `registerFauxProvider()` | `fauxProvider()` + `models.setProvider()` |
|
||||
| `getImageModel('openrouter', id)` / `generateImages(model, ctx, { apiKey })` | `models.getModelOfType('image', 'openrouter', id)` / `models.generateImages(model, ctx)` |
|
||||
|
||||
The separate `ImagesModels`/`ImagesProvider` collection that existed briefly (`createImagesModels()`, `createImagesProvider()`, `openrouterImagesProvider()`, `builtinImagesModels()`) is gone: image models now live on the regular provider. Replace `builtinImagesModels()` with `builtinModels()`, `imagesModels.getModel()` with `models.getModelOfType('image', ...)`, and `createImagesProvider({ models, api })` with `createProvider({ models, images })`. The old plural image type names are removed; use `ImageModel` and `ImageApi`, and add `type: "image"` to image model literals.
|
||||
|
||||
## Development
|
||||
|
||||
@@ -1636,17 +1843,17 @@ Adding a new LLM provider requires changes across multiple files. The layered la
|
||||
Create a new API implementation file (for example `bedrock-converse-stream.ts`) that exports exactly `stream` and `streamSimple`, plus:
|
||||
|
||||
- An options interface extending `StreamOptions` (for example `BedrockOptions`)
|
||||
- Message conversion functions to transform `Context` to provider format
|
||||
- Message conversion functions to transform the `TranscriptContext` messages to provider format; read the prompt and tools from the transcript with `getInitialSystemMessage()`, `getCurrentTools()`, and `resolveTranscript()`
|
||||
- Tool conversion if the provider supports tools
|
||||
- Response parsing to emit standardized events (`text`, `tool_call`, `thinking`, `usage`, `stop`)
|
||||
|
||||
Add a lazy wrapper `src/api/<api-id>.lazy.ts` (`<name>Api()` via `lazyApi()`) so providers can reference the implementation without importing its SDK. Add any root-level `export type` re-exports in `src/index.ts` that should remain available from `@earendil-works/pi-ai`.
|
||||
|
||||
#### 3. Model Generation (`scripts/generate-models.ts`, `scripts/generate-image-models.ts`)
|
||||
#### 3. Model Generation (`scripts/generate-models.ts`)
|
||||
|
||||
- Add logic to fetch and parse models from the provider's source (e.g., models.dev API)
|
||||
- Map chat/tool-capable provider model data to the standardized `Model` interface via `scripts/generate-models.ts`; hydration groups the ignored `src/providers/data/<id>.json` values by API, while stable `src/providers/<id>.models.ts` wrappers derive exact model/API types directly from those JSON keys
|
||||
- Map image-generation provider model data to the standardized `ImagesModel` interface via `scripts/generate-image-models.ts`
|
||||
- Map chat/tool-capable provider data to `Model`, image-generation data to `ImageModel`, and models.dev `type: "decision"` entries to `ClassifierModel`; hydration groups the ignored `src/providers/data/<id>.json` values by API while stable `src/providers/<id>.models.ts` wrappers derive exact model/API types directly from those JSON keys
|
||||
- Keep model ids unique within each provider and model type; emit separate entries when an upstream model supports multiple operations
|
||||
- Handle provider-specific quirks (pricing format, capability flags, model ID transformations)
|
||||
|
||||
#### 4. Provider Factory (`src/providers/<id>.ts`)
|
||||
|
||||
Reference in new issue
Block a user