Monorch

Providers

OpenAI-compatible chat completions with SSE streaming, timeouts, and abort. Point baseUrl at LiteLLM, OpenRouter, or a local gateway. We do not rebuild a proxy.

OpenAI / LiteLLM

model.ts
import { openai } from "@monorch/ai/openai";
// or: import { openai } from "@monorch/ai";

const model = openai("gpt-4.1-mini", {
  apiKey: process.env.OPENAI_API_KEY ?? process.env.LITELLM_API_KEY,
  baseUrl: process.env.LITELLM_URL ?? "https://api.openai.com/v1",
  timeoutMs: 60_000,
  defaultHeaders: { "x-custom": "1" },
});

Generate options

Per-call GenerateOptions on generate / stream: temperature, maxTokens, toolChoice (auto | none | required | named function), signal, and timeoutMs. Agent loops set messages/tools for you; use these when calling the model handle directly.

generate-opts.ts
const handle = model(openai("gpt-4.1-mini"));

await handle.generate({
  messages: [{ role: "user", content: "Summarize the order" }],
  temperature: 0.2,
  maxTokens: 256,
  toolChoice: "none",
  timeoutMs: 15_000,
});

Streaming and abort

openai() implements provider stream() over SSE. Per-call signal and timeoutMs merge into the fetch abort. Agent opts.signal forwards here. Non-OK HTTP responses throw OPENAI_HTTP.

abort.ts
const ctrl = new AbortController();
const handle = model(openai("gpt-4.1-mini"));

const pending = handle.generate({
  messages: [{ role: "user", content: "hi" }],
  signal: ctrl.signal,
  timeoutMs: 15_000,
});
ctrl.abort();

Mock

Script text and tool calls in order for tests and smoke. Each generate consumes the next scripted turn. Mock also respects abort.

mock-model.ts
import { mock } from "@monorch/ai/openai";

const model = mock([
  { toolCalls: [{ id: "1", name: "add", arguments: { a: 1, b: 2 } }] },
  { text: "done" },
]);

Model handle

Wrap any provider with model(provider) for generate, stream, and generateObject (Zod → Rust validate).

generate-object.ts
import { model } from "@monorch/ai";
import { openai } from "@monorch/ai/openai";
import { z } from "zod";

const handle = model(openai("gpt-4.1-mini"));
const reply = await handle.generateObject({
  prompt: "Classify: I want a refund",
  output: z.object({ intent: z.enum(["refund", "faq", "other"]) }),
});

Live smoke

The repo example supports LIVE_SMOKE=1 with OPENAI_API_KEY or LITELLM_API_KEY (optional LITELLM_URL / LIVE_MODEL).

FAQ