completion( )
Generates completion from a language model based on conversation history.
function completion(params): {
tokenStream: AsyncGenerator<string>;
toolCallStream: AsyncGenerator<ToolCallEvent>;
text: Promise<string>;
toolCalls: Promise<ToolCallWithCall[]>;
stats: Promise<CompletionStats | undefined>;
};
| Field | Type | Required? | Default | Description |
|---|
| modelId | string | ✓ | — | The identifier of the model to use for completion |
| history | HistoryMessage[] | ✓ | — | Array of conversation messages |
| stream | boolean | ✗ | true | Whether to stream tokens or return complete response |
| tools | Tool[] | ToolInput[] | ✗ | — | Optional array of tools (Zod-schema ToolInput or full Tool objects) |
| mcp | McpClientInput[] | ✗ | — | Optional array of MCP client inputs for tool integration |
| kvCache | boolean | string | ✗ | — | KV cache configuration — see kvCache |
| Field | Type | Required? | Description |
|---|
| role | string | ✓ | Message role (e.g., "user", "assistant", "system") |
| content | string | ✓ | Message content |
| attachments | Attachment[] | ✗ | Optional file attachments for multimodal models |
| Field | Type | Required? | Description |
|---|
| path | string | ✓ | File path to the attachment |
Full tool definition in JSON Schema format (sent to the model):
| Field | Type | Required? | Description |
|---|
| type | "function" | ✓ | Always "function" |
| name | string | ✓ | Tool name |
| description | string | ✓ | What the tool does |
| parameters | object | ✓ | JSON Schema object describing the tool's input |
| parameters.type | "object" | ✓ | Always "object" |
| parameters.properties | Record<string, { type, description?, enum? }> | ✓ | Parameter definitions |
| parameters.required | string[] | ✗ | Required parameter names |
Simplified tool definition using Zod schemas (auto-converted to Tool internally):
| Field | Type | Required? | Description |
|---|
| name | string | ✓ | Tool name |
| description | string | ✓ | What the tool does |
| parameters | ZodObject | ✓ | Zod schema describing the tool's input |
| handler | (args: Record<string, unknown>) => Promise<unknown> | ✗ | Handler function — when provided, returned ToolCallWithCall objects include an invoke() method |
MCP (Model Context Protocol) client input for tool integration:
| Field | Type | Required? | Description |
|---|
| client | McpClient | ✓ | An MCP client instance |
| includeResources | boolean | ✗ | Whether to include MCP resources |
Duck-typed MCP client interface:
| Method | Signature | Required? | Description |
|---|
| listTools | () => Promise<{ tools: McpTool[] }> | ✓ | Lists available tools |
| callTool | (params: { name, arguments }) => Promise<McpToolCallResult> | ✓ | Calls a tool by name |
| listResources | () => Promise<{ resources: McpResource[] }> | ✗ | Lists available resources |
| readResource | (params: { uri }) => Promise<{ contents: unknown[] }> | ✗ | Reads a resource by URI |
Cache files are organized hierarchically: {kvCacheKey}/{modelId}/{configHash}.bin
The configHash includes model config + system prompt to ensure cache isolation.
| Value | Behavior |
|---|
true | Auto-generate cache key based on conversation history |
"custom-key" | Use provided string as cache key for manual session management |
false / undefined | No caching |
When cache exists, only the last message is sent to the model (includes multimodal attachments). Use deleteCache() to remove cached sessions.
object — Object with the following fields:
| Field | Type | Description |
|---|
| tokenStream | AsyncGenerator<string> | Stream of generated tokens |
| toolCallStream | AsyncGenerator<ToolCallEvent> | Stream of tool call events |
| text | Promise<string> | Complete generated text (resolves after stream ends) |
| toolCalls | Promise<ToolCallWithCall[]> | Tool calls made during completion (may include invoke() when handler is available) |
| stats | Promise<CompletionStats | undefined> | Performance statistics |
| Field | Type | Description |
|---|
| timeToFirstToken | number | Time to first token in milliseconds |
| tokensPerSecond | number | Tokens generated per second |
| cacheTokens | number | Number of cached tokens |
Discriminated union on type. One of:
Tool call:
| Field | Type | Description |
|---|
| type | "toolCall" | Event type |
| call.id | string | Call identifier |
| call.name | string | Tool name |
| call.arguments | Record<string, unknown> | Tool arguments |
| call.raw | string | Raw call text (optional) |
Tool call error:
| Field | Type | Description |
|---|
| type | "toolCallError" | Event type |
| error.code | "PARSE_ERROR" | "VALIDATION_ERROR" | "UNKNOWN_TOOL" | Error code |
| error.message | string | Error message |
| error.raw | string | Raw text (optional) |
Extends ToolCall with an optional invoke() method:
| Field | Type | Description |
|---|
| id | string | Call identifier |
| name | string | Tool name |
| arguments | Record<string, unknown> | Tool arguments |
| raw | string | Raw call text (optional) |
| invoke | () => Promise<unknown> | Executes the tool handler (present when a matching handler was provided) |
| Error | When |
|---|
INVALID_TOOLS_ARRAY | Invalid tools array provided |
INVALID_TOOL_SCHEMA | A tool has an invalid schema |
import { z } from "zod";
const result = completion({
modelId: "llama-2",
history: [
{ role: "user", content: "What's the weather in Tokyo?" }
],
stream: true,
tools: [{
name: "get_weather",
description: "Get current weather",
parameters: z.object({
city: z.string().describe("City name"),
}),
handler: async (args) => {
return { temperature: 22, condition: "sunny" };
}
}]
});
for await (const token of result.tokenStream) {
process.stdout.write(token);
}
for (const toolCall of await result.toolCalls) {
if (toolCall.invoke) {
const toolResult = await toolCall.invoke();
console.log(toolResult);
}
}
