completion( )
Generates completion from a language model based on conversation history.
function completion(params): object;
| Name | Type | Required? | Description |
|---|
| params | object | ✓ | The completion parameters |
| Field | Type | Required? | Description |
|---|
| modelId | string | ✓ | Model identifier |
| history | object[] | ✓ | Conversation history (array of messages with role and content) |
| stream | boolean | ✗ | Whether to stream the response |
| tools | Tool[] | ✗ | Optional array of tools for function calling |
| kvCacheKey | string | ✗ | Optional KV cache key for persistent context |
object — Object with tokenStream generator, toolCallStream generator, text promise, toolCalls promise, and stats promise
| Field | Type | Description |
|---|
| text | Promise<string> | Complete generated text |
| tokenStream | AsyncGenerator<string> | Stream of tokens |
| toolCalls | Promise<object[]> | Tool calls made during completion |
| toolCallStream | AsyncGenerator<object> | Stream of tool call events |
| stats | Promise<object | undefined> | Performance statistics |
| Field | Type | Description |
|---|
| cacheTokens | number | Number of cached tokens |
| timeToFirstToken | number | Time to first token in ms |
| tokensPerSecond | number | Tokens per second |
Tool call event:
| Field | Type | Description |
|---|
| type | "toolCall" | Event type |
| call.id | string | Call identifier |
| call.name | string | Tool name |
| call.arguments | Record<string, unknown> | Tool arguments |
| call.raw | string | Raw call text (optional) |
Tool call error event:
| Field | Type | Description |
|---|
| type | "toolCallError" | Event type |
| error.code | "PARSE_ERROR" | "VALIDATION_ERROR" | "UNKNOWN_TOOL" | Error code |
| error.message | string | Error message |
| error.raw | string | Raw text (optional) |
import { z } from "zod";
// Simple tool definition with Zod
const result = completion({
modelId: "llama-2",
history: [
{ role: "user", content: "What's the weather in Tokyo?" }
],
stream: true,
tools: [{
name: "get_weather",
description: "Get current weather",
parameters: z.object({
city: z.string().describe("City name"),
country: z.string().describe("Country code").optional(),
}),
}]
});
for await (const token of result.tokenStream) {
process.stdout.write(token);
}
