loadModel( )

// Load new model
function loadModel(options: LoadModelOptions): Promise<string>;

// Hot-reload config on an already-loaded model
function loadModel(options: ReloadConfigOptions): Promise<string>;

Supports multiple model types: LLM, Whisper (speech recognition), embeddings, NMT (translation), TTS, and OCR. Handles local file paths, HTTP/HTTPS URLs, Hyperdrive URLs (pear://), and registry URLs.

When onProgress is provided, streaming is used for real-time download progress. Otherwise, a simple request-response pattern is used.

Parameters

LoadModelOptions

Common fields present in all variants:

Delegate

Optional delegation configuration for remote (P2P) inference:

ReloadConfigOptions

Hot-reload configuration on an already-loaded model without reloading the model weights. Currently supported for Whisper models only.

Model type variants

Additional fields depend on modelType:

"llm"

"whisper"

"embeddings"

"nmt"

modelConfig is required and is a discriminated union on engine:

See modelConfig details below for generation parameters.

"tts"

modelConfig is required and is a discriminated union on ttsEngine:

Chatterbox engine (ttsEngine: "chatterbox"):

Supertonic engine (ttsEngine: "supertonic"):

"ocr"

Custom plugin

Any modelType string that is not a built-in type. modelConfig accepts Record<string, unknown>.

modelConfig reference

LLM modelConfig

Whisper modelConfig

Common fields:

Additional fields: n_threads, n_max_text_ctx, offset_ms, duration_ms, audio_ctx, no_context, no_timestamps, single_segment, print_special, print_progress, print_realtime, print_timestamps, token_timestamps, thold_pt, thold_ptsum, max_len, split_on_word, max_tokens, debug_mode, tdrz_enable, suppress_regex, suppress_blank, suppress_nst, length_penalty, temperature_inc, entropy_thold, logprob_thold, greedy_best_of, beam_search_beam_size. All optional. See whisperConfigSchema in the source for details.

NMT modelConfig

Discriminated union on engine. Common generation parameters (all optional):

Engine-specific:

• Opus: from/to accept "en" | "de" | "es" | "it" | "ru" | "ja"
• Bergamot: from/to accept 24 languages (en, ar, bg, ca, cs, de, es, et, fi, fr, hu, is, it, ja, ko, lt, lv, nl, pl, pt, ru, sk, sl, uk, zh). Additional fields: srcVocabPath, dstVocabPath, normalize
• IndicTrans: from/to accept 26 Indic language codes (e.g., "eng_Latn", "hin_Deva")

OCR modelConfig

ModelProgressUpdate

Returns

Promise<string> — Resolves to the model ID (used to reference the model in subsequent API calls).

Throws

Example

// Local file path
const modelId = await loadModel({
  modelSrc: "/home/user/models/llama-7b.gguf",
  modelType: "llm",
  modelConfig: { contextSize: 2048 }
});

// Remote URL with progress tracking
const modelId = await loadModel({
  modelSrc: "https://huggingface.co/.../model.gguf",
  modelType: "llm",
  onProgress: (progress) => {
    console.log(`Downloaded: ${progress.percentage}%`);
  }
});

// Hyperdrive URL
const modelId = await loadModel({
  modelSrc: "pear://<hyperdrive-key>/llama-7b.gguf",
  modelType: "llm",
  modelConfig: { contextSize: 2048 }
});

// Multimodal model with projection
const modelId = await loadModel({
  modelSrc: "https://huggingface.co/.../main-model.gguf",
  modelType: "llm",
  projectionModelSrc: "https://huggingface.co/.../projection-model.gguf",
  modelConfig: { ctx_size: 512 }
});

// Whisper with VAD model
const modelId = await loadModel({
  modelSrc: "https://huggingface.co/.../whisper-model.gguf",
  modelType: "whisper",
  vadModelSrc: "https://huggingface.co/.../vad-model.bin",
  modelConfig: {
    mode: "caption",
    output_format: "plaintext",
    min_seconds: 2,
    max_seconds: 6
  }
});

// With logger forwarding
import { getLogger } from "@qvac/sdk";
const logger = getLogger("my-app");

const modelId = await loadModel({
  modelSrc: "/path/to/model.gguf",
  modelType: "llm",
  logger
});