Realtime (Live) Sessions

createRealtime() opens a persistent WebSocket session to a provider’s live API, normalizing the two very different provider protocols (OpenAI’s typed event stream, Google’s turn-based bidirectional stream) onto one event model.

Beta. Both underlying provider APIs are in beta. Expect breaking changes from providers independent of this SDK.

Supported providers

Provider	Model example	Notes
openai	gpt-realtime-2	Full duplex, text + audio
google	gemini-3.1-flash-live	Turn-based bidirectional, audio-native

Passing any other provider throws immediately with a clear message.

createRealtime(opts) signature

createRealtime(opts: CreateRealtimeOptions): RealtimeSession

CreateRealtimeOptions:

Field	Type	Required	Notes
model	string	yes	Bare (gpt-realtime-2) or namespaced (openai/gpt-realtime-2)
provider	ProviderName	when model is bare	Ignored when model is namespaced
apiKey	string	no	Falls back to engine.apiKeys[provider]
modalities	RealtimeModality[]	no	'text' and/or 'audio'. Default ['text']
audio	AudioOptions	no	{ voice?, format? } for audio output
voice	string	no	Deprecated; use audio.voice
instructions	string	no	System-level instructions for the session
translation	{ targetLanguageCode?, echoTargetLanguage? }	no	Google. Turns the session into a live translator — see below
affectiveDialog	boolean	no	Google. Detect the speaker’s emotion and adapt the reply
inputTranscription	{ mode?: 'VERBATIM' | 'SMART' }	no	Google. How to transcribe what the session HEARS
engine	EngineHandle	no	Defaults to the registered engine

Returns a RealtimeSession immediately (synchronous). The underlying WebSocket connection opens asynchronously; listen for the 'open' event before sending.

Live translation (Google)

gemini-3.5-live-translate is a translator, and until now there was no way to tell it what to translate into — so it could be connected to and had nothing to do. translation is what makes it usable:

const session = createRealtime({
  model: 'google/gemini-3.5-live-translate',
  translation: { targetLanguageCode: 'es', echoTargetLanguage: false },
  modalities: ['audio'],
});

echoTargetLanguage decides what happens when the target language is already being spoken: true parrots it back, false stays quiet. In a two-way conversation that is the difference between hearing yourself repeated and not, so false is usually what you want — and it is sent whenever you mention it, rather than being dropped for being falsy.

Measured 2026-09-30 by streaming the same English sentence into two sessions, which is the only way to check this: a bogus target language also returns setupComplete, so “the server accepted it” proves nothing. With no translationConfig the output transcription came back empty; with targetLanguageCode: 'es' it came back “Buenos días. La reunión se ha”.

RealtimeSession interface

interface RealtimeSession {
  send(input: RealtimeInput, opts?: { turnComplete?: boolean }): void;
  on<E extends RealtimeEventType>(type: E, cb: (e: ...) => void): () => void;
  close(): void;
}

RealtimeInput:

interface RealtimeInput {
  text?: string;
  audio?: Uint8Array;   // raw audio bytes (provider-specific encoding, e.g. PCM16)
}

send() defaults to turnComplete: true — commits the turn and requests a response. Pass turnComplete: false to stream a single turn across multiple send() calls (useful for chunked audio).

on() returns an unsubscribe function. Call it to stop receiving events of that type.

Events

type RealtimeEvent =
  | { type: 'open' }
  | { type: 'text'; delta: string }
  | { type: 'audio'; chunk: Uint8Array; mimeType: string; sampleRate?: number }
  | { type: 'turnComplete' }
  | { type: 'usage'; usage: Usage }
  | { type: 'error'; error: Error }
  | { type: 'close' };

Event	When
open	WebSocket connected and session ready
text	Text delta from the model (stream chunks)
audio	Audio chunk from the model
turnComplete	Model finished a response turn
usage	Token usage reported (fires once per turn; wired into cost pipeline)
error	Transport or protocol error
close	Socket closed (normal or abnormal)

Usage events are automatically forwarded to the onCompletion hook so the CostCollector tracks and prices realtime calls alongside regular completions.

Minimal text example

import { createEngine, createRealtime } from '@combycode/llm-sdk';

createEngine({ apiKeys: { openai: process.env.OPENAI_API_KEY! } });

const session = createRealtime({
  model: 'openai/gpt-realtime-2',
  modalities: ['text'],
  instructions: 'You are a helpful assistant.',
});

const unsubText = session.on('text', (e) => {
  process.stdout.write(e.delta);
});

session.on('turnComplete', () => {
  console.log('\n[turn complete]');
  session.close();
});

session.on('open', () => {
  session.send({ text: 'Hello! Tell me a short joke.' });
});

session.on('error', (e) => console.error('Realtime error:', e.error));
session.on('close', () => unsubText());

Audio example (OpenAI)

import { createEngine, createRealtime } from '@combycode/llm-sdk';

createEngine({ apiKeys: { openai: process.env.OPENAI_API_KEY! } });

const session = createRealtime({
  model: 'openai/gpt-realtime-2',
  modalities: ['audio', 'text'],
  audio: { voice: 'alloy' },
  instructions: 'Respond concisely.',
});

// Collect audio chunks.
const audioChunks: Uint8Array[] = [];

session.on('audio', (e) => {
  audioChunks.push(e.chunk);
});

session.on('turnComplete', () => {
  console.log(`Got ${audioChunks.length} audio chunks.`);
  // Combine and play / write to file as needed.
  session.close();
});

session.on('open', () => {
  // Send pre-encoded PCM16 audio or a text prompt.
  session.send({ text: 'Say "Hello, world!" in a friendly tone.' });
});

Observability hooks

In addition to the onCompletion event (for cost tracking), the network layer emits three realtime-specific hooks on engine.hooks:

Hook	When
onRealtimeOpen	Socket connected (provider, model, url)
onRealtimeFrame	Each frame direction/size (metadata only, no payload)
onRealtimeClose	Socket closed (code, reason)
onRealtimeError	Transport error

engine.hooks.on('onRealtimeOpen', (ctx) => {
  console.log(`Realtime connected: ${ctx.provider}/${ctx.model}`);
});

Related

• Media / Files / Batch — createRealtime is also listed there alongside other media helpers
• Observability / Telemetry — onCompletion, realtime hooks
• Cost Tracking — realtime usage is metered through CostCollector