Realtime Client SDK

The realtime-voice client SDKs are the browser/mobile surface your end-user app uses to hold a live voice call — mic capture, send audio, receive audio and events. They are transport-abstracted behind a single VoiceClient (an RTVI-client-style Transport pattern).

Distinct from the Python SDK, which is server-side REST codegen: these are hand-authored realtime WebSocket clients.

These are implementation packages, not the customer's install path

Per ADR-122 the canonical public SDK identity is @vagary/voice-sdk (npm) and vagary-voice (PyPI). The @vagary/realtime-client* packages below are the realtime implementation — they are reused behind the canonical SDK rather than advertised as separate installs. Code reuse is not identity reuse. None of these is published yet either.

Packages

Package	Status

The JS + React packages are complete and sim-verified end-to-end; the mobile packages are structural scaffolds targeting the same wire — each ships the wire messages, a client skeleton, and a native-audio media seam.

The two wires

Both are transport-abstracted behind one VoiceClient:

Transport	Wire	Direction
GatewayTransport (default)	voice-gateway `/stream`	full-duplex: mic in → STT / dialog / TTS → audio out
RealtimeMediaTransport	realtime-media `/v1/media/stream`	mic in → VAD / turn / barge-in events out (analysis-only, no audio return)

The two wires use different audio keys on purpose — gateway audio-in is {type:"audio","data":<base64>}, realtime-media is {type:"audio","audio_base64":<base64>} — so the transports keep them distinct.

Raw wire protocol (works today, no package required)

Neither package below is published yet (ADR-122 — same status as the @vagary/voice-sdk install in getting-started). The gateway itself is live today: it is a plain WebSocket, so you can hold a session against it with nothing but a WebSocket library, the same way getting-started step 3 calls TTS with requests instead of waiting on vagary-voice. The message shapes below are not invented — they are the same Gw* types @vagary/realtime-client itself sends and parses (services/client-sdks/js/src/messages.ts, cited to the server handler that defines them).

Connecting with no ?token= is rejected at the handshake, not with a JSON error frame — confirmed live against wss://gateway.vagaryvoice.cloud/stream while writing this page:

$ python3 -c "
import asyncio, websockets
async def main():
    async with websockets.connect(
        'wss://gateway.vagaryvoice.cloud/stream',
        additional_headers={'Origin': 'https://vagaryvoice.cloud'},
    ) as ws:
        print(await ws.recv())
asyncio.run(main())
"
websockets.exceptions.InvalidStatus: server rejected WebSocket connection: HTTP 401

With a real RS256 JWT (minted server-side for the authenticated end-user — the same token GatewayTransport sends as ?token=…), a full turn looks like this:

import asyncio
import json
import os
import uuid

import websockets

TOKEN = os.environ["VAGARY_GATEWAY_TOKEN"]  # RS256 JWT, minted server-side per session — never a vgk_ product key
SESSION_ID = str(uuid.uuid4())              # the server rejects a non-UUID session id

async def main():
    uri = f"wss://gateway.vagaryvoice.cloud/stream?token={TOKEN}"
    async with websockets.connect(uri, additional_headers={"Origin": "https://vagaryvoice.cloud"}) as ws:
        print("<-", await ws.recv())  # {"type": "connected", ...}

        await ws.send(json.dumps({"type": "init", "sessionId": SESSION_ID}))
        print("<-", await ws.recv())  # {"type": "session_started", "session_id": ...}

        await ws.send(json.dumps({"type": "text_input", "text": "Hello from the raw wire"}))
        while True:
            msg = json.loads(await ws.recv())
            print("<-", msg)
            if msg["type"] == "bot_response":  # server text turn (response_consumer.ex:649-657)
                break

        await ws.send(json.dumps({"type": "close"}))

asyncio.run(main())

TOKEN is not something this page can hand you — it is minted the same way an authenticated end-user session is minted anywhere else in the platform (identity-issued, per session, never the vgk_ product API key from getting-started step 1). Everything above TOKEN — the handshake, init → session_started, text_input → bot_response — is real wire behavior, not SDK-internal plumbing, so it works whether you write it by hand (above) or let @vagary/realtime-client write it for you once that package is published.

Install (future — once published)

npm install @vagary/realtime-client
# React hooks:
npm install @vagary/realtime-client-react

Core usage (browser)

import {VoiceClient, GatewayTransport} from '@vagary/realtime-client';

const client = new VoiceClient({
  transport: new GatewayTransport({url: 'wss://gateway.vagaryvoice.cloud/stream'}),
});

client.on('transcript', (t) => console.log(t.text));
client.on('audio', () => {/* played automatically */});

await client.connect();
await client.enableMic();
// ... later
await client.disconnect();

React usage

import {VoiceClientProvider, useVoiceState, useTranscript} from '@vagary/realtime-client-react';

function CallPanel() {
  const state = useVoiceState();        // 'idle' | 'connecting' | 'connected' | ...
  const transcript = useTranscript();   // observable transcript
  return <div>{state}: {transcript}</div>;
}

@vagary/realtime-client-react is a thin useSyncExternalStore wrapper over the VoiceClient's observable store — VoiceClientProvider, useVoiceState, useVoiceEvent, useTranscript, useMicEnabled.

Verifying locally

Each package ships a sim-verify smoke suite that drives the real client / transport / codec against in-memory mock sockets and a fake mic/speaker (only the socket + media I/O are faked):

cd services/client-sdks/js && npm run smoke      # gateway JSON+binary, realtime-media, store
cd services/client-sdks/react && npm run smoke    # renders hooks via react-dom/server

Packages​

The two wires​

Raw wire protocol (works today, no package required)​

Install (future — once published)​

Core usage (browser)​

React usage​

Verifying locally​