# GPT Realtime 2.1

Model id on AIHubMix: `gpt-realtime-2.1`
Create an API key: https://console.aihubmix.com/?utm_source=llms-agent&utm_medium=model-llms

- Developer: OpenAI
- Session kind: realtime WebSocket — speech-to-speech conversation
- Input modalities: text, audio, image
- Pricing: see https://aihubmix.com/model/gpt-realtime-2.1 (realtime is billed on audio + text tokens)

## Endpoints (base URL: https://aihubmix.com)

- `GET wss://aihubmix.com/v1/realtime?model=gpt-realtime-2.1` — realtime speech-to-speech conversation over WebSocket. Authenticate the handshake with `Authorization: Bearer $AIHUBMIX_API_KEY`. The model id is carried in the handshake query above (conversation sessions send no `intent`); audio is PCM16 / 24kHz / mono in both directions.

## Example

```python
# pip install websockets
import asyncio
import base64
import json
import os
import websockets

# Realtime conversation is a WebSocket session: model must be in the handshake URL.
URL = "wss://aihubmix.com/v1/realtime?model=gpt-realtime-2.1"

async def main():
    # websockets >= 13 uses additional_headers; older versions use extra_headers
    async with websockets.connect(
        URL, additional_headers={"Authorization": "Bearer " + os.environ["AIHUBMIX_API_KEY"]}
    ) as ws:
        # 1) Configure the speech-to-speech session (voice + server VAD; no input transcription)
        await ws.send(json.dumps({
          "type": "session.update",
          "session": {
            "type": "realtime",
            "audio": {
              "input": {
                "format": {
                  "type": "audio/pcm",
                  "rate": 24000
                },
                "turn_detection": {
                  "type": "server_vad"
                }
              },
              "output": {
                "format": {
                  "type": "audio/pcm",
                  "rate": 24000
                },
                "voice": "marin"
              }
            }
          }
        }))

        # 2) Stream your mic as raw PCM16 / 24kHz / mono in ~100ms chunks. Server VAD
        #    detects when you stop talking and starts the reply automatically —
        #    no commit / response.create needed.
        async def send_audio():
            with open("audio_pcm16_24k.raw", "rb") as f:
                pcm = f.read()
            chunk = 24000 * 2 * 100 // 1000  # 100ms of 16-bit mono samples
            for i in range(0, len(pcm), chunk):
                await ws.send(json.dumps({
                    "type": "input_audio_buffer.append",
                    "audio": base64.b64encode(pcm[i:i + chunk]).decode(),
                }))
                await asyncio.sleep(0.1)  # simulate realtime pacing

        asyncio.create_task(send_audio())

        # 3) Receive the reply: audio streams as base64 PCM16 (24kHz) — write it to a file you
        #    can play; the transcript of what the model says arrives as text deltas.
        reply = open("assistant_reply_pcm16_24k.raw", "wb")
        async for msg in ws:
            evt = json.loads(msg)
            etype = evt.get("type", "")
            if etype == "input_audio_buffer.speech_started":
                # Barge-in: you started talking — stop/flush local playback here
                print("\n[listening…]")
            elif etype.endswith("audio_transcript.delta"):
                print(evt.get("delta", ""), end="", flush=True)
            elif etype.endswith("audio.delta"):
                reply.write(base64.b64decode(evt.get("delta", "")))
            elif etype.endswith("response.done"):
                print("\n[reply complete]")
            elif etype == "error":
                print("\n[error]", evt.get("error"))
                break
        reply.close()

asyncio.run(main())
```

## Response

Realtime conversation is **speech-to-speech over one WebSocket** (audio in, audio out), not a request/response body. After the first `session.update` (which sets the voice and, deliberately, **no input transcription**), stream your mic as `input_audio_buffer.append` frames; server VAD starts the reply automatically — you do **not** send `response.create`. Read events until `response.done`:

- `input_audio_buffer.speech_started` — the user began talking (flush local playback for barge-in)
- `*.audio.delta` — assistant audio as base64 PCM16 / 24kHz (write it out and play it)
- `*.audio_transcript.delta` — text transcript of what the assistant says
- `response.done` — the assistant turn finished
- `error` — a session-level error; the socket closes

Note: the user's own speech is **not** transcribed on this channel (input transcription is off by design — enabling it closes the session). No HTTP response body.

## Errors

Error responses carry a `tid` (trace id) — include it when contacting support. Reference: https://docs.aihubmix.com/en/FAQs/HTTP-Codes.md

- 400 — parameter error; most are passed through from the upstream provider (media: `prompt_missing`, `size_not_supported`, `n_not_within_range`, …)
- 401 — missing `Authorization` header, or the key is invalid/expired
- 403 — `insufficient_user_quota` (top up at https://console.aihubmix.com/?utm_source=llms-agent&utm_medium=model-llms), account suspended, or this key is not allowed to use this model
- 429 — rate limited; back off and retry
- 503 — no channel can serve the request (check the model id and your access), or the upstream provider is throttling; retry later

## More

- Model page: https://aihubmix.com/model/gpt-realtime-2.1
- Try in browser: https://playground.aihubmix.com/?model=gpt-realtime-2.1
- Full parameter schema (machine-readable, authoritative): https://aihubmix.com/model-data/models/gpt-realtime-2.1.5de3e056.json — per-protocol parameters with types, ranges, enums and defaults. Refreshed together with this page; if it ever 404s, re-resolve via `https://aihubmix.com/model-data/index.json`
- Generate runnable code programmatically: npm `@aihubmix/codegen` — the generator behind the Playground's "Get Code" (this realtime example was produced by its `generateRealtimeCode`; 7 languages, media and realtime included); the session frames it builds are the exact frames the gateway receives, so generated snippets and real requests cannot diverge
- Site index for agents: https://aihubmix.com/llms.txt · Onboarding: https://aihubmix.com/agents.md

---

Canonical version of this document: https://aihubmix.com/model/gpt-realtime-2.1/llms.txt