Sutra Voice

Sutra Voice puts phone calls, IVRs and meetings under the control of BPMN processes running on the Sutra workflow engine. A flow answers a number, hears the caller one utterance at a time, speaks back, transfers, hangs up, places calls of its own, or sits in on a conference or a video meeting, recording, transcribing and analysing it through the engine’s llm channels.

Preview. Voice, meeting and video support is a preview in 0.6. The automated suites run against SIPp, a stub model and a local LiveKit server; real carriers, real models and real meeting platforms are manual checks. See Limitations.

The thesis: interactions are processes

A call is not a stream to be piped between a carrier and a model. It is a business interaction with a start, waits, decisions, retries, hand-offs and an outcome, which is exactly what a BPMN process is. When one call is one process instance:

• the menu, the retry counter and the PIN loop are model elements you can read and change, not state hidden in a media server;
• every model call goes through a named llm channel, so keys, retries, redaction and audit are the engine’s, not code in a bot;
• recordings are written to an object store and referenced by hash in the audit trail (q:audit capture="media"), and personal data is tagged and erased by the engine’s rules;
• the trunk’s line limit is an engine capacity pool, shared by inbound and outbound calls.

What it is made of

Sutra Voice does not fork the engine, and adds no BPMN element. It adds one transport (voip), a composition root that links it into an engine image, a separate WebRTC media worker for LiveKit rooms, and four runnable examples. See Architecture.

What it is not

It is not a media server. It integrates with PBXs and SFUs and leaves RTP, SIP signalling, codec negotiation and conference mixing to them. The engine side handles PCM16 and G.711 audio in pure Rust, cuts turns with an energy detector and plays audio back in real time; it never mixes streams or transcodes beyond that. It is also not a realtime speech-to-speech bridge: the flow is the control plane, turn by turn.

Where to go next

• Try it: Quick start.
• Understand it: Architecture and Concepts.
• Everything about BPMN, channels and packaging is in the Sutra book.

Sutra Voice is open source under MIT OR Apache-2.0, like the engine, and is maintained by Startr Trading Technologies Private Limited. Source: github.com/startr-trade/sutra-voice.

Quick start

This runs the ivr-dtmf example: a DTMF phone menu with no model, on your machine, with no phone line. rustpbx is the PBX, and SIPp plays the caller and the phones of the extensions. The output below was copied from a real run; ids, ports and times differ on every run.

Prerequisites

Docker with Compose v2, and a clone of the repository with its submodule:

git clone --recurse-submodules https://github.com/startr-trade/sutra-voice
cd sutra-voice

Until 0.6.0 is published, build the engine image (the engine plus the voip transport) and the sutra CLI from the clone. The CLI comes from the engine submodule; its first build takes several minutes.

make image
cargo build --manifest-path engine/rust/Cargo.toml --profile release-it -p sutra-cli --target-dir rust/target
export PATH="$PWD/rust/target/release-it:$PATH"
export SUTRA_ENGINE_IMAGE=sutra-voice-engine:dev

1. Package and start

C=examples/ivr-dtmf/deploy/compose.yaml
sutra package examples/ivr-dtmf/package --out examples/ivr-dtmf/deploy/deployments | grep '^packaged'
docker compose -f $C --progress quiet up -d --build
until PORT=$(docker compose -f $C port engine 8080 2>/dev/null | cut -d: -f2) &&
      curl -sf "http://127.0.0.1:$PORT/sutra/deployments" | grep -q '"phase":"Active"'; do sleep 1; done
until docker compose -f $C logs ext-203 | grep -q registered; do sleep 1; done
docker compose -f $C logs ext-201 | grep -o 'extension 201 registered'

packaged examples/ivr-dtmf/deploy/deployments/package.sutra (deploymentId dep-8051dbc2291932f5c9d64341)
extension 201 registered

The four extension phones register with the PBX as they start.

2. Place a test call

The caller takes the PBX address, the number it calls from, and the keys to press with pauses between them. 2500ms waits, 2 presses 2, and 4321# presses 4, 3, 2, 1 and #. This call is from an unknown number to a customers-only extension (support, 202), so it is asked for a PIN:

docker compose -f $C --progress quiet run --rm caller \
    pbx:5060 +15550009999 2500ms 2 2500ms 4321# >/dev/null 2>&1; echo "the caller exited with $?"

the caller exited with 0

3. Read the log

docker compose -f $C exec -T engine-db psql -U postgres -d sutra -c \
  "select caller, extension, outcome, method, reason, note, menu_attempts, pin_attempts
     from transfer_log order by logged_at"

    caller    | extension |   outcome   | method | reason |               note                | menu_attempts | pin_attempts 
--------------+-----------+-------------+--------+--------+-----------------------------------+---------------+--------------
 +15550009999 | 202       | transferred | pin    |        | shared with the partner help desk |             1 |            1
(1 row)

One row with outcome transferred, method pin and the PIN’s note. The digits themselves are never logged. The extension’s phone noted the call it answered:

docker compose -f $C exec -T ext-202 cat /capture/ext-202.log

answered

4. Another path: a wrong PIN, three times

The ivr-dtmf example has a call for every path. This one enters three wrong PINs for accounts (203), and the menu gives up and hangs up:

docker compose -f $C --progress quiet run --rm caller \
    pbx:5060 +15550009999 2500ms 3 2500ms 0000# 2500ms 1111# 2500ms 2222# >/dev/null 2>&1; echo "the caller exited with $?"
docker compose -f $C exec -T engine-db psql -U postgres -d sutra -c \
  "select extension, outcome, reason, pin_attempts from transfer_log where extension = '203' order by logged_at"

the caller exited with 0
 extension |   outcome   |         reason         | pin_attempts 
-----------+-------------+------------------------+--------------
 203       | pin-refused | wrong                  |            1
 203       | pin-refused | wrong                  |            2
 203       | pin-refused | wrong                  |            3
 203       | gave-up     | pin-attempts-exhausted |            3
(4 rows)

5. Stop

docker compose -p sutra-ivr-dtmf down -v

Next: how it fits together in Architecture, or the ivr-dtmf example in full.

Overview

The boundary: engine and voice pack

There is exactly one BPMN implementation, and it is the engine’s. Nodes, gateways, events, wait states, channels, llm channels and the q: namespace are all engine features, documented in the Sutra book. Sutra Voice consumes the engine as a git submodule and adds only what the engine’s extension points allow:

The generic seams the pack relies on stay in the engine: llm channels with audio input and media references, the llm channel registry, q:audit capture="media", the object store (with multipart uploads, ranged reads and listings), the transport audit journal, capacity pools and session-end.

Components

flowchart TB
    Caller[Caller phone] -->|SIP and RTP| PBX[Gateway: PBX]
    Room[Video meeting] -->|WebRTC| SFU[LiveKit SFU]
    SFU --> Worker[WebRTC worker]
    PBX -->|media stream and call control| Voip
    Worker -->|media stream| Voip
    subgraph Engine[Sutra engine image]
        Voip[voip transport] --> Flow[BPMN process instance]
        Flow --> LLM[llm channels]
        Flow --> Store[Data stores]
        Voip --> Obj[Object store: recordings and segments]
        LLM -->|media-store reads references| Obj
    end
    LLM --> Model[Model provider]

The media-stream protocol

The transport serves two WebSocket endpoints on the engine’s own HTTP listener, with no extra port: /voip/bridge, the PCM bridge a PBX dials, and /voip/stream, the gateway-neutral media-stream protocol. The protocol carries linear PCM (or G.711) frames in both directions plus small control frames, and one socket can carry a whole meeting: every participant’s own audio and sampled camera frames, tagged by participant. Any gateway that speaks it can drive the engine; the WebRTC worker is one such gateway.

Gateways

A gateway terminates SIP and RTP, negotiates codecs, transcodes at the edge and exchanges PCM or G.711 with the engine. G.729, G.722 or Opus never enter Sutra.

• rustpbx is supported today (checked against 0.5.1). Inbound calls reach the engine through its IVR app in step mode; the transport takes control of the call over rustpbx’s RWI socket to originate, transfer and hang up.
• Generic: the media stream is the whole interface. Call control (originate, transfer) is what a gateway-specific adapter such as the rustpbx one adds.
• Asterisk and FreeSWITCH adapters are planned, see the Roadmap.

The WebRTC worker

For a LiveKit room the worker joins as a visibly labelled bot, subscribes to each participant’s audio and (when requested) camera, and relays them to the engine over the media-stream protocol, one session per meeting. See Meetings and the WebRTC worker.

Recording and the object store

A call or meeting that is recorded streams to an S3-compatible object store through the engine’s object store (multipart uploads of sealed one-second chunks, recoverable after a crash), capped by a maximum duration, and referenced by hash from the audit trail (q:audit capture="media"), never embedded in it. A meeting’s segments are written to the same store, sealed, for the flow to hand to a model by reference. See Recording and consent.

Analysis lives in flows

The transport handles media and computes generic, domain-free measurements (turn and call features, acoustic features of a segment). Anything that interprets them, such as an emotion or engagement analysis, who may be analysed, the prompt and the fusion of the signals, is in the package’s BPMN, DMN and FEEL, versioned with the flow and audited like any other step. See Meetings and the WebRTC worker.

Every model call is a named channel

The transport has no model settings of its own: a voip channel that names a provider, model or key is refused at load. A responder turn, a live transcription and a segment or post-call analysis are all llm channels declared in the package, with the engine’s secret references, retries, redaction and audit. The transport calls one of them itself (the live transcription session); the flows call the rest. Swapping a provider, or the stub used by the tests, is a channel change and not a code change.

Capacity

A trunk’s concurrent-call limit is an engine capacity pool, shared by inbound and outbound calls, and a call holds one slot until its hang-up. See Session end and capacity pools.

Calls and the media protocol

An inbound call

sequenceDiagram
    participant C as Caller
    participant P as PBX (rustpbx)
    participant T as voip transport
    participant F as Process instance
    participant M as llm channel
    C->>P: INVITE (SIP)
    P->>T: webhook: call_created
    T->>F: call-start message (x-call-id)
    P->>T: media WebSocket opens
    C->>P: speech (RTP)
    P->>T: PCM frames
    Note over T: energy VAD cuts a turn
    T->>F: call-turn (audio + features)
    F->>M: audio turn
    M-->>F: spoken answer
    F->>T: play (audio)
    T->>P: paced 20 ms frames
    P->>C: answer (RTP)
    C->>P: hangs up
    P->>T: webhook: call_hangup
    T->>F: call-hangup

The call is one process instance. After each answer it returns to the same wait, an event-based gateway on call-turn, call-dtmf, call-hangup and an idle timer, so the whole conversation is a loop in the model. The transport delivers a call’s messages strictly in order and holds the next until the instance is ready for it.

Why playback is paced

Audio goes to the gateway in real time, 20 ms per frame, never pre-buffered. When the caller starts speaking while an answer is playing, the transport stops sending: that is barge-in, and it only works because there is nothing already queued in the PBX that could not be taken back.

An outbound call

A flow places a call with an voip://originate operation on an outbound channel. The transport asks the gateway to originate through the configured trunk, waits for the outcome and reports it on call-progress; once answered, the call is handed to the engine’s media bridge and from then on looks like any other call. With a capacity pool an originate waits in the outbox until a line is free.

The full channel and operation reference is in The voip transport.

Meetings and the WebRTC worker

Two ways into a meeting

• A conference dial-in. The flow originates a call to the conference number, sends the meeting PIN as DTMF, announces recording, and from then on listens (a listen-only call). The meeting arrives as one mixed stream from the PBX.
• A video meeting. The WebRTC worker joins a LiveKit room as a participant and streams each participant separately, audio and sampled frames, over one tagged socket per meeting.

The meeting session flow

sequenceDiagram
    participant R as LiveKit room
    participant W as WebRTC worker
    participant T as voip transport
    participant F as Process instance
    participant O as Object store
    participant M as llm channels
    W->>R: join as labelled bot
    W->>T: open session (media stream)
    T->>F: call-start (meeting)
    R-->>W: participant audio and camera
    W->>T: tagged audio and frames per participant
    T->>O: record each participant (sealed)
    T->>M: transcribe each participant live
    T->>F: call-participant (join / leave)
    T->>F: call-transcript, call-keyword
    loop each segment
        T->>O: write the segment's audio and frames (sealed)
        T->>F: call-segment (media references)
        F->>F: policy (DMN): analysed?
        F->>M: analyse, media as {ref, sha256}
        M->>O: read, check digest, open
        F->>F: fuse (FEEL), store the row
    end
    R-->>W: room empty
    W->>T: end session
    T->>F: call-hangup (transcript, recordings)
    F->>F: digest, report

Only sparse events and references reach the flow, never media: a participant joining, a phrase being heard, a segment’s references, the end of the meeting. The transport handles media and generic measurements only; everything that interprets them is in the flow.

Where the analysis lives

So a prompt or a policy changes with a package version, not an engine upgrade, and every decision is a step in the instance’s audit trail.

Memory

Per participant the transport holds a bounded window of audio and only the frames, transcript lines and flagged moments of segments not yet written, so its memory does not grow with the length of the meeting. Recordings stream to the store.

Who is analysed

The role of each participant is declared by whoever issues the room token, never inferred. The call-segment channel cuts only the roles it lists, and the flow’s policy decides who of those is analysed; a participant who is not analysed never has media sent to a model. See Emotion analytics and compliance.

A call is a process instance

One call is one instance of one BPMN process. It starts when the call-start channel, bound to a number (bind: "number:+15551230000"), receives the call, and it ends when the process reaches an end event, which may be well after the hang-up if there is post-call work to do.

The instance holds the conversation in its variables. Every message of the call carries the call’s id in the x-call-id header (plus x-caller, x-called and x-call-direction), and the start event declares an alias over that header so that later messages find their instance:

<q:alias name="callKey" expression="getValue(header, &quot;x-call-id&quot;)"
         unique="true" onConflict="correlate"/>

Waits

Everything the call does between steps is a wait: for the caller’s next turn, for a key, for the hang-up, for an idle timer. A conversation is therefore a loop that returns to the same event-based gateway after every answer, and each pass is a new wait with its own timer and correlation. Give every wait of the call an arm for call-hangup: a message for a wait the instance is not parked at is refused.

Why this matters

• A retry rule is a loop with a counter in the model, visible in the diagram, not a branch in a bot’s code.
• The same engine features apply to a call as to any process: audit, personal-data tagging and erasure, redaction of sensitive variables, deployment packages, hot reload.
• Post-call work (analysis, a case record, a follow-up) continues in the same instance with the call’s data to hand.

See Calls and the media protocol for the message sequence.

Turns, barge-in and DTMF

Turns

The transport is turn-based. An energy-based voice activity detector cuts the caller’s audio into utterances: speech ends a turn after silence-ms of silence, or after max-turn-ms. Each turn is delivered to the flow as one call-turn message with the audio and deterministic acoustic features (duration, voiced ratio, pitch and energy contours, overtalk).

Turn detection needs a continuous media stream. A gateway that sends nothing during silence ends a turn only when audio resumes or at max-turn-ms.

Barge-in

Playback is paced in real time at 20 ms per frame. When the caller starts speaking while an answer plays, the transport stops sending, so an interrupted answer reports interrupted: true and the flow can react. A play can be made uninterruptible with x-barge-in: false.

DTMF

Keys arrive as call-dtmf messages, and the flow can send keys with voip://dtmf (to enter a conference PIN, for example). A flow collects digits with a loop: one wait per key until a terminator such as # or a timer, with a counter for retries. Keys pressed while a prompt plays are kept and handled in order when it ends, so a caller who knows the menu can type ahead. The dtmf operation never echoes the digits back, because a PIN is a secret.

The ivr-dtmf example is a complete menu and PIN flow built this way.

Session end and capacity pools

A trunk allows a fixed number of concurrent calls, shared by inbound and outbound. That limit is declared once as an engine capacity pool and referenced from the channels:

sutra:
  capacity-pools:
    trunk-main:
      slots: 30
      reserve: { inbound: 6 }      # outbound can never take the last 6

  - name: call-start
    transport: voip
    bind: "number:+15551230000"
    pool: trunk-main
    pool-call-key: header.x-call-id    # one slot per call
    session-end:
      on-message: call-hangup          # the hang-up frees the line; the instance runs on

• A call to the number takes a slot or is refused busy.
• An originate waits in the outbox, oldest first with no attempt counted, until a line frees, so a batch of calls paces itself to the trunk.
• The slot belongs to the call, not the instance. session-end returns it at the hang-up even when the instance goes on with post-call work, and an end or failure returns anything still held.

The pool settings are engine configuration: see the configuration reference in the Sutra book.

Mixed and per-participant media

A conference dial-in is one mixed channel, so a live transcription cannot say who spoke. Providers such as OpenAI Realtime and Gemini Live label no speakers, so the transcript marks the speaker as unknown with the source none; attribute speakers afterwards, for example with a model over the transcript, and say that they were inferred.

Per-participant media is where emotion and engagement analysis, if you use it, is possible at all, and also where the compliance obligations bite hardest. See Emotion analytics and compliance.

Sutra Voice does not mix: a mixed stream is whatever the PBX produced. Hosted per-participant audio on PBXs is on the Roadmap.

Phone and VoIP calls: the voip transport

The voip transport lets a flow take part in a phone call: answer a number, hear the caller one utterance at a time, speak back, transfer the call, hang up — and place calls of its own. It is turn-based: the caller’s audio is cut into turns and each turn is a message; the flow’s answers are played back in real time.

The transport never touches telephony codecs. A media gateway (a PBX such as rustpbx, or any gateway speaking the media-stream protocol below) negotiates each call leg’s codec, transcodes at the edge, and exchanges linear PCM (or G.711) with the engine. So G.729, G.722 or Opus never enter Sutra; it handles PCM16, G.711 µ-law / A-law and WAV, in pure Rust.

It ships in the sutra-voice-engine image (make image in this repository): the Sutra engine’s default bundle plus this transport. The public engine image does not include it. Everything else a flow uses — channels, llm channels, wait states, capacity pools — is the engine’s, documented in the Sutra book.

How a call reaches a flow

caller ── SIP/RTP ──▶ gateway (PBX) ── media WebSocket ──▶ engine: voip transport ──▶ channels ──▶ flow
                         ▲                                         │
                         └──────── control (rustpbx: RWI) ◀────────┘  originate · transfer · hang-up

• Media arrives on a WebSocket served over the engine’s own HTTP listener (no extra port): GET /voip/bridge — the PCM bridge a PBX dials — or GET /voip/stream — the gateway-neutral media-stream protocol.
• Turns. An energy VAD cuts the caller’s audio into utterances: speech ends a turn after silence-ms of silence, or after max-turn-ms. Each turn is delivered with deterministic acoustic features (duration, voiced ratio, pitch and energy contours, overtalk).
• Playback is paced in real time, 20 ms per frame, so when the caller starts speaking while an answer plays, the transport simply stops sending (barge-in) — it never pre-buffers audio into the gateway where it could not be taken back.
• Every message of a call carries its id in the x-call-id header, plus x-caller, x-called (the numbers, not SIP URIs) and x-call-direction. A flow correlates on x-call-id.

Engine configuration

One gateway per engine, configured through environment variables. Secrets are references (env:…, secret:…), never literals.

Use TLS (wss://) between gateway and engine anywhere outside a private network.

Media credentials

Tokens are redacted from the transport’s log lines.

Channels

Inbound

Each inbound channel declares transport: voip and the event it receives with event: (a call-start channel binds numbers instead). One channel per event per package.

channels:
  - name: call-start
    transport: voip
    bind: "number:+15551230000"   # a list (number:+1…,+1…), a prefix (number:+1555123*), or number:*
    codec: json
    silence-ms: 700               # this much silence ends a turn
    max-turn-ms: 30000            # a turn never runs longer
    dead-air-ms: 2000             # silences at least this long count as dead air

  - name: call-turn
    transport: voip
    event: call-turn
    codec: json                   # turn-body: json (default) | wav — the WAV itself, audio/wav

  - name: call-progress
    transport: voip
    event: call-progress
    codec: json
    statuses: [connected, busy, no-answer, failed]   # deliver only these (default: all)

Number binding. The most specific bind: wins — an exact number over the longest prefix over the catch-all; numbers compare on their digits (+1 (555) 123-0000 = 15551230000). Two channels binding the same number or prefix across the active deployments fail closed (SUTRA.VOIP.BIND_DUPLICATE); a number nothing binds is hung up at once.

Features. A turn’s features: durationMs, voicedMs, voicedRatio, energyDbfs and pitchHz (each {count, mean, min, max, stddev, slopePerS}), overtalkMs, interruptedPlayback. The hang-up’s features cover the whole call: observedMs, callerTalkMs, agentTalkMs (the flow’s own audio), callerTalkRatio, silenceMs, deadAirMs, deadAirEvents, longestSilenceMs, overtalkMs, interruptions, turns. All are computed from the audio, with no model involved.

Outbound: voip://<operation>

• play takes any format the audio module knows (audio/wav, audio/L16;rate=<hz>, audio/PCMU, audio/PCMA) and resamples it to the call’s rate — so a model’s spoken answer (answer.audio) plays as is. As a synchronous channel (ack-mode: on-complete) the task ends when the audio has played; header x-barge-in: false makes it uninterruptible.
• Recorded prompts. {callId, prompt: "ivr/menu"} plays <SUTRA_VOIP_PROMPTS_DIR>/ivr/menu.wav (.wav is implied when the name has no extension). A name is /-separated segments of letters, digits, ., _ and - — never .. — so it cannot leave the directory. The file is read when it is played, so a recording can be replaced without redeploying anything; a missing or non-WAV file fails the play (SUTRA.VOIP.AUDIO_INVALID).
• originate: the flow may name the call itself (callId), so it can correlate before the PBX knows the call; otherwise an id is minted from the outbox entry, so a redelivery never dials twice. A bare number (+15550001111) is dialled through the trunk.
• Toll-fraud controls. originate is deny-all until SUTRA_VOIP_ORIGINATE_ALLOW lists the destinations it may dial. Premium-rate prefixes and international numbers are refused unless allowed explicitly, and per-destination and global rate limits apply. A refusal is a permanent failure (SUTRA.VOIP.ORIGINATE_REFUSED), a rate limit a retryable one (SUTRA.VOIP.ORIGINATE_RATE_LIMITED); both are written to the audit journal (Audit events) with the destination masked.
• Recording an originated call. A request from a <q:audit capture="media"> process is recorded (or one saying record: true). record: "after-consent" makes the recording wait for consent, like a record: after-consent channel: it starts only once a play marked consent: true has played to the end. See Recording and consent.
• An operation on a call that has just ended answers status: "ended" rather than failing — a caller hanging up while the flow is mid-step is an ordinary race.

Listen-only calls

A flow that only listens — a meeting assistant, say — originates with mode: "listen": no turns are cut and nothing barges in; the flow may still play an announcement. The media plane stays in the transport, and two consumers share the far end’s audio:

• Recording. A call whose originate request comes from a <q:audit capture="media"> process (the engine marks the request sutra-audit-capture: media) is recorded: the far end’s audio, filled with silence where the gateway sent nothing (so the recording keeps the call’s clock), with the transport’s own prompts mixed in where they played. It streams to SUTRA_VOIP_RECORDING_STORE while the call is up, sealed chunk by chunk, and survives a crash; the call-hangup carries its REFERENCE (recording, and every recording of the call in recordings). How it is written, sealed, recovered and gated on consent: Recording and consent.
• Live transcription. A call-transcript channel opens a live audio session for each listen-only call, transcription-only (it never answers), THROUGH the llm channel of the same deployment that transcribe.channel names — the model, its key and any fallback are configured there, once, like every model call of the package (the llm channels guide, “Channels a transport calls”). The llm channel must require: [realtime], call (not mode: job) and exist; otherwise the voip channel set fails at load with SUTRA.VOIP.MODEL_CHANNEL_INVALID, as does a transcribe.provider / .model / .api-key written on the voip channel itself. The audio goes from the transport to the provider; it never reaches a process. It delivers each final segment as one message (with channel, provider, model: the candidate that served); startMs / endMs are offsets into the call’s audio, and so into its recording. transcribe.drain-ms (3000) is how long the session may finish after the hang-up; the call-hangup waits for it, so every segment precedes it, and reports transcript: {status, segments, keywords, channel, provider, model, speakerLabels, audioMs, errors}; an llm channel that is not live, or whose every candidate failed to open a session, answers status: "unavailable" with the reason. Speakers: OpenAI Realtime and Gemini Live label no speakers, and a conference dial-in is one mixed channel, so speaker is "unknown" and speakerSource "none" unless a provider labels one — attribute speakers afterwards (a model over the transcript) and say that it was inferred.
• Keywords. A call-keyword channel’s phrases: (case-insensitive) raise one message each time a final segment contains one — sparse events for the flow, not a stream.

  - name: call-transcript
    transport: voip
    event: call-transcript
    codec: json
    transcribe:
      channel: meeting-transcription   # an llm channel of this deployment
      language: en
  - name: meeting-transcription
    direction: outbound
    transport: llm
    bind: "llm://meeting-transcription"
    ack-mode: on-complete              # the transport calls it inline: no reply channel
    llm:
      require: [realtime]              # live audio sessions (openai *realtime*, gemini *live*)
      candidates:
        - provider: openai
          model: gpt-realtime
          transcription-model: gpt-4o-transcribe
          api-key: env:OPENAI_API_KEY  # a reference
  - name: call-keyword
    transport: voip
    event: call-keyword
    codec: json
    phrases: [action item, deadline]

Joining a conference by its SIP dial-in: originate, wait for connected, then send the PIN (voip://dtmf, or dtmfAfterAnswer on the originate) — over the rustpbx bridge the digits reach the far leg as RFC 2833 telephone events. examples/meeting-assistant is the whole flow.

Meeting sessions (many participants, audio and video)

A gateway that sits in a meeting as a participant can stream every other participant on ONE socket of the generic protocol — each participant’s own audio and sampled camera frames — and the transport records and transcribes each participant separately, and cuts each one’s media into segments it hands the flow as references. The transport does not analyse: the flow decides what to analyse and calls the model. The protocol extension, the channel keys (mode, record, video.* on the call-start channel; the call-participant and call-segment channels) and the WebRTC worker that does this for a LiveKit room are described in The WebRTC worker.

Lines: capacity pools and session-end

A trunk allows a fixed number of concurrent calls, shared by inbound and outbound. Declare it once as an engine capacity pool and reference it from the channels:

# engine configuration (or SUTRA_CAPACITY_POOLS=trunk-main, SUTRA_CAPACITY_POOLS_TRUNK_MAIN_SLOTS=30, …)
sutra:
  capacity-pools:
    trunk-main:
      slots: 30
      reserve: { inbound: 6 }      # outbound can never take the last 6

  - name: call-start
    transport: voip
    bind: "number:+15551230000"
    pool: trunk-main
    pool-call-key: header.x-call-id    # one slot per call
    session-end:
      on-message: call-hangup          # the hang-up frees the line; the instance runs on

  - name: originate
    direction: outbound
    transport: voip
    bind: "voip://originate"
    pool: trunk-main
    pool-call-key: header.x-call-id

A call to the number takes a slot or is refused busy. An originate waits in the outbox — oldest first, no attempt counted — until a line frees, so a batch of calls paces itself to the trunk. The slot belongs to the call: session-end returns it at the hang-up even when the instance goes on (post-call work), and End or failure return anything still held.

Writing the flow

• Correlate on the call id. Give the process that starts on call-start a start-event alias over the header, and every later message of the call finds it: <q:alias name="callKey" expression="getValue(header, &quot;x-call-id&quot;)" unique="true" onConflict="correlate"/>. For an originated call, name the call before dialling (callId in the request) and alias on it.
• A conversation is a loop. One instance is the whole call: after each answer the flow goes back to the same wait — an event-based gateway on call-turn, call-dtmf, call-hangup and an idle timer — and every pass is a new wait with its own timer and correlation. Keep the conversation in the instance’s variables. The transport delivers a call’s messages strictly in order, one at a time, and holds the next while the instance is still busy with the last, so turns never overlap and a key pressed during a prompt is handled when the prompt ends. Give every wait of the call an arm for call-hangup: a message for a wait the instance is not parked at is refused.
• Collect digits with a loop. A PIN or an account number is one wait per key, looping back until the terminator (#) or a timer; a retry (invalid choice, wrong PIN) loops back to the prompt with a counter.
• Wait once per attempt. With statuses: on call-progress, an event-based gateway after an originate sees exactly one outcome (or its timer); a retry loops back to the dial with a counter, and each attempt is a new wait.

Three complete, runnable examples:

• examples/voice-assistant/ — an inbound responder (an audio-native model, the conversation as one looping instance) with post-call analytics, and appointment-reminder calls with a retry loop.
• examples/ivr-dtmf/ — a plain DTMF phone menu, no model: recorded prompts, a decision table for the menu, customer lookup and PIN entry against data stores, transfer to PBX extensions, and a transfer log.
• examples/meeting-assistant/ — a listen-only bot: dials a conference, enters its PIN, announces recording and transcription, records (sealed, audited by reference) and transcribes the meeting live, and reports afterwards (minutes, decisions, action items, per-participant talk time and — for analysed parties only — sentiment).

rustpbx

Checked against rustpbx 0.5.1 (ghcr.io/restsend/rustpbx — it publishes only a moving latest tag, so pin the image by digest). Where its documentation and its source disagree, the transport follows the source.

• Inbound. Route the number to rustpbx’s IVR app in step mode, with the engine as its provider: app = "ivr", app_params = { mode = "step", url = "<engine>/voip/rustpbx/ivr-step?token=<media token>" }. The engine answers every step with a media bridge to itself. Add the webhook tap so the engine learns of the call and takes control of it over RWI: [rwi_webhook] url = "<engine>/voip/rustpbx/webhook?token=<media token>", events call_created, call_hangup. (rustpbx announces no inbound call on the RWI socket itself.)
• Outbound. call.originate through a trunk (SUTRA_VOIP_TRUNK), then the call is transferred to the engine’s media bridge once answered. The trunk must list its codecs (codec = ["pcmu", "pcma"]): rustpbx builds an originated call’s offer from them and refuses the call without.
• Hang-up always goes over RWI: rustpbx drops the media socket without a close frame when a call ends, and closing the engine’s side alone would leave a silent call up.
• A trunk is recognised by the SOURCE address of the carrier’s signalling (inbound_hosts).
• RTP uses rustpbx’s rtp_start_port–rtp_end_port; keep the range small and publish exactly it (UDP) when a carrier reaches the PBX from outside its network.
• Video is not available through rustpbx: its media bridge is audio only.

Limits

• Turn detection needs a continuous media stream: a gateway that sends nothing during silence (no RTP during silence suppression) ends a turn only when audio resumes or at max-turn-ms.
• One gateway per engine process.
• Realtime speech-to-speech (media straight between the call and a model’s live API, the flow as the control plane) is not part of this transport.

Audit events

The transport writes its own events to the engine’s audit journal, through the engine’s transport-audit interface: it answers writes_audit_events() = true, so the engine stamps every request it receives with the audit routing of the instance that sent it. An event about a call goes to the journal of the instance that owns the call (an inbound call learns it from the first request of its instance; a recording keeps it in its manifest, so a recovery after a restart still writes there). An event no instance owns goes to the deployment’s journal.

The fields are written only when the process audits at payload level; at any other level the event keeps its type, diagnostic code and time with an empty payload. With no audit sink, or a process that is not audited, nothing is written. In audit_event (SUTRA_AUDIT_SQL=true) the rows carry event_type voip.<EVENT> and the instance’s id.

Diagnostics

SUTRA.VOIP.BIND_INVALID, SUTRA.VOIP.BIND_DUPLICATE (number binding), SUTRA.VOIP.CHANNEL_INVALID (a channel’s settings), SUTRA.VOIP.CONFIG_INVALID (the environment), SUTRA.VOIP.OUTBOUND_INVALID (a request), SUTRA.VOIP.CALL_UNKNOWN, SUTRA.VOIP.CALL_NOT_CONNECTED (no media yet — retried), SUTRA.VOIP.AUDIO_INVALID, SUTRA.VOIP.GATEWAY_UNAVAILABLE (retried), SUTRA.VOIP.GATEWAY_REJECTED, SUTRA.VOIP.NOT_RUNNING, SUTRA.VOIP.RECORDING_UNAVAILABLE (a call to be recorded, no store), SUTRA.VOIP.CONSENT_REQUIRED (record-start before consent), SUTRA.VOIP.ORIGINATE_REFUSED, SUTRA.VOIP.ORIGINATE_RATE_LIMITED (retried), SUTRA.VOIP.RECORDING_RECOVERED (on the recovery audit events), SUTRA.VOIP.MODEL_CHANNEL_INVALID (a transcribe.channel that is not a usable llm channel).

Video meetings: the WebRTC worker and meeting sessions

Sutra can sit in on a video meeting — a LiveKit room — record each participant, transcribe each participant’s own track, and hand the flow each participant’s audio and sampled camera frames segment by segment, as media references, for the flow to analyse (expression, engagement, body language, emotion) with a model of its choosing. Two pieces do the media part:

• sutra-webrtc-worker — a separate binary (its own image) that joins the room as a visible participant, subscribes to everyone else’s audio and video, and streams them to the engine. It is the only piece that links libwebrtc; the engine never does.
• The voip transport’s meeting sessions — an additive extension of its generic media-stream protocol: one socket carries every participant, tagged.

The runnable example is examples/video-meeting-assistant/ (compose stack, stub and live variants, README).

How a meeting reaches a flow

LiveKit room ──(WebRTC)──> sutra-webrtc-worker ──(WebSocket /voip/stream, tagged)──> engine
                                                                                      │
   call-start (session) ─ call-participant (joined/left) ─ call-transcript (per track)
   call-segment (per participant, per segment: media references) ─ call-hangup (references + summary)
                                                                                      │
   the flow: policy (DMN) ─ prompt (FEEL) ─ llm channel with {ref, sha256} parts ─ fusion (FEEL)

1. The worker waits in the room. When the first other participant is present it opens a session: a start frame presenting the room as a call to SUTRA_WORKER_CALLED, which a call-start channel binds (bind: number:…).
2. Each participant is announced with a session index, its display label and the role it was admitted with (a room-token attribute) — never an identity, and never an inferred role.
3. Audio of each participant, and frames of each camera at the rate the channel sets, flow on the same socket. The flow sees none of it.
4. As each segment closes, the transport writes each segmented participant’s audio and frames of it to the recording store (sealed) and delivers a call-segment message with their references. The flow decides whether to analyse it and calls the model.
5. When the room empties, the worker stops the session; the transport finishes (transcripts drained, the last segment written and delivered, recordings completed) and delivers call-hangup.

The protocol extension

A session is tagged when its start frame says "streams": "tagged". Everything else in the generic protocol is unchanged.

Gateway → engine, text frames

Gateway → engine, binary frames — a 12-byte header, then the payload:

Engine → gateway

Channel keys

On the call-start channel:

Channels

A call-participant channel receives {callId, event: joined|left, participantId, participantIndex, label, role, segmented, atMs} (segmented: the participant’s role is one the call-segment channel cuts).

A call-transcript channel (transcribe.channel, as for listen-only calls) now opens one live session per participant through that llm channel (each holds one of its concurrency permits, so size it for the participants): every segment carries speaker = the participant’s label, speakerSource: "track" and participantId, with offsets in session time. Attribution is exact.

Segments: media for the flow

A call-segment channel cuts the session into fixed segments and hands the flow each participant’s media of each one. The transport analyses nothing: it writes and measures.

As each segment closes (a second or two after its end, so the transcript lines of it have arrived), and once more for the last, partial one before the hang-up, the transport writes each segmented participant’s audio of the segment as a WAV file and each of its frames as an image to SUTRA_VOIP_RECORDING_STORE, under <tenant>/<module>/<version>/<callId>/segments/p<index>/<segment>.wav and …/<segment>-f<n>.jpg. Each object is sealed (.sealed, the engine’s single-shot sealed-object format SUTRA-SEALED-1, under SUTRA_VOIP_RECORDING_KEK) unless the encryption is none. Then one message per participant:

{"callId": "standup-1", "key": "standup-1:2:3", "participantId": "PA_sam", "participantIndex": 2,
 "participant": "P2", "label": "Sam", "role": "external", "declaredRole": "external",
 "segment": 3, "startMs": 10000, "endMs": 15000, "speaking": true, "talkMs": 3120, "audioMs": 5000,
 "features": {"durationMs": 5000, "voicedRatio": 0.62, "energyDbfs": {…}, "pitchHz": {…}},
 "flaggedMoments": [{"atMs": 1200, "reason": "turn-boundary", "participant": "P1"}],
 "words": [{"speaker": "P1", "startMs": 0, "endMs": 900, "text": "…"}],
 "present": [{"participant": "P1", "role": "staff", "talkMs": 900}, …],
 "audio": {"kind": "media", "uri": "s3://…/segments/p2/3.wav.sealed", "sha256": "…",
           "participant": "PA_sam", "startMs": 10000, "endMs": 15000, "format": "audio/wav",
           "rate": 16000, "durationMs": 5000, "bytes": 160195, "sealed": true},
 "frames": [{"kind": "media", "uri": "s3://…/segments/p2/3-f1.jpg.sealed", "sha256": "…",
             "participant": "PA_sam", "startMs": 11240, "endMs": 11240, "atMs": 1240,
             "requested": true, "format": "image/jpeg", …}],
 "status": "complete"}

features are generic acoustic measurements (duration, voicing, energy and pitch contours) and words, present and flaggedMoments name participants only by pseudonym. A segment whose media could not be written says status: "failed" with a detail; with no recording store it says unavailable.

Analysing a segment is the flow’s. It decides whether to (a decision table over the declared role, say), builds the prompt, and calls an llm channel with the media as references; the channel’s media-store names the engine’s object store over the same bucket and key-encryption key, so it reads each object, checks its SHA-256 and opens it:

  - name: video-analysis
    direction: outbound
    transport: llm
    bind: "llm://video-analysis"
    ack-mode: on-complete
    llm:
      media-store: recordings          # sutra.object-stores.recordings: the recording bucket
      require: [video-understanding]
      candidates:
        - provider: gemini
          model: gemini-2.5-flash
          api-key: env:GEMINI_API_KEY

messages: [{role: "user", content: "Analyse participant " + segment.participant + " …",
            audio: {ref: segment.audio.uri, sha256: segment.audio.sha256, format: segment.audio.format},
            video: {frames: for f in segment.frames return {ref: f.uri, sha256: f.sha256, format: f.format, atMs: f.atMs}}}]

The engine’s object store is declared in the engine configuration, for example SUTRA_OBJECT_STORES=recordings, SUTRA_OBJECT_STORES_RECORDINGS_URI=s3://meeting-recordings/sessions (the same location as SUTRA_VOIP_RECORDING_STORE), …_ENDPOINT, …_ACCESS_KEY, …_SECRET_KEY, and …_KEK (the same key as SUTRA_VOIP_RECORDING_KEK). The engine book’s “Recordings and media references” covers the store keys and the refusals.

examples/video-meeting-assistant does all of it in segment-analysis.bpmn with rules/analysis-policy.dmn: see Emotion analytics and compliance.

call-analysis and the analyze.* keys of earlier versions are refused at load, with a pointer here.

The call-hangup of a session carries participants (role, segmented, segments delivered, talk time, transcript lines, frame counts, the recording reference, the transcript summary), recordings (the media references <q:audit capture="media"> audits), segments (segmentMs, maxFrames, roles, delivered, failed, skipped while waiting for consent), video (framesRetained: false), flaggedMoments (a count) and consentAnnounced.

Memory

What the transport holds per participant does not grow with the meeting: its audio only for a window (the longer of 120 s and a segment plus a few seconds), its frames, transcript lines, voiced spans and flagged moments only until the segment they fall in has been written. A 30-minute, two-person session with 1 fps frames peaks at about 4.6 MB per participant (16 kHz), the same as in its first five minutes. Recordings stream to the store in parts and hold at most a part.

Compliance

• Roles are declared, never inferred. Whoever issues the room token sets the role attribute (SUTRA_WORKER_ROLE_ATTRIBUTE, default role).
• The transport’s gate: segment.roles limits who is cut into segments at all; anyone else’s media is never written as a segment, and their frames are not kept.
• The flow’s policy: who is analysed, and why not, is a decision in the flow (the example’s decision table: external parties; staff only with the deployment’s switch AND an opt-in, since inferring emotions of people at work is prohibited in the EU, AI Act, Art. 5(1)(f)).
• No identification, no sensitive attributes: the flow’s prompt names pseudonyms (P1, P2) and the flow keeps only the fields it asked for. See Emotion analytics and compliance.
• Media are referenced, sealed and erasable. Recordings are audio only; segment audio and frames are sealed objects named by their SHA-256 in the flow’s variables and audit events, so subject erasure and retention delete them.
• Disclosure is part of the flow: the example’s bot announces recording and analysis in the room, and its display name says so; record: after-consent keeps everything off until then.

Running the worker

Build and check it (the host needs only Docker — the builder installs clang 21):

make worker-image    # sutra-webrtc-worker:dev, both binaries
make worker-lint     # fmt + clippy -D warnings in Docker, cargo deny + audit
make worker-test     # unit tests in Docker
make worker-scan     # Trivy over the image

The same image carries sutra-webrtc-participant, a scripted participant (synthetic microphone on a talk script, synthetic camera) for tests and demos — not a client.

Not covered yet

• Video recording (LiveKit Egress to the object store): the recordings are audio only; only the sampled frames of segmented participants are written, as segment media.
• Phone participants through LiveKit SIP.
• Several rooms per worker (run one worker per room).
• Live (real-provider) runs are manual: the tier-2 test uses the stub model.

Recording and consent

When a call is recorded

A call to be recorded with no store configured is refused (SUTRA.VOIP.RECORDING_UNAVAILABLE).

The consent rule

Consent is given on a call in one of two ways:

• An announcement played to the end. A play marked consent: true that is not interrupted (barge-in) records the moment it ended. The recording reference then says consentAnnounced: true and consentAnnouncedAtMs.
• The flow vouching for it. voip://record-start with consent: true, once the caller agreed some other way (pressed a key, say).

With after-consent the recording starts at that moment and not before: the audio before the announcement, and the announcement itself, are not in it, and consentAnnouncedAtMs (where the announcement started, relative to the recording’s start) is negative. A call that ends before consent has no recording: recording: {status: "not-started", detail: "consent was never given, so nothing was recorded"}.

record: true records from the moment the media connects and leaves consent to the deployment (an announcement the PBX plays before the call reaches the engine, say).

Record-start and record-stop

A call can be recorded in several stretches: the second recording of call c1 is c1-2.wav, and the call-hangup carries every one in recordings (the last in recording). On a meeting session both operations act on every present participant.

The reference

Neither the flow nor the audit trail ever holds audio. The call-hangup (and record-stop) carries a media reference:

A store that cannot be written answers recording: {status: "failed", detail}.

How a recording is written

The recording streams to SUTRA_VOIP_RECORDING_STORE while the call is up. Memory does not grow with the call:

1. The audio (the far end, silence-filled where the gateway sent nothing so the recording keeps the call’s clock, with the transport’s own prompts mixed in where they played) is cut into one-second chunks.
2. Each chunk is sealed and appended to a spool file (SUTRA_VOIP_RECORDING_SPOOL_DIR).
3. Every time the spool holds 5 MiB (the smallest part an S3 store accepts) it is uploaded as the next part of a multipart upload, and the spool is emptied.
4. At the hang-up the rest is uploaded, the upload completed, and the reference delivered.

The engine’s object store client (sutra-objectstore) does the uploads, so any S3-compatible store works.

The sealed-stream format

"SUTRA-SEALED-STREAM-1\n"     magic, 22 bytes
u32 big-endian                length of the header
header                        JSON: {scheme, keyId, wrappedKey (base64), contentType, cipher}
frame*                        u32 big-endian length || nonce (12) || AES-256-GCM(chunk) || tag (16)

One fresh data key per recording, wrapped under SUTRA_VOIP_RECORDING_KEK (the engine’s envelope encryption). Each frame’s associated data binds it to the key id, the object key, its index and whether it is the last one, so frames cannot be reordered, dropped, moved to another object or passed off as the end. Frame 0 is the WAV header and the last frame is marked final, so a truncated object verifies up to its last good frame and reads as incomplete. sutra_transport_voip::recording::open_stream opens one.

The WAV header is written before the length is known, so it carries the streaming sizes 0xFFFFFFFF in its RIFF and data fields. open_stream answers the file with its exact sizes; a plain (encryption: none) recording keeps the streaming sizes, which most players accept.

The manifest and the markers

Each recording has a manifest, <key>.json next to it in the store (and in the spool directory): the call it belongs to (callId, tenant, moduleKey, a meeting’s participantId), the audit routing of the instance that owns the call, who is writing it (owner, boot), its upload id, the next part number, and its status: recording, complete (with sha256, bytes), failed, recovered or empty. While the upload is in progress a copy of the manifest is also kept under sutra-voip-uploads/ (a marker), because some S3-compatible stores list multipart uploads only by exact key; the marker is deleted when the recording completes.

Crash recovery

A sweep runs at start-up and every SUTRA_VOIP_RECORDING_SWEEP_SECONDS (600). It finds the recordings no live call is writing:

• this host’s spools left by an earlier run of the process (SUTRA_VOIP_INSTANCE_ID, default the host name, says who “this host” is), and
• markers in the store of uploads nobody has touched for longer than the longest recording plus ten minutes (another host’s, or one whose spool is gone).

For each, it uploads what the spool still holds, completes the upload, verifies the stored object frame by frame, and amends the manifest (status: "recovered", complete: false unless the final frame made it, the captured durationMs, sha256). An upload with no audio is aborted (status: "empty"). Unless the call’s hang-up already went out, the reference is then delivered on the call’s call-hangup channel with reason: "recording-recovered" and recovered: true. Both steps are audit events in the owning instance’s journal (voip.RECORDING_RECOVERED, voip.RECORDING_RECOVERED_DELIVERED).

Walkthrough: a dial-in recorded after consent

examples/meeting-assistant dials a conference, enters the PIN, plays the consent announcement and listens. Its originate asks record: "after-consent", from a process that audits at capture="media". This walkthrough shows the recording starting after the announcement, the sealed stream and its manifest in the bucket, and the audit trail carrying only the reference.

Prerequisites. Docker with Compose v2, curl and python3, and a checkout of this repository with its submodule. Until 0.6.0 is published, build the engine image and the sutra CLI from the checkout (the CLI comes from the engine submodule; the first build takes several minutes):

git submodule update --init
make image
cargo build --manifest-path engine/rust/Cargo.toml --profile release-it -p sutra-cli --target-dir rust/target
export PATH="$PWD/rust/target/release-it:$PATH"
export SUTRA_ENGINE_IMAGE=sutra-voice-engine:dev

1. Package the example and start the stack (engine, PostgreSQL, MinIO, rustpbx, SIPp as the conference, the stub model):

C=examples/meeting-assistant/deploy/compose.yaml
sutra package examples/meeting-assistant/variants/meeting-stub \
    --overlay examples/meeting-assistant/shared \
    --out examples/meeting-assistant/deploy/deployments | grep '^packaged'
docker compose -f $C --progress quiet up -d --build
until PORT=$(docker compose -f $C port engine 8080 2>/dev/null | cut -d: -f2) &&
      curl -sf "http://127.0.0.1:$PORT/sutra/deployments" | grep -q '"phase":"Active"'; do sleep 1; done
echo "the engine is on port $PORT"

packaged examples/meeting-assistant/deploy/deployments/meeting-stub.sutra (deploymentId dep-8ef8bcdca12f51cc8c078bf1)
the engine is on port 33003

(sutra package also prints three warnings about the schema-less JSON channels; the grep leaves them out. The engine’s port is dynamic.)

2. Ask the bot to join the meeting:

curl -s -w "HTTP %{http_code}\n" -X POST "http://127.0.0.1:$PORT/channels/join" \
  -H 'X-Api-Key: dev-only-meeting-key' -H 'Content-Type: application/json' \
  -d '{"meetingId":"M-1","title":"Launch plan review","dialIn":"+15550142000","pin":"4321",
       "participants":[{"name":"Alex Chen","role":"internal"},{"name":"Sam Rivera","role":"external"}]}'

HTTP 202

The conference answers, the bot enters the PIN and plays the 1.6 s announcement, and the recording starts when the announcement ends. The stand-in conference then plays about 11 s of conversation and hangs up.

3. Read the recording’s reference from the stored report, once the flow has written it (about 25 s after the join):

db() { docker compose -f $C exec -T engine-db psql -U postgres -d sutra -tAc "$1"; }
until [ -n "$(db "select 1 from data_store where store_key = 'report:M-1'" 2>/dev/null)" ]; do sleep 2; done
db "select store_value from data_store where store_key = 'report:M-1'" | python3 -c '
import json, sys
r = json.load(sys.stdin)["recording"]
for k in ["kind", "uri", "format", "durationMs", "startMs", "consentAnnounced", "consentAnnouncedAtMs", "complete", "encryption", "sha256"]:
    print(f"{k}: {json.dumps(r[k])}")'

kind: "media"
uri: "s3://meeting-recordings/calls/default/meeting-assistant/1.0.0/meeting-M-1.wav.sealed"
format: "audio/wav"
durationMs: 23006
startMs: 2612
consentAnnounced: true
consentAnnouncedAtMs: -1582
complete: true
encryption: {"cipher": "aes-256-gcm", "keyId": "recordings", "scheme": "sutra-sealed-stream-1"}
sha256: "ac3d2789346ac5737beec078a2648d969cf7d523998167e9cdb4719d4192f199"

startMs is where in the call the recording began, 2.6 s after the media connected: after the PIN and the announcement. consentAnnouncedAtMs is where the announcement started, relative to the recording’s own start: it began 1.6 s before the recording, and the recording started when it ended. (The data_store table is created on the first write, so the loop’s first probes fail quietly.)

4. The objects in the bucket. The MinIO image carries the mc client; point it at the server, then list the recording, its manifest, and the (now empty) markers prefix:

docker compose -f $C exec -T recordings mc alias set local http://localhost:9000 dev-only-access dev-only-secret-key >/dev/null
docker compose -f $C exec -T recordings mc ls -r local/meeting-recordings

[2026-10-26 07:29:48 UTC] 360KiB STANDARD calls/default/meeting-assistant/1.0.0/meeting-M-1.wav.sealed
[2026-10-26 07:29:48 UTC]   983B STANDARD calls/default/meeting-assistant/1.0.0/meeting-M-1.wav.sealed.json

The recording and its manifest. No marker is left under sutra-voip-uploads/: the upload completed.

5. The manifest and the sealed stream:

KEY=calls/default/meeting-assistant/1.0.0/meeting-M-1.wav.sealed
docker compose -f $C exec -T recordings mc cat local/meeting-recordings/$KEY.json | python3 -c '
import json, sys
m = json.load(sys.stdin)
for k in ["scheme", "status", "complete", "callId", "moduleKey", "sealed", "keyId", "nextPart", "hangupDelivered", "audit", "sha256", "bytes"]:
    print(f"{k}: {json.dumps(m[k])}")'
docker compose -f $C exec -T recordings mc cat local/meeting-recordings/$KEY | head -c 22; echo
docker compose -f $C exec -T recordings mc cat local/meeting-recordings/$KEY | sha256sum

scheme: "sutra-sealed-stream-1"
status: "complete"
complete: true
callId: "meeting-M-1"
moduleKey: "default/meeting-assistant/1.0.0"
sealed: true
keyId: "recordings"
nextPart: 2
hangupDelivered: true
audit: {"sutra-audit-journal": "capture=media;sink=sql", "sutra-audit-deployment": "dep-8ef8bcdca12f51cc8c078bf1", "sutra-audit-instance": "e23d9e93-f1d2-4431-ab01-ac20b28e2b5c"}
sha256: "ac3d2789346ac5737beec078a2648d969cf7d523998167e9cdb4719d4192f199"
bytes: 369198
SUTRA-SEALED-STREAM-1

ac3d2789346ac5737beec078a2648d969cf7d523998167e9cdb4719d4192f199  -

A 360 KiB recording fits in one part (nextPart: 2). audit is the routing of the instance that owns the call, kept so a recovery would write its audit events to that instance’s journal. The object starts with the sealed-stream magic, and its SHA-256 is the reference’s: the integrity of the recording can be checked without the key.

6. The audit trail carries the reference, never the audio:

db "select event_type, node_id, payload_json from audit_event where payload_json like '%\"media\"%' order by seq desc limit 1"

NODE_ENTERED|End|{"media":[{"bytes":369198,"callId":"meeting-M-1","channels":1,"complete":true,"consentAnnounced":true,"consentAnnouncedAtMs":-1582,"durationMs":23006,"encryption":{"cipher":"aes-256-gcm","keyId":"recordings","scheme":"sutra-sealed-stream-1"},"format":"audio/wav","kind":"media","rate":8000,"sha256":"ac3d2789346ac5737beec078a2648d969cf7d523998167e9cdb4719d4192f199","startMs":2612,"truncated":false,"uri":"s3://meeting-recordings/calls/default/meeting-assistant/1.0.0/meeting-M-1.wav.sealed"}]}

7. Clean up:

docker compose -p sutra-meeting-assistant down -v

Consent in practice

Recording and analysis rules differ by jurisdiction: some require every party’s consent, some an announcement, some a way to opt out. The software cannot decide which applies; the flow carries the announcement, and the platform makes it provable.

• Say it before anything is recorded. Play the announcement as the first step, with consent: true on the play, and record after-consent. The recording reference then states consentAnnounced and consentAnnouncedAtMs.
• Meeting bots are visible. The WebRTC worker joins under a display name that says it records (default Meeting assistant (recording)), and the example flow announces recording and analysis in the room before anything is recorded or cut into segments (record: after-consent).
• Personal data. Mark stores holding interaction records as personal, audit metadata and not payloads, treat the caller’s turns as sensitive variables so they are redacted wherever they would be logged, and mark the caller number as a subject key so erasure requests find it. Recordings and segments are referenced from the audit trail, so the engine’s subject erasure and retention delete them (see the engine book’s “Recordings and media references”).
• A menu that does not record says so by not recording. The ivr-dtmf example neither records nor analyses; if you add either, update the welcome prompt first.

The full key list is in Configuration. This page is engineering guidance, not legal advice.

Emotion analytics and compliance

Sutra Voice can analyse emotion, sentiment, engagement and expression, through the engine’s llm channels, for calls (from the caller’s audio) and for video meetings (from each participant’s audio and sampled camera frames). Because this is the most regulated thing the project does, the defaults are restrictive and the restrictions are enforced by the software where possible.

Where it is decided

The voip transport analyses nothing. For a call it delivers the caller’s turns and generic acoustic features; for a meeting it writes each participant’s media segment by segment and hands the flow references. The policy, the prompt, the model call and the reading of the answer are all in the example flows, as BPMN, DMN and FEEL that ship with the package and are audited like any other step.

Who is analysed

• Customers and external participants by default. The call examples analyse the customer’s voice (analyzedParties: ["customer"]). An agent’s leg never reaches the engine: a transfer hands the call to the PBX.
• Staff only by explicit opt-in. Inferring the emotions of people at work is prohibited in the EU (AI Act, Article 5(1)(f)), which includes contact-centre agents and colleagues in an internal meeting. The meeting examples decide with a decision table, rules/analysis-policy.dmn: an external party is analysed; a staff role (staff, internal, employee, agent, host) only when the deployment’s switch (allowStaff in the video example, internalAnalysisAllowed in the dial-in one, both false as shipped) is on AND the people concerned opted in; anyone else, including a participant with no declared role, is not. Turn the switch on only where the prohibition does not apply, and document why it is lawful there.
• Two gates in a video meeting. The call-segment channel’s segment.roles limits who is cut into segments at all ([external] in the example), so a staff participant’s media is never even written as a segment; the flow’s table then decides again for every segment.
• Roles are declared, never inferred. Whoever issues the room token sets the role attribute.
• A participant who is not analysed never reaches the model. No media of theirs is sent; the analysis row says not-analysed and why.

What the model is asked, and what is kept

• No identification and no sensitive attributes. The flow’s prompt names pseudonyms (P1, P2) and tells the model not to identify anyone or infer age, gender, ethnicity, health, religion, political or sexual orientation. The flow then keeps only the fields it asked for: anything else in the answer is dropped and listed in removedKeys.
• Only what was sent counts. A signal (voice, face, body, words) the model claims for an emotion but that was not sent is false, not guessed (unsupportedClaims); an emotion no sent signal supports is withheld.
• Disagreement is reported, never averaged. When the signals disagree (a calm voice, a tense posture) the row keeps the model’s disagreement, and the report counts the segments it happened in.
• Media are referenced, sealed and erasable. Recordings are audio only. A segment’s audio and frames are sealed objects in the recording store, named by SHA-256 in the flow’s variables and audit events (capture="media"), so subject erasure and retention delete them.

Advisory, not a decision

The analysis is data for people to review and one input among others. It must never be wired to an automated decision about a caller or a participant, such as refusing service, scoring performance or routing an employee’s case. Keep a person between the analysis and any outcome.

Disclosure

Tell people before they are analysed: in the greeting of a call, and in the room and the bot’s display name for a meeting. See Recording and consent.

This page is engineering guidance, not legal advice. Check the rules that apply where your callers and participants are.

Moving to a real SIP trunk

The examples run against SIPp, which stands in for a carrier. Moving to a real trunk is configuration of the PBX and the package; a real trunk stays a manual check, because the automated suites do not reach a carrier. Start with the voice-assistant example’s checklist and adapt it to your carrier.

Steps

1. The trunk. In the PBX configuration (deploy/rustpbx/config.toml, section [proxy.trunks.carrier]) set dest to the carrier’s SIP proxy, username and password to your credentials, and did_numbers to your number. Keep the codec list (pcmu, pcma): rustpbx builds an originated call’s offer from it and refuses the call without.
2. Inbound recognition. rustpbx recognises a trunk by the source address of the carrier’s signalling. Set inbound_hosts to the carrier’s signalling addresses and nothing else.
3. The number. Change the route’s "to.user" match and the call-start channel’s bind: "number:..." in the package.
4. The network. Set external_ip to the address the carrier sends media to, publish the SIP port and exactly the RTP range (rtp_start_port to rtp_end_port, kept small) over UDP, and restrict acl_rules.
5. Secrets. Replace the development control and media tokens, use wss:// for the control socket over an untrusted network (SUTRA_VOIP_RWI_CA_FILE for a private CA), and put the engine behind TLS for the media WebSocket. Secrets are references (env:, file:), never literals.
6. The trunk size. Set the capacity pool’s slots to the carrier’s concurrent-call limit.

Registration and authentication

Some carriers authenticate by source IP, others require the PBX to register with a username and password (SIP digest). Use the mechanism the carrier documents: IP authentication needs the correct inbound_hosts and a stable public address; registration needs the credentials in the trunk section and keep-alives through any NAT. Carrier credentials belong in secret references, not in a committed file. Extension phones that register with your PBX need their own strong passwords: the examples ship development-only ones.

Toll fraud

A PBX exposed to the internet is a target for toll fraud: attackers register or call through it to premium-rate and international numbers at your expense.

• Keep the SIP port reachable only from the carrier where possible, and apply acl_rules.
• Never leave default or example passwords on extensions or trunks.
• Restrict outbound destinations (country and premium-rate prefixes) at the PBX and at the carrier. Flows that dial numbers supplied by users should validate them first.
• The engine refuses too: voip://originate dials nothing until SUTRA_VOIP_ORIGINATE_ALLOW lists what it may (glob patterns such as +1555*, or sip:*@pbx.internal). Premium-rate prefixes (SUTRA_VOIP_ORIGINATE_BLOCK_PREFIXES, a built-in list) and international numbers (outside SUTRA_VOIP_ORIGINATE_HOME_PREFIX) are refused unless an allow pattern names them specifically or _ALLOW_PREMIUM / _ALLOW_INTERNATIONAL is true; each destination may be dialled _RATE_PER_DESTINATION times an hour (10) and all of them _RATE_GLOBAL times a minute (60). Every refusal is a voip.ORIGINATE_REFUSED or voip.ORIGINATE_RATE_LIMITED event in the requesting instance’s audit trail.
• Set a spending limit and call-rate alerts with the carrier, and use the capacity pool to cap concurrent calls.
• Watch call records for unusual destinations and off-hours volume.

See also Security notes.

ivr-dtmf

A classic phone menu with no model: a recorded welcome and menu, a DMN decision table that turns a key into an extension and an access rule, customer lookup, PIN entry with retries, transfer to PBX extensions, and a log of every outcome (never the digits). The whole call is one instance of one BPMN process whose menu and PIN loops are real loops.

Its walkthrough is the Quick start, run against the real stack. It runs with rustpbx and SIPp, and includes a table of calls that exercise every path: an open extension, a customer, a valid, an inactive and an expired PIN, three wrong PINs, an invalid choice, and no choice at all.

The PINs in the example’s store are kept in clear text, deliberately, so every path can be tried; the README explains the production alternative (a verification service holding salted hashes).

Full README, data and commands: examples/ivr-dtmf.

voice-assistant

A clinic front desk on the phone. A call to the example number is greeted, and each utterance goes as audio to an audio-native model whose spoken answer is played back, with barge-in and key fallbacks (0 for an agent, # to end). The call is recorded once the greeting, which says so, has played to the end (record: after-consent). After the call the customer’s voice is analysed for emotion and sentiment alongside the transport’s acoustic metrics, a decision table works out first-contact resolution and repeat contact, and the interaction is stored. A second flow places appointment-reminder calls through the same capacity pool, with a retry loop.

It runs with the stub model server: no key and no phone line. A set of *-live variants swaps in real models. The README also has the checklist for a real trunk and a consent section that applies to anything that records or analyses callers.

Walkthrough: a call, a repeat call, and a reminder

Prerequisites. Docker with Compose v2, curl and python3, and a checkout of this repository with its submodule. Until 0.6.0 is published, build the engine image and the sutra CLI from the checkout (the CLI comes from the engine submodule; the first build takes several minutes):

git submodule update --init
make image
cargo build --manifest-path engine/rust/Cargo.toml --profile release-it -p sutra-cli --target-dir rust/target
export PATH="$PWD/rust/target/release-it:$PATH"
export SUTRA_ENGINE_IMAGE=sutra-voice-engine:dev

1. Package both modules (the inbound assistant and the reminders) and start the stack (engine, PostgreSQL, rustpbx, SIPp as the carrier, the stub model):

C=examples/voice-assistant/deploy/compose.yaml
sutra package examples/voice-assistant/inbound/variants/inbound-stub \
    --overlay examples/voice-assistant/inbound/shared --out examples/voice-assistant/deploy/deployments | grep '^packaged'
sutra package examples/voice-assistant/reminders/variants/reminders-stub \
    --overlay examples/voice-assistant/reminders/shared --out examples/voice-assistant/deploy/deployments | grep '^packaged'
docker compose -f $C --progress quiet up -d --build
until PORT=$(docker compose -f $C port engine 8080 2>/dev/null | cut -d: -f2) &&
      [ "$(curl -sf "http://127.0.0.1:$PORT/sutra/deployments" | grep -o '"phase":"Active"' | wc -l)" = 2 ]; do sleep 1; done
echo "the engine is on port $PORT"

packaged examples/voice-assistant/deploy/deployments/inbound-stub.sutra (deploymentId dep-7395ba4564bbcbdc4c76649b)
packaged examples/voice-assistant/deploy/deployments/reminders-stub.sutra (deploymentId dep-9faf75d6967125d67f28a186)
the engine is on port 33011

2. Call the assistant. SIPp dials the number from +15557654321, waits for the greeting, says one thing (a synthesised 1.5 s utterance), listens to the answer and hangs up:

docker compose -f $C --progress quiet run --rm caller >/dev/null 2>&1; echo "the caller exited with $?"
db() { docker compose -f $C exec -T engine-db psql -U postgres -d sutra -tAc "$1"; }
until [ "$(db "select count(*) from data_store where store_name = 'interactions' and store_key like 'call:%'" 2>/dev/null)" = 1 ]; do sleep 1; done
db "select store_value from data_store where store_name = 'interactions' and store_key like 'call:%'" | python3 -c '
import json, sys
r = json.load(sys.stdin)
print("outcome:", r["outcome"], "firstContactResolution:", r["firstContactResolution"], "repeatContact:", r["repeatContact"])
print("analysis:", r["analysis"]["customer"]["overall"], [t["emotion"] for t in r["analysis"]["customer"]["trajectory"]], "audioPreserved:", r["audioPreserved"])
print("acoustic:", {k: r["acoustic"][k] for k in ["callerTalkMs", "agentTalkMs", "turns", "interruptions"]})
rec = r["recording"]
print("recording:", rec["kind"], rec["format"], "startMs=%s consentAnnouncedAtMs=%s complete=%s" % (rec["startMs"], rec["consentAnnouncedAtMs"], rec["complete"]))'

the caller exited with 0
outcome: first-contact-resolution firstContactResolution: True repeatContact: False
analysis: neutral ['concerned', 'calm'] audioPreserved: True
acoustic: {'callerTalkMs': 1380, 'agentTalkMs': 780, 'turns': 1, 'interruptions': 0}
recording: media audio/wav startMs=805 consentAnnouncedAtMs=-805 complete=True

The model heard the voice itself (audioPreserved: true). The recording started when the greeting ended (startMs), and the greeting itself began before it (consentAnnouncedAtMs is negative): nothing was recorded before the caller had heard that the call is recorded.

3. Call again from the same number. The decision table recognises a repeat contact, and the first call no longer counts as resolved at first contact:

docker compose -f $C --progress quiet run --rm caller >/dev/null 2>&1; echo "the caller exited with $?"
until [ "$(db "select count(*) from data_store where store_name = 'interactions' and store_key like 'call:%'")" = 2 ]; do sleep 1; done
sleep 2
db "select store_value from data_store where store_name = 'interactions' and store_key like 'call:%'" | python3 -c '
import json, sys
for line in sys.stdin:
    r = json.loads(line)
    print(r["startedAt"][:19], r["outcome"], "firstContactResolution=%s repeatContact=%s" % (r["firstContactResolution"], r["repeatContact"]))' | sort
db "select store_value from data_store where store_key = 'caller:+15557654321'"

the caller exited with 0
2026-10-26T07:34:21 first-contact-resolution firstContactResolution=False repeatContact=False
2026-10-26T07:34:34 resolved-on-repeat firstContactResolution=False repeatContact=True
{"contacts":2,"lastAt":"2026-10-26T07:34:46.835616706Z","lastCallId":"818dc3f32fc740d7a3773b9db28efcd9","lastResolved":true}

4. Send a reminder. The engine dials the carrier (through the same pool of trunk lines), which answers and says one thing; the outcome lands in the reminders store:

curl -s -w "HTTP %{http_code}\n" -X POST "http://127.0.0.1:$PORT/channels/reminder" \
  -H 'X-Api-Key: dev-only-voice-key' -H 'Content-Type: application/json' \
  -d '{"appointmentId":"A-1","name":"Sam","phone":"+15550001111","when":"Tuesday at 10:00"}'
until [ -n "$(db "select 1 from data_store where store_name = 'reminders' and store_key = 'A-1'")" ]; do sleep 1; done
db "select store_value from data_store where store_name = 'reminders' and store_key = 'A-1'" | python3 -c '
import json, sys
r = json.load(sys.stdin)
print(r["outcome"], "attempts=%s" % r["attempts"], r["patientReply"])'

HTTP 202
answered attempts=1 STUB VOICE ANSWER: your appointment is confirmed for Tuesday at ten.

5. Clean up:

docker compose -p sutra-voice-assistant down -v

Full README, data and commands: examples/voice-assistant.

meeting-assistant

A bot that dials into a conference by its SIP dial-in number, enters the meeting PIN, announces that the call is recorded and transcribed, and then only listens. The transport records the meeting from the end of the announcement on (sealed, referenced by hash in the audit trail) and transcribes it live; the flow receives only sparse events. Afterwards it asks a model for minutes, decisions and action items, applies the analysis policy (a decision table) and renders a report.

Because a dial-in is one mixed channel, speakers are not labelled by the transcript provider, and the report says attribution is inferred. See Mixed and per-participant media.

Walkthrough: join, transcribe, report, and a refused dial

Prerequisites. Docker with Compose v2, curl and python3, and a checkout of this repository with its submodule. Until 0.6.0 is published, build the engine image and the sutra CLI from the checkout (the CLI comes from the engine submodule; the first build takes several minutes):

git submodule update --init
make image
cargo build --manifest-path engine/rust/Cargo.toml --profile release-it -p sutra-cli --target-dir rust/target
export PATH="$PWD/rust/target/release-it:$PATH"
export SUTRA_ENGINE_IMAGE=sutra-voice-engine:dev

1. Package the example and start the stack:

C=examples/meeting-assistant/deploy/compose.yaml
sutra package examples/meeting-assistant/variants/meeting-stub \
    --overlay examples/meeting-assistant/shared \
    --out examples/meeting-assistant/deploy/deployments | grep '^packaged'
docker compose -f $C --progress quiet up -d --build
until PORT=$(docker compose -f $C port engine 8080 2>/dev/null | cut -d: -f2) &&
      curl -sf "http://127.0.0.1:$PORT/sutra/deployments" | grep -q '"phase":"Active"'; do sleep 1; done
echo "the engine is on port $PORT"

packaged examples/meeting-assistant/deploy/deployments/meeting-stub.sutra (deploymentId dep-8ef8bcdca12f51cc8c078bf1)
the engine is on port 33004

2. Ask the bot to join. The request asks for the internal participant to be analysed too (internalOptIn), which this deployment does not allow:

curl -s -w "HTTP %{http_code}\n" -X POST "http://127.0.0.1:$PORT/channels/join" \
  -H 'X-Api-Key: dev-only-meeting-key' -H 'Content-Type: application/json' \
  -d '{"meetingId":"M-1","title":"Launch plan review","dialIn":"+15550142000","pin":"4321",
       "participants":[{"name":"Alex Chen","role":"internal"},{"name":"Sam Rivera","role":"external"}],
       "analysis":{"internalOptIn":true}}'

HTTP 202

The bot dials the conference through rustpbx, sends 4321# as telephone events, plays the announcement, and listens. The stand-in conference (SIPp) plays about 11 s of a two-voice conversation and hangs up.

3. The transcript, one row per final segment, stored while the call was up (no speaker labels: a dial-in is one mixed channel):

db() { docker compose -f $C exec -T engine-db psql -U postgres -d sutra -tAc "$1"; }
until [ -n "$(db "select 1 from data_store where store_key = 'report:M-1'" 2>/dev/null)" ]; do sleep 2; done
db "select seq, speaker, start_ms, end_ms, text from transcripts where call_id = 'meeting-M-1' order by seq"

1|unknown|9000|10960|Good morning, everyone. Let's go through the launch plan for the new booking page.
2|unknown|11900|13660|Thanks. The design is ready; the open question is the release date.
3|unknown|14600|16960|Then we agree to ship on the fifteenth. Action item: Priya books the accessibility review.
4|unknown|17900|19480|Agreed. I will send the summary to the client by Friday.

4. The report: the model’s minutes, and the analysis policy applied to what it said about each speaker:

db "select store_value from data_store where store_key = 'report:M-1'" | python3 -c '
import json, sys
r = json.load(sys.stdin)
print("summary:", r["summary"])
print("decisions:", r["decisions"])
print("analyzedRoles:", r["analyzedRoles"])
for s in r["speakers"]:
    print("speaker:", s["name"], s["role"], "analyzed=%s sentiment=%s emotion=%s" % (s["analyzed"], s["sentiment"], s["emotion"]))
print("keywords:", [k["phrase"] for k in r["keywords"]])
print("recording:", r["recording"]["uri"], r["recording"]["sha256"])'

summary: The team reviewed the launch plan for the new booking page and set the release date.
decisions: ['Ship the new booking page on the fifteenth.']
analyzedRoles: ['external']
speaker: Alex Chen internal analyzed=False sentiment=None emotion=None
speaker: Sam Rivera external analyzed=True sentiment=positive emotion=engaged
keywords: ['action item']
recording: s3://meeting-recordings/calls/default/meeting-assistant/1.0.0/meeting-M-1.wav.sealed 79bfa05fb576c7efba5ccee37d935ba1e429387d0d029449e066dabed4484943

The decision table (rules/analysis-policy.dmn) said no for the internal role, because the deployment’s switch is off whatever the request asks, so the flow dropped the sentiment and emotion the model returned for Alex.

5. A dial the toll-fraud controls refuse. The engine dials only what SUTRA_VOIP_ORIGINATE_ALLOW lists (+1555* here). A meeting whose dial-in is elsewhere is refused before anything is dialled, and the refusal is an event in that instance’s own audit trail:

curl -s -w "HTTP %{http_code}\n" -X POST "http://127.0.0.1:$PORT/channels/join" \
  -H 'X-Api-Key: dev-only-meeting-key' -H 'Content-Type: application/json' \
  -d '{"meetingId":"M-2","title":"Off the list","dialIn":"+447700900123","pin":"1111",
       "participants":[{"name":"Kim Park","role":"external"}]}'
until [ -n "$(db "select 1 from audit_event where event_type = 'voip.ORIGINATE_REFUSED'")" ]; do sleep 1; done
db "select event_type, diagnostic_code, payload_json from audit_event where event_type like 'voip.%'"
ID=$(db "select instance_id from audit_event where event_type = 'voip.ORIGINATE_REFUSED'")
db "select seq, event_type, coalesce(node_id, '') from audit_event where instance_id = '$ID' order by seq, id"

HTTP 202
voip.ORIGINATE_REFUSED|SUTRA.VOIP.ORIGINATE_REFUSED|{}
1|INSTANCE_STARTED|
2|NODE_ENTERED|Start
3|NODE_LEFT|Start
4|NODE_ENTERED|Init
5|NODE_LEFT|Init
6|NODE_ENTERED|PolicyInputs
7|NODE_LEFT|PolicyInputs
8|NODE_ENTERED|Policy
9|NODE_LEFT|Policy
10|NODE_ENTERED|AnalysedRoles
11|NODE_LEFT|AnalysedRoles
12|NODE_ENTERED|PrepDial
13|NODE_LEFT|PrepDial
14|NODE_ENTERED|Dial
15|NODE_LEFT|Dial
16|NODE_ENTERED|Outcome
17|INSTANCE_SUSPENDED|
17|voip.ORIGINATE_REFUSED|

The event sits in the refused meeting’s own journal, numbered with the instance’s latest event (a transport event takes the sequence number of the event it follows, so it never takes one the engine writes next). Its payload is empty: the process audits at capture="media", so no field (and never the destination) is written.

6. Clean up:

docker compose -p sutra-meeting-assistant down -v

The recording and consent guide walks through the recording itself on the same stack: the sealed stream, its manifest, and the audited reference.

Full README, stub setup and live-provider notes: examples/meeting-assistant.

video-meeting-assistant

A bot in a LiveKit room, through the WebRTC worker. It records and transcribes each participant’s own track. The transport cuts each external participant’s media into 5 s segments and hands the flow their references; the flow decides who is analysed (a decision table), asks a video-understanding model about each segment with the media as references, keeps only what it asked for, and builds a per-participant digest and a report after the meeting.

Analysis covers external participants by default; staff are opt-in only, and in this example never even cut into segments. Read Emotion analytics and compliance before changing that. The stack runs with a stub model and two scripted participants, no keys.

Walkthrough: a meeting, analysed in the flow

Prerequisites. Docker with Compose v2 and python3, and a checkout of this repository with its submodule. Until 0.6.0 is published, build the engine image, the worker image (its first build downloads libwebrtc and takes several minutes) and the sutra CLI from the checkout:

git submodule update --init
make image worker-image
cargo build --manifest-path engine/rust/Cargo.toml --profile release-it -p sutra-cli --target-dir rust/target
export PATH="$PWD/rust/target/release-it:$PATH"
export SUTRA_ENGINE_IMAGE=sutra-voice-engine:dev

1. Package the example and start the stack. The worker joins the room standup and waits:

C=examples/video-meeting-assistant/deploy/compose.yaml
sutra package examples/video-meeting-assistant/variants/video-stub \
    --overlay examples/video-meeting-assistant/shared \
    --out examples/video-meeting-assistant/deploy/deployments | grep '^packaged'
docker compose -f $C --progress quiet up -d
until docker compose -f $C logs worker | grep -q 'joined the room'; do sleep 1; done
echo "the bot is in the room"

packaged examples/video-meeting-assistant/deploy/deployments/video-stub.sutra (deploymentId dep-5b4edf77ce8251f956940453)
the bot is in the room

2. Two participants join, scripted (a synthetic microphone on a talk script, a synthetic camera): Alex, admitted as staff, and Sam, admitted as external. Each stays 16 s:

docker compose -f $C --progress quiet --profile demo up -d alex sam
db() { docker compose -f $C exec -T engine-db psql -U postgres -d sutra -tAc "$1"; }
until [ -n "$(db "select 1 from data_store where store_key like 'session:%'" 2>/dev/null)" ]; do sleep 2; done
db "select store_name, count(*) from data_store group by 1 order by 1"

analysis|4
meetings|1
transcripts|4

The session is over about 30 s after they join. The bot played the announcement first; the session is record: after-consent, so nothing was recorded, and no segment cut, before it ended.

3. Who was analysed, and why. The roster’s decisions, from the decision table:

db "select store_value from data_store where store_key like 'session:%'" | python3 -c '
import json, sys
r = json.load(sys.stdin)
for p in r["participants"]:
    print(p["label"], p["role"], "segmented=%s analysed=%s policy=%s segments=%s emotion=%s engagement=%s disagreements=%s"
          % (p["segmented"], p["analysed"], p["analysisPolicy"], p["segments"], p["emotion"], p["engagement"], p["disagreements"]))
a = r["analysis"]
print("analysis:", a["channel"], a["provider"], a["model"], "results=%s failed=%s" % (a["results"], a["failed"]))'

Alex staff segmented=False analysed=False policy=staff-not-opted-in segments=0 emotion=None engagement=None disagreements=0
Sam external segmented=True analysed=True policy=analysed segments=4 emotion=interested engagement=high disagreements=0
analysis: video-analysis gemini gemini-video-stub results=4 failed=0

4. One analysed segment. The row keeps the media references (never the media), what the model received (the stub reports it), the fused result, and what the flow dropped:

db "select store_value from data_store where store_name = 'analysis' order by store_key limit 1" | python3 -c '
import json, sys
a = json.load(sys.stdin)
print("segment", a["segment"], a["startMs"], "-", a["endMs"], "ms:", a["status"], a["analysisPolicy"])
print("audio:", a["media"]["audio"]["uri"].split("/1.0.0/")[1], a["media"]["audio"]["sha256"][:16])
print("frames:", [f["uri"].split("/segments/")[1] for f in a["media"]["frames"]])
print("the model saw:", a["seen"])
print("emotion:", json.dumps(a["result"]["emotion"]))
print("disagreement:", json.dumps(a["result"]["disagreement"]))
print("removedKeys:", a["removedKeys"], "unsupportedClaims:", a["unsupportedClaims"])'

segment 1 0 - 5000 ms: complete analysed
audio: standup-1793000726015/segments/p2/1.wav.sealed d230f38962afbd98
frames: ['p2/1-f1.jpg.sealed', 'p2/1-f2.jpg.sealed', 'p2/1-f3.jpg.sealed', 'p2/1-f4.jpg.sealed', 'p2/1-f5.jpg.sealed', 'p2/1-f6.jpg.sealed', 'p2/1-f7.jpg.sealed', 'p2/1-f8.jpg.sealed']
the model saw: {'audio': True, 'frames': 8}
emotion: {"confidence": 0.6, "label": "interested", "modalities": {"body": true, "face": true, "voice": true, "words": true}}
disagreement: {"detail": null, "flagged": false}
removedKeys: ['audioSeen', 'framesSeen'] unsupportedClaims: []

The video-analysis channel read each referenced object from its media-store, checked its SHA-256, opened it and sent it: the stub counts the frames and the audio it received. The stub’s own bookkeeping keys (framesSeen, audioSeen) are not fields the flow asked for, so they are dropped and listed, as an inferred age or identity would be.

5. The segments in the bucket: Sam’s only (participant p2 in this run: indexes follow the order people joined); Alex’s media was never written as a segment.

docker compose -f $C exec -T recordings mc alias set local http://localhost:9000 dev-only-access dev-only-secret-key >/dev/null
docker compose -f $C exec -T recordings mc ls -r local/meeting-recordings | awk '{print $NF}' | sed 's|.*/1.0.0/[^/]*/||' | sort

p1.wav.sealed
p1.wav.sealed.json
p2.wav.sealed
p2.wav.sealed.json
segments/p2/1-f1.jpg.sealed
segments/p2/1-f2.jpg.sealed
segments/p2/1-f3.jpg.sealed
segments/p2/1-f4.jpg.sealed
segments/p2/1-f5.jpg.sealed
segments/p2/1-f6.jpg.sealed
segments/p2/1-f7.jpg.sealed
segments/p2/1-f8.jpg.sealed
segments/p2/1.wav.sealed
segments/p2/2-f1.jpg.sealed
segments/p2/2-f2.jpg.sealed
segments/p2/2-f3.jpg.sealed
segments/p2/2-f4.jpg.sealed
segments/p2/2-f5.jpg.sealed
segments/p2/2-f6.jpg.sealed
segments/p2/2.wav.sealed
segments/p2/3-f1.jpg.sealed
segments/p2/3-f2.jpg.sealed
segments/p2/3-f3.jpg.sealed
segments/p2/3-f4.jpg.sealed
segments/p2/3-f5.jpg.sealed
segments/p2/3-f6.jpg.sealed
segments/p2/3.wav.sealed
segments/p2/4-f1.jpg.sealed
segments/p2/4-f2.jpg.sealed
segments/p2/4.wav.sealed

Each participant’s recording (p1.wav.sealed, p2.wav.sealed) with its manifest, and Sam’s segments: the audio of each and its frames, sealed.

6. The report:

db "select store_value from data_store where store_key like 'session:%'" | python3 -c '
import json, sys; print(json.load(sys.stdin)["report"])' | sed -n '/## Participants/,/^## How/p'

## Participants
- P1 Alex (staff): about 5940 ms of speech, 2 transcript segments; not analysed (staff-not-opted-in).
  Recording: s3://meeting-recordings/sessions/default/video-meeting-assistant/1.0.0/standup-1793000726015/p1.wav.sealed (SHA-256 f4a7f8732dca24685bd6d90aa4f83fa0c4c3dd13a2ca4603794dcc9125835e94)
- P2 Sam (external): about 5940 ms of speech, 2 transcript segments; analysed over 4 segments — mostly interested, engagement high, 0 segments where voice, face and words disagreed.
  Recording: s3://meeting-recordings/sessions/default/video-meeting-assistant/1.0.0/standup-1793000726015/p2.wav.sealed (SHA-256 787e03527cfba86a29afbffc4727a89d3a947095c71350a4a216b1447881228c)

## How this was produced

7. Clean up:

docker compose -p sutra-video-meeting-assistant down -v

Full README, compose stack and cost notes: examples/video-meeting-assistant.

Images

Each release publishes both as ghcr.io/startr-trade/sutra-voice:<version> and ghcr.io/startr-trade/sutra-webrtc-worker:<version>: linux/amd64 and linux/arm64, signed with cosign (keyless, verifiable against this repository’s release workflow), with provenance and an SPDX SBOM attached. A release image is pinned to a released engine of the same minor version.

The :dev tags are mutable development tags. Rebuild from the commit you are testing before trusting a Docker-level result, since a stale tag silently tests an older binary. The examples read the engine image from SUTRA_ENGINE_IMAGE; set it to sutra-voice-engine:dev when working from a clone.

The engine image runs exactly like the Sutra engine’s (SUTRA_DEPLOYMENTS_DIR, SUTRA_HTTP_PORT, SUTRA_DATASOURCE_*) plus the SUTRA_VOIP_* settings. Pin third-party images such as rustpbx, which publishes only a moving latest tag, by digest in anything beyond a local trial.

Configuration

Configuration is environment variables, and every secret is a reference (env:NAME, file:/path, secret:...), never a literal. A literal key is refused.

Engine (voip transport)

The complete table with defaults is in The voip transport. In groups:

Channel-level settings (turn detection, mode, record, transcribe, segment.*, video.*, phrases, pool, session-end) are keys on the package’s channels, described in the guides.

A flow that analyses meeting segments reads them through an llm channel’s media-store, which names an engine object store (SUTRA_OBJECT_STORES, SUTRA_OBJECT_STORES_<NAME>_URI, _ENDPOINT, _ACCESS_KEY, _SECRET_KEY, _KEK, _KEY_ID) declared over the recording bucket with the recording KEK.

Engine (general)

SUTRA_DEPLOYMENTS_DIR, SUTRA_HTTP_PORT, SUTRA_DATASOURCE_* and the capacity-pool settings (SUTRA_CAPACITY_POOLS, SUTRA_CAPACITY_POOLS_<NAME>_SLOTS, …), the object stores and the audit sinks (SUTRA_AUDIT_SQL) belong to the engine. See the Sutra configuration reference.

WebRTC worker

LIVEKIT_URL, SUTRA_WORKER_ROOM, SUTRA_WORKER_API_KEY, SUTRA_WORKER_API_SECRET, SUTRA_WORKER_MEDIA_TOKEN, SUTRA_WORKER_ENGINE_URL, SUTRA_WORKER_CALLED, SUTRA_WORKER_IDENTITY, SUTRA_WORKER_NAME, SUTRA_WORKER_AUDIO_RATE, SUTRA_WORKER_ROLE_ATTRIBUTE, SUTRA_WORKER_EMPTY_GRACE_MS, SUTRA_WORKER_ONCE. Defaults are in The WebRTC worker.

Diagnostics

Misconfiguration surfaces as SUTRA.VOIP.* diagnostic codes (for example CONFIG_INVALID, CHANNEL_INVALID, BIND_DUPLICATE, MODEL_CHANNEL_INVALID, RECORDING_UNAVAILABLE); the list is at the end of the voip transport guide.

Security notes

• The media token. With no SUTRA_VOIP_MEDIA_TOKEN configured, every media request is refused. A /voip/stream client (the WebRTC worker, any media-stream gateway) sends it as Authorization: Bearer; a token in that URL is refused. The PCM bridge never sees it: each bridge URL carries a token minted for that one call, short-lived (SUTRA_VOIP_BRIDGE_TOKEN_TTL_SECONDS, 60 s) and good for one socket. Only the PBX’s webhook and IVR-step URLs carry the media token as ?token=, because a PBX’s webhook client sends no headers: keep those access logs private, rotate the token, and use TLS (wss://, https://) between gateway and engine anywhere outside a private network. Tokens are redacted from the transport’s logs. See Media credentials.
• The call-control token. SUTRA_VOIP_RWI_TOKEN gives control of calls on the PBX (originate, transfer, hang up). Use wss:// over untrusted networks; add a private CA with SUTRA_VOIP_RWI_CA_FILE.
• Secrets are references. Model keys, LiveKit keys, store credentials and the recording KEK are env: or file: references. The development tokens and extension passwords in the examples are for local use only.
• Toll fraud. An internet-facing PBX is attacked for premium-rate calling. originate is deny-all until SUTRA_VOIP_ORIGINATE_ALLOW is set, refuses premium-rate and international numbers unless allowed, rate-limits per destination and in total, and writes every refusal to the audit journal (voip.ORIGINATE_REFUSED, voip.ORIGINATE_RATE_LIMITED). See Moving to a real SIP trunk.
• Recordings and segments. Sealed with envelope encryption by default; the key-encryption key is a secret to protect and back up, since losing it makes sealed recordings unreadable. A crash leaves the part in progress spooled on disk, sealed (SUTRA_VOIP_RECORDING_SPOOL_DIR): put the spool on a volume only the engine can read.
• Personal data. Callers’ audio, numbers and transcripts are personal data. Tag stores, audit metadata and not payloads, and use sensitive variables so they are redacted.
• PINs. Do not keep PINs in the engine’s data stores in production (see the ivr-dtmf example); the dtmf operation never echoes digits.
• Supply chain. CI runs cargo audit, cargo deny and Trivy scans on source and images.

Limitations

Voice, meeting and video support is a preview in 0.6. This page says what that means.

Verified and not verified

• Automated: the suites run against SIPp (as carrier, caller and phones), a stub model server, a local rustpbx, a local LiveKit server, MinIO and PostgreSQL in Docker.
• Manual only, not verified live by the automated suites: a real SIP carrier, real model providers (OpenAI, Gemini or others), real conference bridges, and real meeting platforms. The examples’ live variants are provided and documented, but a run against them is a manual check.

Product limits

• One gateway per engine process. Only rustpbx has a call-control adapter (checked against 0.5.1). Asterisk and FreeSWITCH are planned.
• Turn-based, not realtime speech-to-speech. Media is not bridged straight between a call and a model’s live API with the flow as control plane.
• Turn detection needs continuous media. A gateway that sends nothing during silence ends a turn late.
• Video is not available through rustpbx: its media bridge is audio only. Video comes through the WebRTC worker.
• Worker: no video recording (recordings are audio only; only the sampled frames of segmented participants are written, as segment media), no phone participants through LiveKit SIP, one room per worker.
• Dial-in transcripts do not label speakers (one mixed channel).
• A recording’s length is capped by SUTRA_VOIP_RECORDING_MAX_SECONDS (it streams to the store, so memory does not grow with it). A recording interrupted by a crash is recovered by the sweep with what reached the spool or the store; the audio of the last chunk in flight is lost.
• Meeting analysis is the flow’s: the transport delivers segments as media references and generic measurements; the prompt, the policy and the fusion are in the example flows, and their quality is the model’s and the flow’s.
• Not a media server: no conferencing, mixing or transcoding beyond PCM16 and G.711 handling.
• Prompts in the IVR example are tones, not speech; replace them with recordings.
• Legal compliance is yours: the software provides disclosure and restriction hooks, not a determination that your use is lawful.

Roadmap

These are plans, not promises. Nothing here is committed to a date, and the order may change.

Hosted per-participant meetings on Asterisk and FreeSWITCH

Today per-participant audio comes through the WebRTC worker and LiveKit, while a PBX conference reaches the engine as one mixed stream. The aim is to host meetings on a PBX and receive each participant’s audio separately, with gateway adapters for both Asterisk and FreeSWITCH alongside the existing rustpbx adapter. What each PBX can expose per participant differs, and the plan depends on that.

Platform connectors

Connectors that let a flow take part in meetings on hosted platforms, through the platforms’ own supported media interfaces:

• Zoom, via real-time media streams (RTMS).
• Microsoft Teams, via media bots.
• Google Meet, via the Meet Media API.
• Post-meeting artifacts: fetching the platform’s own recordings and transcripts after a meeting for the flow to process.

Each platform has its own approval, hosting and consent requirements, and each connector would arrive as a gateway behind the same media-stream protocol: no new BPMN element.

Smaller items

• Video recording from the WebRTC worker to the object store.
• Phone participants through LiveKit SIP.
• Several rooms per worker.

Follow progress in the repository and its changelog.