An app call works like a web call: the agent hears the caller, can show cards and call the app's own functions, and the call is listed, analyzed and billed like any other. Your server keeps the API key and gives the app a one-time address or token for the signed-in user.
| App | How it connects |
|---|---|
| iOS | Audio over a WebSocket, with iOS's echo cancellation |
| Android | Audio over a WebSocket, with Android's echo cancellation |
| React Native | A web call token and Pipecat's React Native client |
For iOS and Android, your server starts the call and answers the app with its websocket_url, which can be opened once within 15 minutes:
curl https://evasay.com/api/v1/voice/calls/websocket \
-X POST \
-H "Authorization: Bearer sk-your-key" \
-H "Content-Type: application/json" \
-d '{ "agent_id": "AGENT_ID", "encoding": "pcm_s16le", "sample_rate": 16000, "user_id": "u_42", "page_tools": ["open_order"] }'
For React Native, it creates a web call token with POST /api/v1/voice/web-calls and answers the app with token and web_call_url.
Open the websocket_url with URLSessionWebSocketTask. Record with AVAudioEngine with voice processing on (the agent then does not hear itself through the speaker), convert the microphone to 16-bit PCM at the call's sample rate and send it in binary frames of 20 ms. Binary frames from EvaSay are the agent's voice in the same format; play them with an AVAudioPlayerNode.
let session = AVAudioSession.sharedInstance()
try session.setCategory(.playAndRecord, mode: .voiceChat, options: [.defaultToSpeaker])
try engine.inputNode.setVoiceProcessingEnabled(true)
let socket = URLSession.shared.webSocketTask(with: websocketURL)
socket.resume()
// microphone tap → 16 kHz Int16 mono → socket.send(.data(frame))
// socket.receive: .data → player.scheduleBuffer, .string → the call's events
Text frames are the call's events as JSON. On clear (the caller interrupted) stop the player and drop what it has queued; on ended close the socket. Add NSMicrophoneUsageDescription to the app's Info.plist.
Open the websocket_url with OkHttp. Record with AudioRecord from VOICE_COMMUNICATION, which turns on the phone's echo cancellation, and play with an AudioTrack for USAGE_VOICE_COMMUNICATION, both 16-bit mono at the call's sample rate.
val record = AudioRecord(
MediaRecorder.AudioSource.VOICE_COMMUNICATION, 16000,
AudioFormat.CHANNEL_IN_MONO, AudioFormat.ENCODING_PCM_16BIT, bufferBytes)
val socket = OkHttpClient().newWebSocket(Request.Builder().url(websocketUrl).build(), listener)
// a thread reads 640 bytes (20 ms) at a time → socket.send(frame.toByteString())
// listener.onMessage(bytes) → track.write(...); onMessage(text) → the call's events
On clear pause and flush the AudioTrack; on ended close the socket. The app needs the RECORD_AUDIO and INTERNET permissions.
POST { "token": "…", "pageTools": ["open_order"] } to https://evasay.com + web_call_url. The answer's data is the call's connection. When its transport is cf, the call goes through Cloudflare Realtime at cfUrl: connect with Pipecat's React Native SmallWebRTC transport (@pipecat-ai/react-native-small-webrtc-transport, with @pipecat-ai/client-js) whose connection setup follows EvaSay's signalling below. When the answer has an offerUrl, the unchanged transport connects to it.
// Offer to cfUrl, then answer each offer Cloudflare makes at /api/cf/answer
// until the reply names the two data channels.
await pc.setLocalDescription(await pc.createOffer());
let reply = await post(cfUrl, { sdp: pc.localDescription.sdp, type: 'offer', requestData });
const pcId = reply.pc_id;
for (;;) {
if (reply.answer) await pc.setRemoteDescription({ type: 'answer', sdp: reply.answer });
if (reply.offer) {
await pc.setRemoteDescription({ type: 'offer', sdp: reply.offer });
await pc.setLocalDescription(await pc.createAnswer());
reply = await post(answerUrl, { pc_id: pcId, sdp: pc.localDescription.sdp });
} else if (reply.next) reply = await post(answerUrl, { pc_id: pcId });
else break;
}
// "to-agent" is the channel RTVI messages are sent on, "to-caller" the agent's.
dc = pc.createDataChannel('to-agent', { negotiated: true, id: reply.channels.toAgent });
pc.createDataChannel('to-caller', { negotiated: true, id: reply.channels.toCaller });
Before the offer, add one audio transceiver (sendonly) with the microphone; the agent's track comes in Cloudflare's offer. Cloudflare needs no ICE candidates from the app, and a dropped connection ends the call.
Cards, page functions and notices arrive as RTVI server-messages (evasay.cards, evasay.tool, evasay.notice). Send typed words with send-text, and page reports and function answers as client-messages evasay.context and evasay.tool_result ({ id, result } or { id, error }).
Add a function of type Runs on your web page ("kind": "page") to the agent, and answer it in the app: over the WebSocket the call sends {"type":"tool","id","name","arguments"} and the app answers {"type":"tool_result","id","result"}. The agent may call only the functions the app named (page_tools or pageTools). They run signed in as the user: ask for confirmation in the app before anything that costs money or cannot be undone.