What three booleans couldn't say about a live voice session

The voice came out of the earpiece instead of the speaker.

Expo React Native, iOS and Android, live conversation with a speech model over WebRTC. Starting a session takes three async steps, any of which can fail, and they only work in one order: claim the platform audio session, fetch a short-lived token, open the peer connection. Nothing in the code said so.

What the flags couldn't say

The screen that ran sessions before the rewrite tracked all of this with three booleans:

const [isStarting, setIsStarting] = useState(false)
const [isSessionActive, setIsSessionActive] = useState(false)
const [isStopping, setIsStopping] = useState(false)

Eight combinations. Four of them describe a session that could actually exist. Tap Stop while the connection is still opening and you get isStarting and isStopping both true, which the type is perfectly happy with. The audio stack is not, because teardown is now running against setup, and whichever finishes second decides where the microphone ends up.

So every handler picked up a guard:

async function handleStart() {
  if (isStarting || isSessionActive || isStopping) return
  // ...
}

That line exists because the type is wrong.

One status enum instead of three booleans makes the illegal combinations unrepresentable. If that is the whole problem, stop there. It is an afternoon's work. Mine went further. An enum has no idea how far the start sequence got, so cleanup still has to reconstruct that from whatever other variables are lying around. Timers stay where they were, owned by nobody in particular. And an enum will happily let you call the three steps in the wrong order, which is the bug at the top of this page. That last one is what pushed me to XState v5 rather than a reducer.

The phase, and everything else

It is easy to overdo this. Move every piece of data into the state nodes and you end up with connected_micOn_speakerOff, plus a node for every other combination. The split I settled on: the phase is finite and mutually exclusive, everything else goes in context.

type SessionState =
  | 'uninitialized'
  | 'configuringAudio'
  | 'fetchingToken'
  | 'connecting'
  | 'connected'
  | 'waitingForFinalAudio'
  | 'summary'
  | 'teardown'

interface SessionContext {
  peerConnection: RTCPeerConnection | null
  audioTrack: MediaStreamTrack | null
  isMuted: boolean
  lastError?: Error
}

Mute is the test case. It changes constantly and it is legal in most phases, so promoting it to a phase of its own would double the chart and buy nothing.

Every src, actions and guard string in the snippets below is a key in this one block:

const sessionMachine = setup({
  types: {} as { context: SessionContext; events: SessionEvent },
  actors: { setupAudioPlatform, getCredentials, connectWebRTC },
  actions: { releaseAudioSession, logAudioFailure, showTokenError },
  guards: { isClientCompletelySilent },
}).createMachine({
  /* states go here */
})

The order is the whole argument

The old screen fetched the token first, claimed the audio session second, grabbed the microphone last. The machine has run audio, token, connection since its first commit.

That first step is not a getUserMedia wrapper. It claims the native call audio session through InCallManager (AVAudioSession on iOS, AudioManager on Android) and sets the category flags through expo-audio: play in silent mode, allow recording, whether to route through the earpiece. It also checks for a connected external device, which is how it decides between speaker and headset. The microphone itself arrives much later, inside the WebRTC step.

On iOS, acquire the microphone before the call audio session has been claimed and the track lands under the wrong category, routing already decided. Which is the earpiece bug. The fix sets speakerphone three times, at 100ms, 300ms and 500ms after the session goes active, under a comment saying iOS sometimes needs it set more than once. That is still in the code, because leaving it there is cheaper than being certain I have fixed the order on every device.

You could argue the token fetch belongs first, since it is the cheapest step to fail and it would save the other two the trouble. But cheap to fail also means cheap to retry, so nothing is lost by running it second. The audio session is different: the microphone cannot exist until it does.

I arrived at that ordering empirically, after days on the routing, and the explanation above is the model I built afterwards to account for it.

Teardown that knows how far it got

In a try/catch, the cleanup block has to work out what it is cleaning up:

try {
  await configureAudioSession()
  const token = await fetchAuthToken()
  await connectWebRTC(token)
} catch (err) {
  // Which of these actually happened?
  if (audioConfigured) releaseAudioSession()
  if (webrtcInitialized) closePeerConnection()
}

Those two booleans are the same mistake as the first three, one level down. In the machine each step is its own node, and the failure target is part of the declaration:

configuringAudio: {
  invoke: {
    src: 'setupAudioPlatform',
    onDone: 'fetchingToken',
    onError: { target: 'teardown', actions: 'logAudioFailure' },
  },
},
fetchingToken: {
  invoke: {
    src: 'getCredentials',
    onDone: 'connecting',
    onError: {
      target: 'teardown',
      actions: ['releaseAudioSession', 'showTokenError'],
    },
  },
},

The second one is the case the try/catch could not express. Token fetch fails. The audio session is already claimed, so it has to be released. The peer connection does not exist yet, so it must not be touched. Both facts sit in the transition, next to the failure they describe.

Where the timers live

The model speaks first. If the microphone is live during that, the user talks over the opening line and cuts it off, so the track stays disabled until the model is done.

As a setTimeout inside a hook, that is one more thing to clean up, and it leaks the moment the flow exits somewhere the timer did not expect. Attach it to the phase instead and the node owns it. Enter, it starts. Leave, it is gone.

waitingForFinalAudio: {
  on: { FINAL_AUDIO_RECEIVED: 'summary' },
  after: { 15000: 'summary' },
},

Final audio arrives, the machine moves to summary, and the fifteen second fallback dies with the node it lived in.

The guard that has never run

This next one I like, and it has never executed once in production.

The server can send a timeout event when it thinks the user has gone quiet. The client sometimes disagrees, because the microphone is picking up speech that has not been committed yet. Rather than settle that inside a listener, I made the disagreement a condition on the transition:

DATA_CHANNEL_MESSAGE: [
  {
    target: 'teardown',
    guard: 'isClientCompletelySilent',
  },
  // further candidates for other message types
],

When the guard returns false, XState falls through to the remaining candidates in the array. None of them match a timeout message, so the event is dropped. Nothing re-arms it and nothing keeps count. The only backstop is a separate maximum duration check that runs when speech stops.

None of which has ever happened. Turning on the server-side idle timeout means setting idle_timeout_ms when the session is created, and the API caps that at 30 seconds. I wanted five minutes. The line is commented out in the token function, in the same commit that added the guard, and nothing has set it at runtime since. So the timeout event never arrives, and the branch that arbitrates it has never been reached.

I left it in on purpose. A guarded transition says what should happen more precisely than a TODO comment would, and if the cap ever moves, or I find another way to trigger it, the decision is already made.

What it cost

The machine file is bigger than the component it replaced, and it has stayed bigger. Nobody got fewer lines out of this.

The audio stack is no cleaner either. iOS still wants the speaker set three times with delays, and there is still a 200ms sleep before the session resets, to dodge an AVAudioSession failure that shows up as !pri and OSStatus 561017449. What changed is where that code sits. It runs inside a named phase now, with a defined entry and exit, rather than in a component that might be mounted, unmounted, or halfway through something else.

What I did not expect was how much earlier the arguments happen. Whether a server timeout should beat live microphone input used to be a question you answered by reading a listener. Now it is a named guard sitting in a file doing nothing, which is at least visible.