A 7-stage speaker-identification cascade

Given a call recording, work out who said what. Diarization gives you "speaker 1" and "speaker 2". Turning those into people is the actual problem.

Seven stages, first match wins, ordered by confidence: calendar attendees matched on email, internal directory matched on name and company, vision analysis of sampled frames, external enrichment for people outside the organization, manual assignment, contextual inference, anonymous fallback.

The ordering is the design, not an afterthought. Calendar attendees cost nothing to check and resolve most internal calls outright. Vision analysis is accurate and expensive — a model reading sampled frames for a name on a caption, a badge, or a shared screen, on its own task definition so a RAM-heavy call can't starve the summary worker's quota — so it sits behind two cheap checks that resolve most cases first. In practice it runs on a small minority of speakers.

"Most accurate" isn't "most cost-effective when two cheap checks already agree" — which is why vision doesn't go first.

Two details mattered more than expected. The directory match requires name and company, not name alone — otherwise two John Smiths at different companies collapse into one, and that error propagates into every downstream relationship. And the stage that produced each assignment is stored on the row, which turns the cascade from a black box into an audit trail: an operator can see a name came from an inferred nickname rather than a calendar invite, and treat it accordingly.

Merging two speakers who turn out to be the same person has to be reversible, and reversibility is a schema decision, not a UI one: the merge preserves the pre-merge owner on every segment and soft-deletes the redundant per-conversation row rather than destroying it. The cross-conversation directory is never touched — a per-call correction stays a per-call correction.

The cost of first-match-wins is that a high-confidence stage can never be overruled by a lower-confidence one that happens to be right: a stale calendar invite beats the evidence on screen. We took that trade because it makes the cascade deterministic, and determinism is what makes it replayable.