Senior Platform Engineer · Nov 2025 – Jul 2026
Multi-tenant conversation intelligence
Turning 100 hours of calls a week into something you can query
Built for an employer, described with their approval. The product, its clients and its internals stay unnamed — the architecture carries the signal, those names don’t.
Upload lands in object storage, seven FIFO queues fan work across nine workers, one Postgres holds state, and two surfaces read it. Every queue has a dead-letter queue behind it — a stage that fails three times stops retrying and becomes visible instead of silently looping.
Anonymous labels resolve to real people through seven checks ordered by confidence, not by cleverness. The cheap deterministic matches run first and settle most cases; the expensive vision call sits at stage three and therefore almost never runs. Every match records how it was made, so a wrong one can be traced and reversed.
A Lambda computes backlog-per-task every sixty seconds and target tracking holds it at five messages per task. CPU is a lagging indicator — it tells you a worker is busy, not that work is waiting. Queue depth is the honest signal, and switching to it is most of where the ~94% compute saving came from.
Four stages, promoted on evidence thresholds — 3 pieces to emerging, 5 across 2 conversations to validated, 10 across 3 to decision-grade. There is no path back down: a signal's stage is a fact about evidence accumulated, not a value that can flap between runs, so 31 downstream artifact types can gate on a threshold comparison instead of a graph walk.
Thinking tokens stream on their own channel while the model decides which of 24 tools to call; every call is dispatched in parallel and passes through one executor that rewrites venture scope before anything touches the database. A prompt injection that convinces the model to name a foreign venture still can't reach it — the boundary isn't the model's judgment, it's the layer underneath it.
The problem
About 120 calls a week, roughly 100 hours of them. Every one contained something worth keeping — an objection that would recur, a pattern across portfolio companies, a decision and the reasoning behind it. All of it evaporated into recordings nobody rewatched.
The ask was not "transcribe the calls." Transcripts are just longer recordings. The ask was to end up with something a person could query — and, later, something an agent could query.
The pipeline
Nine workers across seven queues: transcription, speaker identification, summary, learning
extraction, collage, reflection, and the supporting ingestion and reprocessing stages. Each queue
has its own dead-letter queue with bounded retries, and a shared concurrent message-pool library is
the reliability backbone underneath all nine — not just a name for "shared retry logic" but four
specific mechanisms: a visibility-timeout heartbeat that extends SQS visibility by 7 minutes every
3 minutes so long-running jobs survive past the default 15-minute window, a circuit breaker that
trips at a 50% error rate in a 30-second window, ECS scale-in protection enabled the moment a job
starts and disabled in a finally block so the autoscaler can never kill a task mid-job, and a
graceful SIGTERM drain that waits up to 30 seconds for in-flight messages before exiting. Every
worker entrypoint shrinks to about 50 lines of glue on top of it.
Queue-per-stage rather than one queue with a type field, because the stages have genuinely different shapes. Transcription is long and I/O-bound. Speaker identification is a cascade of cheap checks with an expensive vision call at the end. Collage is pure FFmpeg compute. Giving each its own queue means each gets its own retry policy, its own concurrency, and its own scaling signal — and one stage backing up never starves another.
Two scars from running this in production, not just designing it. A documented Gemini 2.5 Pro bug
returns empty responses when maxOutputTokens is set below 2048, so the summary worker pins a
floor of 8192 regardless of what a given response actually needs. And OpenAI embedding calls are
batched at 100 requests a second with a deliberate 1-second delay between batches — not for
throughput, but to stay under the per-org token-per-minute cap. Without that throttle, scaling
workers horizontally would blow straight through the limit and every worker would start failing at
once.
Threads, scored not matched
Deciding whether a new call continues an existing thread — the same deal, the same recurring
conflict — isn't a lookup, it's a score: 0.50 × company_match + 0.35 × jaccard(speakers) + 0.15 × time_decay. Above a threshold, the conversation joins the highest-scoring active thread; below it,
it starts a new one.
The 50/35/15 split wasn't guessed. It was tuned empirically against 200 labeled conversation pairs — "same thread" or "different thread" — picking the weighting and threshold that maximized F1. Company match dominates because it's the strongest signal a person would use too; speaker overlap catches the case where the company changes but the actual conversation is the same; time decay stops two unrelated calls six months apart from linking just because the same two people happened to be on both.
The decision I'd defend hardest
Underneath the promotion ladder is a three-layer model that's the actual spine of the pipeline: Learnings (per-conversation insights the learning-extraction worker pulls straight out of a transcript), Promoted Signals (the same insight once it recurs across multiple conversations, clustered by cosine similarity), and Artifacts (31 generated documents — briefs, digests, one-pagers — created once their prerequisite signals exist). Everything downstream of transcription exists to move a fact from the first layer to the third.
Extracted learnings climb a four-stage ladder — candidate, emerging, validated, decision-grade — promoted on evidence thresholds: three pieces to reach emerging, five across two distinct conversations to reach validated, ten across three to reach decision-grade.
Signals never demote. There is no path back down the ladder.
That constraint is the whole point. It means a signal's stage is a fact about evidence accumulated, not a value that can flap between runs as clustering shifts. Downstream consumers — artifact gating, the assistant's tools, the dashboards — can treat a stage as stable and cache against it. Without monotonicity, every consumer needs to handle a signal moving backwards, and in practice none of them would have.
It also let 31 prerequisite-chained artifact types be evaluated in two database queries instead of recursive traversal: if stages only ever increase, "is this artifact's prerequisite met" is a threshold comparison, not a graph walk.
The thresholds are deliberately conservative. A false positive in the artifact layer misleads a founder; a false negative just means an artifact hasn't generated yet. Those costs are not symmetric, so the thresholds aren't either.
Retrieval
3,072-dimension embeddings in pgvector, spanning transcript segments, conversation summaries, and reflection insights — three different granularities in one index, so a query can match a passing remark or a whole-call theme.
The clustering that feeds signal promotion is cosine similarity over the same index. Worth being honest about a rough edge: nearest-neighbour ties break on row order, so cluster membership can shift slightly between re-runs. That's tolerable precisely because the promotion thresholds are robust to small shifts — a signal sitting on exactly five pieces of evidence is rare, and one that does isn't load-bearing.
The surfaces
Two, deliberately.
A chat assistant with a 24-tool function-calling registry across five categories — search, RAG, list, detail, temporal — with thinking-token streaming and parallel tool dispatch over SSE. Tools run concurrently and settle together rather than serially, because most useful questions need three or four lookups and doing them in sequence is the difference between two seconds and eight.
And an MCP server exposing the same intelligence layer to external agents. Same data, same authorization, no bespoke integration per consumer.
Isolation and cost
Tenants are isolated with Postgres row-level security in a single deployment. The alternative — one deployment per customer — is easier to reason about and enormously more expensive to operate; every migration, every deploy, every incident multiplies by tenant count. RLS puts the isolation in one place that can be tested exhaustively.
The cost work was the least clever and the most valuable. Per-task serverless containers are excellent until workload shapes diverge, and ours had three: model-inference workers wanting sustained CPU, a tiny ingestion service idling most of the day, and FFmpeg jobs that were pure compute and indifferent to architecture. Paying one premium across all three meant overpaying on at least two.
Three EC2 capacity providers, each tuned to a shape, with autoscaling driven by queue backlog per task rather than CPU. Queue depth alone is the wrong signal — fifty messages with ten workers is fine, fifty with one is not — and CPU lags reality by minutes on a queue-driven system. Backlog-per-task is the honest unit. About 94% off the compute bill.
Waking a service back up from zero tasks needed its own trick. Target-tracking alarms fire on >,
not >=, so a metric sitting exactly on the target value never breaches — and with zero tasks
running, backlog-per-task isn't a number the alarm can even compute. The backlog-metric Lambda
works around it: when there are no running tasks and at least one message waiting, it reports
max(messages, target) + 0.1 instead of the literal backlog. That reliably trips the alarm and
brings a task back online. It's a small, deliberately hacky fix for a real gap in how
target-tracking alarms are defined.
What I'd do differently
The promotion thresholds were tuned by judgment and never A/B'd against outcomes. They're defensible and they're conservative, but "conservative" was an argument rather than a measurement. Given the time again, I'd instrument which signals actually influenced a decision and let that calibrate the ladder.