Staff Software Engineer

LLM optimization is a young field with no settled playbook, so the optimizations you invent here are genuinely new, and they ship into production rather than a paper.

And they matter: real enterprises run enormous volumes of LLM traffic and feel every wasted token, so the work you do lands on live customer requests and solves a problem they actually have. It's about as close to the frontier, and as close to the product, as engineering gets.03What you'll doInvent and ship optimizations. Design new strategies across every layer of a request: prompt and context compression, tool pruning, file and dev-command trimming, semantic caching, cache-prefix stabilization.

Then take them all the way to the hot path. Each one has to cut tokens on real traffic without degrading output.Prove every win. Build the evals and the token/quality instrumentation that separate a genuine saving from a plausible-sounding one.

An optimization only ships when the numbers back it.Live in the caching trade-offs. Provider prompt caches only pay off on byte-stable prefixes, and a naive rewrite can cost more than it saves. A lot of the job is knowing exactly when an optimization actually wins: deterministic transforms, per-session decision pins, semantic dedup.Build and own the gateway that runs them.

The real-time proxy that carries production LLM traffic, with streaming, retries and backpressure, executing every optimization on the live request path.Keep it fast and deploy anywhere. Defend a tight latency budget on the hot path, and package the whole thing so it installs cleanly into any customer's cloud, whether that's AWS, GCP, Azure, containers or Kubernetes, and upgrades without drama.Own systems end to end. From first commit to code serving live customer traffic, plus the spend-visibility and governance product built on top, with a direct line to the founders.Requirements: You've built and run backend or infrastructure systems in production, owned them end to end, and care about latency, correctness, and observability.You're comfortable in the request hot path.

Concurrency, streaming, tail latency and failure modes are things you reason about by default.You've shipped to cloud environments and can build something that installs cleanly in someone else's.You're strong in TypeScript/Node, which is our stack, or fluent enough in a systems language that picking it up is quick.You already build AI-native. Coding agents and LLM tooling are part of how you ship.You understand, or want to go deep on, LLMs, tokens, and how model APIs actually behave.This position is open to all candidates.