Product Manager, LLM Router

About the role

The company's LLM router lets developers and enterprises select a model for each request based on task quality, cost, latency, reliability and customer policies. It is hiring a Product Manager to own the routing product and lead the engineering team that builds and operates it: setting the roadmap, running a small team of senior engineers and being accountable for what ships and how it performs in production.

Key responsibilities

• Lead the engineering team and own delivery: set priorities, clarify ownership, agree milestones, review progress, resolve blockers and balance features, reliability and technical debt.
• Own the routing roadmap: define how the router selects a model and which signals guide it (prompt characteristics, historical performance, real-time latency, customer policies).
• Drive evaluation strategy: design benchmarks for coding, reasoning, summarization, tool use and creative tasks, using offline evaluations and production data.
• Build for developers and enterprises: single API with sensible defaults, plus custom routing policies, governance, audit logs, private deployments and SSO.
• Shape the developer experience through APIs, SDKs, dashboards and observability tools.
• Improve cost and reliability: budget controls, per-route cost caps, automatic fallbacks, retries, caching and service-level commitments.
• Manage the model portfolio: choose which models and providers to add, promote or retire, working with providers on integration and pricing.
• Stay close to customers and measure outcomes such as routing quality, cost per request, latency and retention.

Required profile

• 5+ years of product management, including 2+ years on developer-facing, API-first or infrastructure products.
• Experience leading a software engineering team and owning its delivery; experience leading senior engineers is especially relevant.
• Hands-on use of AI tools such as Claude or Cursor.
• Practical understanding of LLM fundamentals (prompts, context windows, sampling, tool use, model evaluation) and the model ecosystem.
• Track record of shipping technical products with trade-offs among quality, latency, cost and reliability.
• Strong analytical skills, clear written communication and decision-making with incomplete information.

Bonus points

• Engineering background or experience building production systems with LLM APIs.
• Building LLM-powered applications such as agents, RAG systems or copilots.
• Understanding of inference economics (token pricing, batching, prompt caching, model serving).
• Enterprise sales or support experience and experience working with model providers.