Design feedback loops so that using a product generates data that improves it. Instrument systems so failures arrive with enough context to reproduce, then turn the recurring ones into evaluation cases.
Design the boundary between a model and the tools it calls. Build or improve the model harness, decide what the model handles, what it delegates, and how to keep it working from the state it actually fetched.
Work to a latency and cost target per request: streaming, partial results, caching, model routing, prompt structure.
Build the internal harnesses and tools (CLI, MCP, and others) your team needs to iterate quickly, and share the parts that generalize with other teams.
Raise the bar on your team through code and design review, and bring other engineers up on eval practice.
Some of the technical questions in this area are still open. You will help answer them.
Participate in an on-call rotation to ensure our systems remain available to customers 24/7. Team members alternate as the on-call primary responder, which may require immediate response outside normal working hours, including weekends.
What We Believe are Important Traits for This Role
Required Education/Certification:
Bachelors Degree in STEM discipline and 5+ years of software engineering experience, with production ownership of services, pipelines, or SDKs.
Technical Skills & Tools (Must-Haves):
2+ years shipping LLM-backed features to real users, in systems that carried error budgets, on-call rotations, and customers who noticed regressions.
Data engineering depth: SQL, at least one distributed processing framework, and experience with pipelines where a wrong record mattered more than a slow one.
Fluency with tool calling and agent orchestration, including the failure modes: stale context, hallucinated arguments, silent partial success, unbounded loops.
Strong Python or TypeScript, and comfort reading code in whatever language the caller happens to be written in.
Core Competencies & Scope:
Direct experience or deep understanding of designing evaluations for non-deterministic systems. You can describe a dataset you built and the failure it caught.
Working knowledge of more than one agent harness, and opinions about where each of them is weak.
Experience diagnosing latency in a distributed request path.
Comfort with ambiguity, and the judgment to ship something narrow that works while the general solution is still unclear.
Nice to Have Traits for this Role
Geospatial data experience: routing, geocoding, POI or address data, OpenStreetMap, or conflation of overlapping sources.
Public API or SDK design experience, particularly for developers you never talk to.
Experience building against MCP or similar tool transports.
Experience running evals in CI, with a commercial harness or one you built.
Automotive, in-vehicle infotainment, CarPlay, or Android Auto experience.
Voice pipeline experience: streaming ASR, TTS, barge-in, endpointing, wake word.
Work under constrained compute, offline, or intermittent connectivity.
Experience operating a product through its first external integrations, where the customer finds the gaps before you do.
Sponsorship data shows employers' past filings in public US government records (DOL, USCIS, E-Verify) and is not a promise that any job will be sponsored. Immigration rules change. Verify your situation with your DSO or an immigration attorney.