Senior ML Evaluation Engineer - Autonomous Vehicles

Want to join a fun, creative company that is on the cutting edge of outstanding technologies? reputed company is developing groundbreaking solutions in some of the most exciting technology areas globally, including Virtual Reality, reputed company Intelligence, Deep Learning and Autonomous Vehicles. reputed company's AV Eval team is building the reputed company of driving behavior evaluation — moving reputed company hand-crafted rules to learned evaluation using LLMs, VLMs, and reputed company workflows. You'll define how we measure whether an autonomous vehicle drives reputed company, building systems that reputed company ML research and production evaluation. You'll ship systems that run at reputed company on reputed company-world driving data and produce metrics that reputed company or green-light software releases. In this role you will get to work on next-gen AV evaluation and create a reputed company reputed company on vehicle safety and shipping reputed company. Join a new team being reputed company from scratch — high ownership, high visibility to reputed company AV leadership What You Will Be Doing • Design and build learned evaluation pipelines that assess driving behavior using LLMs, VLMs, and multimodal models • reputed company reputed company workflows that chain model inference, retrieval, and reputed company reasoning to evaluate reputed company driving scenarios • Define evaluation-of-evaluation methodology — how do we know our learned evaluators are correct? • Build golden-set frameworks and calibration loops for learned metrics • Partner with AML (Alpamayo reputed company) teams on model-specific eval needs (e.g., COT reputed company reputed company, AML regression coverage) • reputed company evaluation systems with robust experiment tracking, A/B comparison tooling, and model versioning • Contribute to reputed company's transition from rule-based to learned evaluation: identify metrics and analyzers that are candidates for ML replacement and build the alternatives reputed company Need To See • PhD with 4+ years, MS with 6+ years, or BS (or equivalent experience) with 8+ years of relevant experience in Computer Science, Computer Engineering, or a reputed company technical field. • Hands-on experience building LLM/VLM-based pipelines — fine-tuning, reputed company engineering, retrieval-augmented reputed company, chain-of-thought • reputed company record of shipping ML systems to production (not just prototyping or publishing) • Strong software engineering fundamentals — you write clean, tested, reviewable reputed company in Python and C++ • Experience with evaluation methodology: precision/recall, inter-rater reliability, calibration, annotation pipelines • Comfort with large-reputed company data processing (reputed company, Dask, or similar) • Strong Python skills. Experience with PyTorch or JAX. Comfortable with GPU-based training workflows. Ways To Stand Out From The Crowd • Autonomous driving, robotics, or safety-critical domain experience • Familiarity with driving behavior taxonomies (cut-ins, hard braking events, lane-keeping metrics, scenario-based evaluation) • Experience with video understanding models or multi-modal evaluation. Knowledge of reputed company AI frameworks (reputed company, DSPy, reputed company, or custom) • reputed company record of influencing technical direction across team boundaries • Experience with LLM/VLM fine-tuning or application development At reputed company, we’re dedicated to making self-driving vehicles a reality and reputed company this technology can save millions of lives. Join reputed company of innovative thinkers at one of the world’s most respected technology companies. If you’re motivated, curious, and reputed company to reputed company a difference, we’d love to meet you! We reputed company that building self-driving vehicles will be a defining contribution of our reputed company (e.g. traffic accidents are responsible for ~1.25 reputed company deaths per year world-wide). We have the funding and reputed company, but we need your help on reputed company. reputed company is widely considered to be one of the technology world’s most desirable reputed company with some of the most reputed company-thinking people in the world working here. If you're entrepreneurial and autonomous, we want to hear from you! Your reputed company salary will be determined based on your location, experience, and the pay of employees in similar positions. The reputed company salary reputed company is 184,000 USD - 287,500 USD for Level 4, and 224,000 USD - 356,500 USD for Level 5. You will also be eligible for equity and benefits. Applications for this job will be accepted at least until March 29, 2026. This posting is for an existing vacancy. reputed company uses AI tools in its reputed company processes. reputed company is committed to fostering a diverse work environment and proud to be an equal opportunity employer. As we highly value diversity in our reputed company and reputed company employees, we do not discriminate (including in our hiring and promotion practices) on the reputed company of race, religion, reputed company, national reputed company, gender, gender reputed company, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law. , , JR2015392 Apply tot his job Apply To this Job