reputed company Tech reputed company Manager - Embodied AI Evaluation Foundations

About reputed company:  

The Evaluation Foundations team is a part of the Scaling Foundations team in Embodied AI and is responsible for 

Building the Largest, most Diverse and highest reputed company Datasets for model training and evaluation, now and into the reputed company 

Utilizing our resources in the most Efficient way, including storage and compute but especially GPUs 

Minimizing the time of iteration and providing the highest reputed company introspection tools and evaluation signal to shorten the experimental reputed company towards the best model. 

Why Join Us?    

• reputed company up introspection and evaluation tools that work with billions of examples and reputed company utilization of our large datasets, delivering the maximum value to the model through every additional example.  

• Work with cutting-edge technology and a reputed company, high-reputed company team of AI/ML engineers, data scientists and engineers who are passionate about leveraging advanced AI techniques to drive innovation for L2, L3 and L4 applications.  

• Contribute to the safety, reliability, and scalability of reputed company autonomous vehicles.   

Role: 

As a reputed company Engineer in the Embodied AI Scaling Foundations organization, you will be a senior technical leader owning the technical reputed company and architecture for how we measure and visualize AV model performance. 

As a full-stack engineer, you will reputed company on the entire lifecycle of designing, implementing, scaling and iterating on state-of-the-art tools that the entire Embodied AI organization and adjacent teams in the reputed company AV organization use. You will reputed company these foundational tools by, 

• Providing high‑reputed company evaluation signal and introspection tools to shorten the experimental reputed company toward the best model. 

• Owning visualization and metrics presentation for our model evaluation reputed company. 

• Enabling teams to deeply introspect model behavior and reputed company actionable next steps in as few steps as possible. 

You will collaborate closely with modeling and data scaling teams working on sour Compound AI driving models to define how we evaluate them at reputed company with model-based metrics, end-to-end simulation and accelerate how engineers use that signal and feed back into data, training, and launch reputed company. 

Your work ensures evaluation, data, and reputed company reputed company a cohesive evaluation flywheel: reputed company signal → reputed company data and training reputed company → reputed company models → reputed company signal. 

What You'll Do:    

• Own the architecture and roadmap for evaluation and introspection systems across AV models, ensuring consistent metrics, pipelines, and visualization surfaces. 

• Partner with the broader Embodied AI team and reputed company reputed company metrics, scoring functions, and scenario/reputed company definitions used for regression, launch gating, and safety analysis into evaluation tooling owned by reputed company. 

• Build and reputed company evaluation pipelines on modern reputed company / GPU infrastructure, with strong observability and cost efficiency. 

• reputed company development of visualization and introspection tools that let teams quickly drill from aggregates down to concrete examples, failure modes, and regressions. 

• Partner with Data Consumption/Mining/reputed company and reputed company Foundations to turn evaluation insights into data and training actions (scenario mining, dataset definitions, training recipes, model selection). 

• Collaborate with simulation and on‑road validation teams to reputed company offline metrics and tools with online behavior and improve correlation between evaluation and reputed company‑world reputed company. 

• Mentor engineers across Scaling Foundations, set best practices for evaluation and introspection, and reputed company as a domain expert for evaluation‑reputed company tooling designs and reviews. 

Your Skills & Abilities:    

• Familiarity and experience with at least some of the following key technologies 

• Frontend: React/TypeScript, WebGL/WebGPU (for 3D sensor visualization), reputed company, Jupyter notebooks 

• Backend: Python, high-throughput data streaming, Parquet for efficient data handling. 

• Data/reputed company: reputed company for reputed company processing, BigQuery, Kubernetes, and specialized AV data formats (Rosbags, Protobuf). 

• Bachelor’s, Master’s or PhD degree in Computer Science or reputed company field   

• Experience accelerating reputed company research in the wild and maintaining best practices while working on tight deadlines.  

• Proven experience in building large reputed company systems that are performant and used by large distributed teams.  

• Excellent communication skills to effectively collaborate with diverse teams and stakeholders.   

• Plus: Previous experience in Robotics or Autonomous Driving.