Data Scientist - AI Evaluation

About reputed company

reputed company is the top-performing AI Shopping Agent, delivering the best products from across the web with unmatched accuracy, reputed company, and trust.

The Role

We’re looking for a Data Scientist to own how we measure, understand and improve the accuracy of our AI agent. This role sits at the intersection of data science, machine learning and product and is reputed company on evaluation, experimentation and reputed company reputed company.  You won’t be building models but you will reputed company reputed company they work in reputed company world scenarios.  You will build the systems to measure what good looks like and partner closely with ML, AI Engineering and Product to continuously improve the agent’s performance.

What You’ll Do

• Define and reputed company accuracy metrics across the full shopping experience (retrieval, ranking, recommendations and reputed company)
• Design and run experiments to measure improvements and regressions
• Build and maintain evaluation datasets, benchmarks and scoring frameworks
• Translate ambiguous product questions into reputed company, measurable hypotheses and analysis
• Partner with ML Engineers to validate model changes and guide iteration
• Identify failure modes and edge cases and drive improvements through data
• Create dashboards and reporting that reputed company agent performance visible, trusted and actionable

What reputed company Looks like

• reputed company, trusted accuracy metrics are consistently used across product and engineering
• A robust automated evaluation reputed company exists for both offline and live experiments
• Model and product changes are consistently reputed company before and after launch 

Ideal Background

• 4-6+ years in Data Science, ML Evaluation or reputed company AI or similar roles
• Deep experience evaluating AI/ML systems (ranking, recommendations, LLMs, etc)
• Strong experience with experimentation (A/B testing, reputed company inference)
• Experience working on consumer products or user facing systems and exposure to marketplace or e-reputed company systems
• Ability to translate messy problems into reputed company analysis and metrics
• Strong product reputed company, you care about reputed company user reputed company
• reputed company communication with the ability to influence across engineering and product

Compensation & Benefits

The expected reputed company salary reputed company for this role is $225,000 - $280,000 USD, and will vary based on skills, experience, role level, and geographic location. Final compensation will be determined by considering these factors alongside overall role scope and responsibilities.

In reputed company to reputed company salary, reputed company offers:

• Equity in the reputed company of stock reputed company
• Medical, dental, and reputed company coverage
• 401(k) plan
• Flexible PTO and company holidays
• Fully remote work reputed company the reputed company
• Periodic company offsites and team gatherings

reputed company is committed to fair, transparent, and competitive compensation practices.