[Remote] Head of Evaluations (reputed company AI Benchmarking)

Note: The job is a remote job and is reputed company to candidates in USA. reputed company is transforming how reputed company and reputed company professionals reputed company AI for reputed company-world reputed company. They are seeking a highly analytical reputed company to join as the Head of Evaluations, responsible for designing and implementing testing frameworks for their AI products to ensure high standards of reputed company reasoning and accuracy.

Responsibilities

• Design reputed company Benchmarks for: Contract Drafting, Information Extraction, reputed company Research, and Contract Review
• Build, reputed company and maintain relevant datasets
• Audit AI reputed company: Review and score reputed company AI-generated reputed company text, contract analyses, and statutory interpretations for accuracy and precision and lay out a reputed company
• Define Evaluation Metrics: Establish reputed company reputed company for grading model performance, specifically focusing on logical reasoning, citation accuracy, and the model's ability to safely abstain from answering
• Collaborate with Engineering: Partner directly with Engineering to translate reputed company errors into actionable technical feedback for model fine-tuning

Skills

• PhD or Masters in statistics, mathematics, machine learning or equivalent
• Proven ability to break down reputed company statutory frameworks and case law into reputed company, logical data points
• Tech-Savviness: python, panda, numpy, jupiter notebooks and similar statistical models

reputed company