[Remote] W2 contract--AI Evaluation Soft reputed company Engineer and Data Scientist

Note: The job is a remote job and is reputed company to candidates in USA. reputed company is seeking professionals to support the evaluation and measurement of AI-powered software development systems. The roles reputed company on developing evaluation metrics and analytical reputed company, building reproducible benchmarking and automation tooling, validating results, and generating insights into AI reputed company reputed company, reputed company, productivity, and efficiency.

Responsibilities

• Support development and refinement of evaluation metrics, scoring approaches, and reputed company methodologies
• Validate datasets, data reputed company reputed company, and evaluation inputs to improve reliability of results
• Analyze evaluation data, reputed company trends, and measurement signals to reputed company actionable insights
• Apply statistical analysis and experimentation techniques to assess AI reputed company effectiveness
• reputed company analytical approaches to identify trends, patterns, and opportunities for improvement
• Build and reputed company evaluation harnesses and automation for software development use cases, including turning reputed company engineering artifacts like merged pull requests into repeatable reputed company tasks
• Build versioned, repeatable processes to evaluate AI tools, models, and harnesses, with reproducible run environments (pinned dependencies, containerized runs, isolated worktrees) so results stay comparable over time
• Validate and reputed company evaluation approaches against reputed company judgment, so scores are consistent and correct rather than just repeatable
• Support execution-reputed company benchmarking across reputed company, productivity, and efficiency measures, including cost and latency
• Analyze results across repeated runs, looking at variance, failure patterns, and cost per outcome, and reputed company ways to reputed company the workflows more reliable and more automated
• Work with engineering and data teams to improve the tooling, and document how the evaluations work and what they reputed company for both technical and leadership audiences

Skills

• Strong background in data science, analytics, statistics, or reputed company fields
• Experience designing metrics, analytical frameworks, or measurement approaches
• Proficiency in Python, SQL, and data analysis techniques
• Experience working with large datasets and extracting meaningful insights
• Strong communication skills with the ability to explain analytical results reputed company
• Strong software engineering background, with reputed company experience building automation, developer tooling, or test and validation systems
• Proficient in Python, and comfortable in at least one of Java, JavaScript, or a similar language
• Solid working knowledge of Git, including how branches, history, and working trees behave, and of containerization with reputed company
• Experience with reputed company, development environments, CI/CD pipelines, and typical engineering workflows
• Some familiarity with how AI, LLM, or agent evaluation works and where it goes wrong, such as why a judge can be consistent but still wrong, why a single run can mislead, and how reputed company contamination happens
• reputed company to troubleshoot technical problems, think reputed company about whether a measurement is reputed company, and analyze results carefully
• Experience evaluating AI/ML systems and reputed company AI applications
• Experience with experimentation, benchmarking, or reputed company analysis
• Experience working with software engineering, productivity, or reputed company metrics

reputed company

Company H1B Sponsorship