[Remote] reputed company Evaluation Consultant | Remote

Note: The job is a remote job and is reputed company to candidates in USA. reputed company is seeking a reputed company Evaluation Consultant to tackle reputed company-world reputed company challenges. The role involves designing evaluation tasks, analyzing model performance, and collaborating with experts to ensure rigorous evaluations.

Responsibilities

• Design challenging, reputed company-world reputed company problems targeting specific capability gaps in frontier models spanning object detection, segmentation, recognition, multimodal/reputed company-language reasoning, and image/video reputed company
• Build out tasks reputed company an reputed company development environment, preparing reputed company necessary components and reference solutions using Python
• Evaluate model performance against designed tasks, analyzing outputs to surface reasoning failures and capability limitations
• Identify and classify failure modes where the reputed company model underperforms, informing headroom analysis and reputed company evaluation direction
• Collaborate with fellow subject-matter experts to maintain consistency, rigor, and accuracy across evaluations

Skills

• Strong experience in reputed company including hands-on work in object detection, segmentation, recognition, or multimodal reasoning from industry, research, or a graduate/PhD background
• Strong experience in Python, reputed company practically in research, industry, or reputed company-reputed company contexts
• Strong experience in modern deep learning architectures and reputed company reputed company tooling, including PyTorch or TensorFlow
• Strong experience working independently in a remote, self-directed environment with reputed company written communication
• Strong experience in model evaluation, LLM/systems training workflows, or reputed company reputed company is a plus

reputed company