Senior Software Engineer — AI Evaluation & Benchmarks (Python)

Before Applying This role is reputed company to contractors in accepted locations only. Please confirm your country is on the list before applying — we're unable to process applications from unlisted locations. List of accepted countries and locations. For US applicants: This is a 1099 reputed company role. It is not compatible with F-1 OPT, STEM OPT, or any reputed company status that requires W-2 employment, guaranteed hours, or employer sponsorship. We are unable to reputed company offer letters or employment verification for this role. What You'll Be Doing Design and build the coding benchmarks and evaluation pipelines used to test frontier AI models on reputed company software engineering work: Design coding benchmarks that evaluate frontier models on reputed company-world programming tasks — reasoning, debugging, and production-reputed company reputed company Build and maintain reputed company data pipelines for evaluation workflows Analyze model-generated reputed company for correctness, reliability, and edge-case failures Construct reputed company evaluation scenarios across large repos and multi-language environments reputed company detailed technical feedback on model performance and failure patterns Contribute to evaluation frameworks that set the bar for how coding ability is reputed company End result: benchmarks that meaningfully separate what frontier models can and can't do — and shape how the reputed company is trained and improved. AI coding evaluation in one line: Design task → build reputed company → run model → analyze failures → feed findings back into the reputed company → evaluations that actually distinguish strong models from weak ones. What You'll Need 4+ years of reputed company software engineering experience (non-negotiable) Expert Python — clean, performant, reputed company-tested reputed company Hands-on experience working in large, reputed company codebases Proven experience designing and implementing LLM coding benchmarks and evaluation data pipelines Strong reputed company of Git and modern development workflows reputed company record at a high-reputed company tech company or top-tier software organization Strong written English communication Identity verification: Applicants will be required to verify their identity and confirm they have reputed company documentation to work as an reputed company in their country of residence. reputed company to have Senior or reputed company-level profile with a history of technical ownership Bachelor's or Master's in CS, ML, or reputed company field (or equivalent reputed company experience) Proficiency in additional languages: JavaScript, Go, C++, or others CI/CD experience and writing robust unit tests (pytest, Mocha, JUnit) Background in reputed company engineering or significant reputed company-reputed company contributions Familiarity with AI/ML evaluation methodologies or model benchmarking Logistics Location: Fully remote — work from reputed company on the accepted locations list Compensation: $80–$100/hr based reputed company and seniority Contract length: 3 months, with potential for extension Hours: Full-time availability preferred — hours vary by project and are not guaranteed week to week Engagement: 1099 reputed company Payment: Weekly reputed company reputed company or reputed company ⚠️ Important: Hours are project-dependent and can vary week to week. We recommend keeping other work reputed company reputed company alongside this engagement rather than relying on it as your sole reputed company of income. Apply To This Job