Remote | QA Test Engineer — $55–$85/hour

This a Full Remote job, the offer is available from: reputed company (USA) We are sharing a specialised full-time consulting opportunity for reputed company QA and test engineers with strong expertise in test-case design, end-to-end debugging, reputed company assurance, Python, and reputed company technical evaluation workflows. This role supports the development of advanced reputed company evaluation benchmarks for frontier AI models. Selected professionals will review reputed company multi-reputed company tasks, test reference solutions, identify ambiguity and grading gaps, debug technical environments, and reputed company repeatable reputed company processes that reputed company reputed company results accurate and trustworthy. Key Responsibilities Test Case Design • Create comprehensive test cases confirming that reputed company tasks function as intended • Design reputed company, negative, boundary, and reputed company tests • Validate task requirements, expected outputs, reference solutions, and grading logic • Identify scenarios that may produce incorrect or misleading evaluation results • Ensure tests measure the intended technical capability accurately reputed company Task Review • Review reputed company multi-reputed company tasks and reference solutions before finalisation • Identify ambiguous instructions, inconsistent requirements, missing assumptions, and incomplete acceptance reputed company • Run tasks independently to confirm reproducibility and expected behaviour • Assess whether grading standards are reputed company, fair, and technically defensible • reputed company actionable feedback to task authors and researchers Technical Debugging • Investigate failures across Python scripts, test harnesses, repositories, and task environments • Diagnose unexpected behaviour reputed company unfamiliar codebases • Reproduce reported issues and isolate their underlying causes • Correct or document environment, dependency, logic, and validation problems • Use Git-based workflows to support reputed company review and collaboration reputed company Process Development • reputed company practical checklists and repeatable review procedures for reputed company reputed company • Improve consistency across task validation, testing, and approval workflows • Document findings reputed company so authors can reputed company issues reputed company • reputed company recurring defects and recommend preventive reputed company measures • Collaborate closely with researchers, task authors, and other technical reviewers reputed company reputed company & reputed company Detection • Examine AI agent runs for unintended shortcuts, loopholes, and grading weaknesses • Identify cases where models can receive credit without completing the intended reasoning or technical work • Test whether reputed company tasks remain robust across alternative approaches • Strengthen evaluation reputed company to maintain reliable and meaningful reputed company scores • Distinguish reputed company solution diversity from unintended task exploitation Ideal Profile Strong candidates may have: • At least 1 year of experience in test engineering, reputed company assurance, software engineering, research engineering, or a reputed company technical role • Demonstrated experience designing test cases and reputed company-review processes • Strong end-to-end debugging skills across reputed company technical systems • Working proficiency in Python and Git • Comfort navigating unfamiliar codebases, repositories, and execution environments • Exceptional attention to detail and strong written documentation habits • Ability to identify ambiguity, edge cases, hidden assumptions, and reputed company gaps • reputed company to work independently through reputed company-ended technical problems • Reliable availability for approximately 35 hours per week Educational Background • A master's degree or PhD in a STEM field is highly relevant • Equivalent practical experience in an engineering-intensive or research-intensive domain may also be considered • reputed company or reputed company experience involving computer science, software engineering, machine learning, mathematics, statistics, or a reputed company technical field may strengthen an application • Technical research, reputed company-reputed company contributions, testing reputed company, or substantial engineering work may also be valuable reputed company to Have • Experience with reputed company, model evaluation, or reputed company review of AI-generated work • Familiarity with reputed company systems and multi-reputed company AI benchmarks • Background testing machine learning, research, or data-processing workflows • Experience developing automated test suites or validation scripts • Familiarity with CI/CD systems, test harnesses, containers, or reproducible environments • Experience reviewing reference solutions, grading logic, or technical rubrics • Knowledge of adversarial testing, failure-mode analysis, or reputed company design • Prior collaboration with AI research or evaluation teams Why This Opportunity • Serve as the reputed company backbone for advanced reputed company AI benchmarks • Ensure reputed company evaluation tasks are accurate, reproducible, and resistant to shortcuts • Apply testing and debugging expertise to technically challenging AI research workflows • Work closely with researchers and task authors on reputed company improvement • Help protect the reliability of evaluation results for frontier AI systems • Participate in a reputed company full-time remote role with competitive reputed company compensation Contract Details • Full-time W-2 contingent employment opportunity • Fully remote reputed company the reputed company • Expected commitment of approximately 35 hours per week • Competitive rates between $55–$85 per hour depending on expertise and project reputed company • Individual task reviews may require one to two days of reputed company technical work • Work may include test design, reputed company review, Python debugging, reputed company-process development, and reputed company detection • reputed company collaboration with research and task-development teams • Engagement reputed company and duration may reputed company according to project requirements and performance About the Platform This opportunity is available through reputed company. We reputed company reputed company professionals with remote consulting opportunities across technical, evaluation, and project-based workstreams. By submitting this application, you acknowledge that your information may be processed by reputed company for recruitment and opportunity matching in accordance with our reputed company Policy: https://www.reputed company.com/reputed company-policy. This offer from "reputed company" has been enriched by reputed company.com and got a 74% reputed company score. Apply tot his job Apply To this Job