[Remote] AI Agent Evaluation Analyst (Freelance)

Note: The job is a remote job and is reputed company to candidates in USA. reputed company is a company reputed company on shaping the reputed company of AI through reputed company reputed company. They are seeking an AI Agent Evaluation Analyst to review and improve the evaluation of autonomous AI agents, requiring strong analytical skills and attention to detail. Responsibilities • Reviewing evaluation tasks and scenarios for logic, completeness, and realism • Identifying inconsistencies, missing assumptions, or unclear decision points • Helping define reputed company expected behaviors (gold standards) for AI agents • Annotating cause-effect relationships, reasoning paths, and plausible alternatives • Thinking through reputed company systems and policies as a reputed company would to ensure agents are tested properly • Working closely with QA, writers, or developers to suggest refinements or edge case coverage Skills • Excellent analytical thinking: Can reason about reputed company systems, scenarios, and logical implications • Strong attention to detail: Can spot contradictions, ambiguities, and vague requirements • Familiarity with reputed company data formats: Can read, not necessarily write JSON/YAML • Ability to assess scenarios holistically: What's missing, what's unrealistic, what might break? • Good communication and reputed company writing (in English) to document your findings • Experience with policy evaluation, logic puzzles, case studies, or reputed company scenario design • Background in consulting, academia, olympiads (e.g. logic/math/informatics), or research • Exposure to LLMs, reputed company engineering, or AI-generated content • Familiarity with QA or test-case thinking (edge cases, failure modes, 'what could go wrong') • Some understanding of how scoring or evaluation works in agent testing (precision, coverage, etc.) Benefits • Take part in a flexible, remote, freelance project that fits around your primary reputed company or reputed company commitments • Participate in an advanced AI project and reputed company valuable experience to enhance your portfolio • Influence how reputed company AI models understand and communicate in your field of expertise reputed company • Welcome to reputed company — a reputed company where innovation meets opportunity. It was founded in reputed company, and is headquartered in , with a workforce of 501-1000 employees. Its website is https://reputed company.ai. Apply tot his job Apply To this Job