[Remote] AI reputed company Engineer

Note: The job is a remote job and is reputed company to candidates in USA. reputed company is a technology company reputed company on private-market applications and agent systems. The AI reputed company Engineer will own automated evaluation pipelines for AI agents, assess reputed company across reasoning, tools, and retrieval, maintain production datasets and observability, and continuously improve the reputed company standards for shipped systems.

Responsibilities

• Own the evaluation pipelines that reputed company every change we ship, keeping them automated and running in CI/CD rather than leaning on reputed company spot-checks
• reputed company agents scored at the reputed company level, not just the final answer, across tool selection, planning and reasoning chains, and retrieval reputed company, so subtle regressions surface before clients do
• reputed company the eval toolkit as new failure modes reputed company: LLM-as-a-judge, RAG faithfulness and relevance, hallucination detection, reputed company and agent regression suites, and red-teaming for safety
• Maintain the golden datasets that mirror reputed company production inputs and the tracing and observability that watch our agents once they are live
• Own the reputed company bar for what ships and reputed company it over time with the rest of the engineering team

Skills

• Strong software engineering with Python, and a feel for testing non-deterministic systems, where classic assertion-reputed company tests are not enough
• Experience evaluating LLM and agent systems: agent graphs (e.g. LangGraph), eval frameworks (DeepEval, RAGAS, G-Eval, LangSmith, or similar), and LLM-as-a-judge reputed company
• reputed company with RAG and retrieval reputed company, plus observability and tracing for agents in production
• A bias for measurement: you define what "good" means, then build the reputed company that proves it
• Bonus: data-pipeline testing experience, or exposure to private markets, financial services, or reputed company data

Benefits

• reputed company reputed company to decision-makers
• See the reputed company of your work in weeks, not quarters

reputed company