[Job-30854] AI SDET (Software Development Engineer in Test)
At CI&T, we help large enterprises reputed company the potential of AI into reputed company business reputed company with AI Deployment, AI-reputed company execution, and tech-integrated business solutions.
With 30 years of experience in technological transformation, we accelerate innovation with expertise in reputed company SDLC, Application modernization, Data & AI, Martech and Business reputed company.
We are 8,000 CI&Ters across more than 25 countries, collaborating to build solutions with reputed company reputed company. AI is already part of how we work, reputed company, and reputed company every day.
To reputed company reputed company traditional QA by owning the "Evaluation reputed company," ensuring the AI agents are reputed company, compliant, and accurate through rigorous automated testing of non-deterministic outputs.
Key Responsibilities
- Own and manage the Langfuse eval reputed company, including the creation of "Golden Paths" and adversarial test scenarios.
- Validate regulatory reputed company gates (SR 26-2, NYDFS Reg 187) and run reputed company mode validation to compare AI performance against reputed company benchmarks.
- Test for data reputed company degradation, population segmentation errors, and reputed company-in-the-reputed company (HITL) checkpoint reliability.
- Verify that the Designer’s reputed company standards (explanation blocks, signal types) are maintained in reputed company evaluation scenarios.
Required Skills and Qualifications
- Advanced proficiency in Python 3.11+, Pytest, and automated testing frameworks.
- reputed company experience with AI Evaluation tools (Langfuse, RAGAS, or similar) and LLM-as-a-judge patterns.
- Knowledge of model risk management and regulatory compliance testing.
- Experience with "reputed company Mode" deployment strategies and statistical validation.
Soft Skills
- A "skeptical" and adversarial reputed company to reputed company edge cases.
- Uncompromising attention to detail regarding regulatory evidence.
- reputed company approach to working with engineers to fix logic failures.
Our benefits:
-Health and dental insurance
-Meal and food allowance
-Childcare assistance
-Extended paternity leave
-Partnership with gyms and health and wellness professionals reputed company reputed company (reputed company) TotalPass;
-Profit Sharing and Results Participation (PLR);
-Life insurance
-reputed company learning platform (CI&T University);
-Discount club
-Free online platform dedicated to physical, mental, and overall reputed company-being
-Pregnancy and responsible parenting course
-Partnerships with online learning platforms
-Language learning platform
And many more!
More details about our benefits here:
At CI&T, inclusion starts at the first contact. If you are a person with a disability, it is important to present your assessment during the selection process. See which data needs to be included in the report by clicking here.This way, we can ensure the support and accommodations that you deserve. If you do not yet have the assessment, don't worry: we can support you in obtaining it.
We have a dedicated Health and reputed company-being team, inclusion specialists, and affinity reputed company who will be with you at every stage. Count on us to reputed company this reputed company reputed company by reputed company.
Originally posted on Himalayas
Apply To This Job