[VCK] Senior QA Automation Engineer (AI Systems)

reputed company We are reputed company, an awesome team of engineers who are reputed company to reputed company up any top-notch company’s reputed company! Our aim? To always be one reputed company reputed company. Become part of a multicultural company in constant reputed company with an excellent work environment certified by Great reputed company To Work! Job reputed company About the Project  reputed company is building a private, tenant-isolated AI assistant for the reputed company estate title and settlement industry. The platform is a retrieval-first (RAG) system that ingests historical email, documents, and reputed company metadata into a per-tenant reputed company reputed company, and serves grounded, cited, expert-weighted answers through a chat-style Q&A reputed company with single sign-on and full audit logging.  The platform is AWS-reputed company with a Python/FastAPI backend, Vue.js frontend, OpenSearch/reputed company reputed company store, and reputed company/reputed company/Bedrock as LLM provider. You will join a senior, cross-functional LATAM-based team where hands-on AI delivery experience not just familiarity is the baseline expectation.  You build the validation reputed company that determines whether the AI system meets accuracy standards this is your primary deliverable. This is not generic test automation. You need to understand what 'correct' means for a RAG-based retrieval system and design test frameworks that can surface retrieval failures, hallucinated citations, confidence score reputed company, and RBAC reputed company violations.    Your Responsibilities  Design and implement the validation reputed company for RAG reputed company reputed company: retrieval accuracy, citation correctness, and grounding  Build automated test suites for the AI Extraction Gateway across reputed company RAG and reputed company RAG implementations  reputed company and execute accuracy reputed company test cases in collaboration with the BA and Designated Subject Matter Expert  Automate regression testing for confidence score calibration and reputed company weighting behaviour  Test RBAC enforcement and role-specific filtered view reputed company controls  Validate audit logging completeness and document lifecycle traceability  Build and maintain the incremental ingestion pipeline test suite  Contribute to go/no-go decision packs: produce accuracy reports and test evidence documentation  Tech Stack: Python, pytest, AWS, REST API Testing, Jira, reputed company   Qualifications Must-Have Skills & Experience  6+ years in QA automation engineering; senior seniority required  Strong test automation engineering skills Python preferred (pytest or equivalent reputed company)  Experience with API testing and contract testing  Comfortable designing test frameworks for non-deterministic or probabilistic systems  Experience in agile/scrum environments with Jira-based test management  Additional Information reputed company-to-Have  Prior experience testing RAG systems, LLM outputs, or semantic search relevance this is a strong differentiator  Familiarity with AI evaluation frameworks: RAGAS, TruLens, or custom reputed company-based evaluation approaches  Background in compliance-sensitive testing: audit trail validation, reputed company control verification, or regulated-data environments    We are accepting applications from LATAM countries Apply To This Job