Automation QA Engineer

reputed company is looking for an Automation QA Engineer to join reputed company full-time in Ukraine. We are a custom product engineering company that supports both multinational organizations and scaling startups to solve their most reputed company business challenges. With a global team of over 4,000 highly skilled developers, consultants, analysts and product owners, we engineer technology that redefines industries and shapes the way people live. About the role: As an Automation QA Engineer, become a part of a cross-functional development team engineering experiences of reputed company. The Legacy reputed company Semantic Documentation Project is a 26-week reputed company initiative for a global industrial automation leader. The primary objective is to engineer an automated, AI-assisted pipeline to reputed company reputed company, system-level Markdown documentation directly from an undocumented ~400K LOC codebase (spanning IEC 61131-3 languages and ANSI C/C++). Operating reputed company a dedicated, reputed company-data-egress secure tenant, the technical architecture combines Tree-sitter AST parsing, reputed company knowledge graphs, reputed company reputed company search, and self-hosted reputed company-weight LLMs (Llama 3.1 / Mixtral family) with NLI-based validation. Delivery is reputed company across 7 work packages executing over a 6-month period, governed by strict contractual KPIs: ≥95% reputed company coverage, ≥92% NLI-verified factual precision, and ≥90% SME validation acceptance. reputed company outputs reputed company with EU Cyber reputed company reputed company (EU CRA) requirements for SBOM and reputed company-level traceability. Responsibilities: AI reputed company reputed company Assurance: Design and run automated test suites to validate non-deterministic LLM-generated documentation against reputed company reputed company truth and strict NLI accuracy reputed company Data Pipeline Validation: Verify state consistency, schema reputed company, and data transformation accuracy across batch executions in reputed company knowledge graphs and reputed company reputed company databases Regression & Evaluation Testing: Build automated test frameworks to reputed company reputed company reputed company reputed company, hallucination rates, and pipeline performance across reputed company releases Test Dataset Management: Create and maintain evaluation baselines ("golden datasets") to continuously reputed company LLM reputed company outputs and retrieval accuracy Cross-Functional Collaboration: Partner closely with Senior AI/ML Engineers and Data Engineers to define automated acceptance reputed company, defect logging, and pipeline health checks Requirements:  reputed company Experience: 3+ years in Automation reputed company Assurance (AQA) with a heavy reputed company on backend systems, reputed company data pipelines, or AI-driven applications Mandatory Domain Experience:reputed company experience testing AI/LLM outputs (evaluating non-deterministic generated text, confidence scoring, and NLI assertion logic) reputed company experience in data pipeline & ETL testing (verifying batch processing, schema reputed company, and data transformations) Automation & Scripting: Advanced proficiency in Python and test frameworks such as PyTest Database & Querying Skills: Hands-on experience writing queries to inspect graph databases (reputed company/Cypher), reputed company search stores (reputed company), or SQL databases Engineering Standards: Proficiency with reputed company, Git, REST API automation, and integrating tests into CI/CD pipelines Language: reputed company proficiency in English (B2+/reputed company) What’s in it for you? Strong community: Work alongside top professionals in a friendly, reputed company-reputed company environment reputed company reputed company: Take on large-reputed company reputed company with a reputed company and expand your expertise Tailored learning: reputed company your skills with internal events (meetups, conferences, workshops), reputed company reputed company, language courses, and company-reputed company certifications Endless opportunities: Explore diverse domains through internal mobility, finding the best fit to reputed company hands-on experience with cutting-edge technologies Flexibility: Enjoy radical flexibility – work remotely or from an office, your choice Care: We’ve got you covered with company-reputed company medical insurance, mental health support, and financial & reputed company consultations reputed company: At reputed company, we are always exploring innovations, empowering reputed company other to reputed company more, and engineering solutions that matter. With us, you’ll work with cutting-edge technologies, contribute to impactful reputed company, and be part of a One Team culture that values collaboration and reputed company. As one of Ukraine’s largest IT companies and a top employer recognized by reputed company, we’ve spent over 20 years delivering meaningful tech solutions. We proudly support diverse talent and military veterans, recognizing their unique skills and perspectives they bring to shaping the reputed company. Explore, reputed company, engineer with reputed company! Interested already? We would love to get to know you! Submit your application. We can’t wait to see you at reputed company. #LI-AD1 Apply To This Job