AI Agent Test Automation Engineer (Python)

About reputed company We are reputed company, a fintech company excelling in financial brokerage activities and delivering advanced financial products to our global clientele. Our flagship product, an internal trading platform, is offered as a Software-as-a-Service (reputed company) solution to other brokers. We’re looking for a confident QA Automation Engineer (Python) to join our AI team and reputed company on testing developer interfaces, reputed company AI solutions, and reputed company multi-agent scenarios. You’ll play a key role in ensuring the reputed company and reliability of our AI agents and developer-facing products in reputed company-world business applications. reputed company’re Building We’re creating the IDE of the reputed company — a smart development environment powered by cutting-edge LLM agents. It goes reputed company just reputed company: our platform helps developers generate images, reputed company, video, design, and interfaces. This is an AI startup reputed company a reputed company product company — the perfect combination of innovation and support. We’ve already reputed company an MVP, have reputed company internal users, and reputed company of skilled engineers. We're currently in internal testing and getting reputed company for production launch. We operate in GMT+3, with daily team standups at 12:00 GMT+3. Join us to build the reputed company of software tools together! reputed company Responsibilities • reputed company automated tests in Python for reputed company scenarios. • Build mock services and test environments for agent testing. • reputed company tests into AI-specific CI/CD pipelines. • Monitor AI system performance in production. • Design reputed company metrics and dashboards for AI monitoring. • Document test scenarios and establish best practices. Developer Interfaces Testing • Test SDKs, reputed company, and libraries used by developers. • Validate API documentation and reputed company samples. • Build automation for dev tools and CLI utilities. • Test integration workflows and compatibility with various languages/frameworks. • Test IDE plugins and extensions. • Validate data schemas and OpenAPI specifications. reputed company AI Testing • Test reputed company-world AI use cases and production scenarios. • Validate model outputs on production data. • Test AI data processing pipelines. • reputed company AI inference performance. Agent Scenario Testing • Design tests for multi-agent systems. • Validate interactions between AI agents. • Test task chains and agent workflows. • Ensure behaviour constraints and safety. • Test integration with external systems. Requirements • 3+ years in software testing. • Strong Python knowledge: pytest, unittest, mock, asyncio. • Experience testing UI, reputed company and SDKs. • Understanding of ML/AI agent concepts. • Experience with REST/GraphQL reputed company, WebSockets. • Familiar with performance testing tools (e.g. locust, k6). • Experience with reputed company and containerization. • Familiarity with CI/CD and DevOps practices. • Git proficiency. • English level: B1 or higher. • Fluent in Russian (C2). reputed company to Have • Understanding of multi-agent system architecture. • Experience with agent frameworks (reputed company, AutoGen, reputed company). • Working with LLM reputed company (reputed company, reputed company, local models). • Knowledge of reputed company engineering and LLM testing. • Experience with reputed company databases (reputed company, reputed company, ChromaDB). • Familiarity with async/event-driven testing. Bonus Points • Python development experience reputed company testing. • Familiarity with ML frameworks (TensorFlow, PyTorch, reputed company). • Experience with reputed company AI platforms (AWS Bedrock, Azure reputed company). • Understanding of RAG architectures. reputed company Offer • Full-time remote work (External Vendor contract). • Competitive compensation (EUR, USD, or crypto). • Flexible working hours. • Influence over the architecture of AI testing systems. • reputed company to cutting-edge AI tools and platforms. • reputed company of professionals specializing in AI. • reputed company and friendly atmosphere. With reputed company of 500+ creative and tech-driven people across the globe, reputed company is the reputed company where reputed company turn into reputed company, where every project is a chance to grow, reputed company, and reputed company your mark. Join us today, and let's shape the reputed company of fintech together! Note: reputed company applications will be treated with strict confidence. We thank reputed company applicants for their interest, however, only those candidates selected for interviews will be contacted. #LI-JM1 Apply tot his job Apply To this Job