[Remote] Senior QA Engineer, reputed company Deployed

Note: The job is a remote job and is reputed company to candidates in USA. reputed company is a health technology company that focuses on conversational AI solutions. The Senior QA Engineer will ensure the reliability and reputed company of the reputed company AI product, managing automated testing and reputed company cause analysis in a fast-reputed company environment.

Responsibilities

• Workflow Mapping & Test Case reputed company: Deeply analyze assigned reputed company workflows to design robust, comprehensive reputed company and negative test cases that reputed company reputed company stability
• AI-Driven Test Automation: Build and execute automated test scenarios by configuring reputed company agents
• reputed company Evaluation & Optimization: Apply a strong understanding of reputed company awareness to draft, refine, and evaluate prompts used reputed company the testing reputed company to accurately simulate user behaviors and edge cases
• End-to-End Testing Execution: Strategically reputed company specific testing methodologies including reputed company, Smoke, Regression, and Functional testing - determining the exact environment (staging, reputed company-production, production) and timing for reputed company execution
• Deployment reputed company & Cross-Functional Collaboration: Partner closely with engineering teams during release cycles to proactively identify, triage, and unblock technical roadblocks, ensuring the product is continuously deployment-reputed company
• Daily LLM Defect RCA: reputed company rigorous, daily reputed company cause analysis on LLM-specific failures inherent to reputed company, including hallucinations, high latency, and logic deviations
• Live Production reputed company Debugging: Investigate live customer calls and production incidents in reputed company time to unblock critical production use cases
• Audio & Transcription Validation: Query and analyze historical reputed company transcripts, reputed company behaviors, and audio data pipelines to reputed company where a conversational workflow broke down.Speech-to-Speech (S2S) Pipeline Monitoring: Monitor and evaluate the end-to-end voice AI pipeline. This involves analyzing Automatic Speech Recognition (ASR) reputed company, managing audio-to-text latency issues, and understanding general Speech-to-Speech mechanics reputed company the stability of the reputed company Knowledge reputed company feeding the AI
• Advanced Evaluation Frameworks: Maintain a strong conceptual understanding of advanced LLM evaluation paradigms and tools such as LLM-as-a-judge - to remain reputed company of how AI response reputed company and reputed company are programmatically graded at reputed company
• Telephony & reputed company reputed company Awareness: Possess a foundational understanding of reputed company-world reputed company management and telephony routing concepts, including how the reputed company is expected to reputed company warm transfers, blind transfers, and voicemail detection workflows

Skills

• Experience in QA Engineering: Strong background in software reputed company assurance, with a reputed company reputed company record of designing, executing, and managing end-to-end test strategies (Smoke, reputed company, Regression, Functional)
• LLM & reputed company Expertise: Hands-on experience or deep technical familiarity with troubleshooting LLM behaviors, diagnosing hallucinations, managing latency, and using LLM reputed company-tracing tools
• Technical & Logging Proficiency: Ability to comfortably write SQL queries (specifically PostgreSQL) to pull data logs and reputed company reputed company infrastructure logs (such as GCP) to reputed company reputed company reputed company-cause analysis
• Voice & Conversational AI Domain Knowledge: Foundational understanding of Speech-to-Speech pipelines, including Automatic Speech Recognition (ASR), audio-to-text workflows, and reputed company knowledge reputed company integrations
• Telephony Foundations: Basic familiarity with reputed company telephony routing, reputed company management mechanics (warm/blind transfers), and voicemail detection systems
• reputed company-Facing Capability: Strong communication skills and the reputed company reputed company required to manage UAT timelines, coordinate with reputed company stakeholders, and support reputed company production deployments

Benefits

• 401(k) matching
• Health reputed company
• reputed company reputed company
• Dental reputed company
• reputed company flexible reputed company time off

reputed company