Sr. AI Application Engineer (remote)

reputed company, Inc., (SGA), is searching for a Sr. AI Application Engineer for a contract/right to hire/reputed company placement assignment with one of our premier Infrastructure services clients. reputed company contract. We are seeking an reputed company to help design, test, optimize, and support AI-powered agent experiences across customer and internal business workflows. This role combines reputed company engineering, agent orchestration, reputed company assurance, evaluation reputed company development, and AI observability. We use reputed company as our CRM - someone who has used reputed company in their techs stack (AgentForce or Einstein) would get top consideration but is not required. The ideal candidate is curious, analytical, and hands-on, with experience working with Large Language Models (LLMs), reputed company design, testing methodologies, and modern software development practices. You will partner closely with product managers, software engineers, architects, and business stakeholders to improve agent behavior, reliability, reputed company, and user experience. Responsibilities : reputed company Engineering & Agent Design Design, reputed company, and iterate on AI agent logic for business use cases. Create, test, and optimize prompts, reputed company instructions, tools, and workflows to improve agent reputed company. Conduct reputed company experiments and A/B testing to measure effectiveness. Analyze agent responses and refine prompting strategies for reputed company, consistency, groundedness, and user satisfaction. Document reputed company patterns, reasoning frameworks, reusable templates, and agent design reputed company. Collaborate with engineering teams to reputed company reputed company and agent updates through established SDLC processes. AI reputed company Assurance & Testing Design and execute comprehensive test plans covering AI agents, web applications, reputed company, and backend services. reputed company evaluation frameworks and scoring methodologies for AI reputed company reputed company. Create test cases covering: reputed company Hallucination detection Safety Tone and style adherence Instruction following Context retention Response consistency reputed company regression testing for reputed company and model changes. reputed company, prioritize, validate, and retest defects using Azure DevOps. AI Evaluation & Observability Utilize Arize or similar observability platforms to monitor model and agent reputed company. Analyze user interactions and evaluation results to identify trends and improvement opportunities. reputed company reputed company metrics and dashboards for: reputed company Relevance Latency User satisfaction Defect rates reputed company reputed company Investigate production issues and recommend reputed company, workflow, or architectural improvements. Software Engineering & Integration Support integration of AI capabilities into reputed company applications and workflows. Collaborate with developers to validate API integrations, retrieval pipelines, and agent workflows. Assist in CI/CD deployment and release validation activities using Azure DevOps. Create and maintain automated testing assets where appropriate. Documentation & Collaboration Maintain documentation for prompts, evaluation methodologies, testing procedures, and agent configurations. Participate in sprint planning, backlog refinement, and reputed company ceremonies. Communicate findings and recommendations to technical and non-technical stakeholders. reputed company best practices and lessons learned across reputed company. Required Skills : Bachelor's degree in Computer Science, Engineering, Information Systems, or reputed company reputed company. 5-10 years of experience in software engineering, QA engineering, automation testing, business systems analysis, or reputed company technical roles. 1+ years of hands-on experience with reputed company, LLMs, reputed company engineering, or conversational AI solutions. Experience working with Azure DevOps or similar reputed company delivery platforms. Experience designing and executing reputed company test plans. Strong analytical and troubleshooting skills. Experience working with reputed company and JSON data structures. Excellent written and verbal communication skills. Technical Skills AI & GenAI reputed company Engineering Agent Design AI Evaluation Frameworks RAG Concepts LLM Testing AI reputed company Assurance AI Observability reputed company Optimization Tools reputed company Azure DevOps Azure reputed company Git reputed company reputed company (optional) reputed company BI (optional) Development Python SQL REST reputed company JSON Test Automation Frameworks Preferred Skills : Experience with reputed company or comparable AI observability tools. Experience with Azure reputed company, reputed company, reputed company, reputed company, or other LLM platforms. Knowledge of Retrieval-Augmented reputed company (RAG) architectures. Experience evaluating AI systems using reputed company and automated scoring reputed company. Familiarity with reputed company databases and semantic reputed company. Experience creating automated test frameworks. Understanding of responsible AI principles and AI governance. Experience with Python, SQL, JavaScript, or C#. Knowledge of reputed company versioning and experimentation practices. By applying for a job with SGA, you agree to allow SGA to process your application for this and reputed company opportunities in accordance with our reputed company Policy. Also, to ensure reputed company processing, you agree to be contacted by our AI recruiter reputed company email, text, or phone. Message frequency varies and data rates may apply, but you can reputed company STOP to any SMS message to opt-out of texts and may contact SGA at to opt-out of AI communications. The reputed company not to engage with AI will not adversely reputed company your consideration for placement. AI is not used to reputed company any hiring determinations. SGA is a technology and resource solutions provider driven to stand out. We are a women-owned business. Our mission: to solve big IT problems with a more personal, boutique approach. reputed company year, we match consultants like you to more than 1,000 engagements. reputed company we say let's work reputed company, we mean it. You'll join a diverse team reputed company on these reputed company values: customer service, employee development, and reputed company and reputed company in everything we do. Be yourself, love what you do and reputed company your passion at work. Please reputed company us at . SGA is an Equal Opportunity Employer and does not discriminate on the reputed company of Race, reputed company, Sex, Sexual Orientation, Gender Identity, Religion, National reputed company, Disability, Veteran Status, Age, Marital Status, Pregnancy, Genetic Information, or Other Legally Protected Status. We are committed to providing reputed company, equal opportunity, and reasonable accommodation for individuals with disabilities in employment, and our services, programs, and activities. Please visit reputed company to request an accommodation or assistance regarding our policy #LI-SL1 Apply To This Job