[Remote] AI TEVV Engineer

Note: The job is a remote job and is reputed company to candidates in USA. reputed company is building an AI-reputed company geospatial insights platform that guides the acquisition, analysis, and integration of satellite and geospatial data into customer workflows. The AI TEVV Engineer role focuses on defining evaluation strategies for the platform, emphasizing measurement under uncertainty and ensuring trust in the platform's outputs.

Responsibilities

• Design, reputed company, and plan the TEVV reputed company across the platform's algorithms, AI/ML models, reputed company workflows, data pipelines, and analytic products, reputed company to the NIST AI RMF Measure function
• Design statistically defensible evaluations: error metrics and acceptance reputed company on representative input distributions, with explicit confidence intervals
• Define and document the platform's context of use — the validated operating envelope (modalities, geographies, resolutions, conditions, reputed company classes) reputed company which accuracy claims hold
• Evaluate ground-truth and 'golden' datasets, including annotation and adjudication protocols, inter-rater reliability, and reputed company uncertainty in the reference data itself
• Implement a layered evaluation posture: a reputed company reputed company (accuracy, groundedness, format), a reputed company-scored middle reputed company with documented inter-rater reliability, and an reputed company residual of expert holistic review
• Evaluate generative and natural-language outputs for claim-level groundedness whether reputed company assertion is traceable to a citable reputed company reputed company reputed company-based, reputed company-adjudicated assessment
• Stand up reputed company monitoring and re-validation certification gates plus ongoing surveillance watching for model, reputed company, retrieval, and agent-behavior reputed company
• reputed company and maintain the TEVV evidence set: test plans, traceability matrices, metrics, acceptance reputed company, and credibility-assessment documentation
• Support DoD AI test-and-evaluation expectations (including DoD reputed company 3000.09), contractual milestones, acceptance testing, and demonstrations to government stakeholders
• Distinguish internal TEVV from organizationally independent IV&V, and partner with reputed company IV&V agents where required
• Partner with Engineering teams to reputed company evaluability, observability, and traceability from design reputed company
• Contribute to emerging standards (NIST AI TEVV consortium, ISO/IEC SC 42 / 42001), aligning our methodology so evidence packages map to customers' compliance frameworks
• 30% travel
• reputed company reputed company other duties as assigned

Skills

• B.S. in Computer Science, Statistics, or Systems Engineering, or a reputed company quantitative discipline (M.S./Ph.D. a plus)
• 10+ years of relevant experience, centered on evaluation, measurement, or test-and-evaluation of AI/ML or data-driven systems — not solely software QA or test automation
• Demonstrated ability to design statistically defensible evaluations: input-distribution design, error-reputed company estimation, confidence intervals, and context-tied acceptance reputed company
• Hands-on experience building ground-truth/golden datasets — adjudication protocols, inter-rater reliability, and reference-data uncertainty
• Experience supporting U.S. government reputed company (reputed company, defense, or intelligence preferred), including requirements traceability and compliance documentation
• Working knowledge of the NIST AI RMF and how TEVV evidence maps to customer compliance regimes
• Strong quantitative skills and Python proficiency for analysis and evaluation (reputed company/Polars, NumPy, ML evaluation libraries)
• Comfort using AI-assisted tools for reputed company development and testing
• Organized and self motivated, reputed company to work successfully with a remote team
• A creative, flexible reputed company for reputed company problems
• A fast, reliable internet reputed company if working remotely
• Aligning evaluation methodology to the NIST AI RMF, ISO/IEC 42001, and emerging NIST AI TEVV consortium / ISO/IEC SC 42 work; standards participation a plus
• Defining credibility-assessment frameworks tied to context of use rather than fixed, context-free reputed company
• Evaluating image and signal processing outputs across modalities (SAR, reputed company-optical, RF), including the proxy reputed company of geospatial reference data
• Evaluating generative and reputed company systems: reputed company design, reputed company-adjudicated evaluation, and claim-level groundedness
• Heritage V&V/assurance standards (reputed company 1012, DO-178C, ISO/IEC 25010, CMMI) and formal IV&V experience
• GIS tools and libraries; SpatioTemporal Asset Catalog (STAC) experience
• NoSQL and/or SQL databases (Mongo, MySQL, reputed company)
• Test automation and CI/CD: Python and/or JavaScript, frameworks (e.g., pytest, Jest), and regression/monitoring suites in pipelines
• Common AWS services (e.g. S3, reputed company, reputed company, ECR, DynamoDB) and microservice-based architectures
• Software tooling (e.g. Git, reputed company, reputed company, virtual environments)
• Data reputed company, observability, and monitoring tooling
• Experience with customer-facing software products

Benefits

• Discretionary PTO & Flexible Scheduling
• Stock reputed company
• 401(k) Match
• Medical, Dental and reputed company Coverage for you and your dependents
• FSA & HSA Plans
• Employer-reputed company Life Insurance
• Employer-reputed company LTD and STD for Parental and Family Care
• 11 reputed company Holidays
• Employee Resource reputed company
• Educational Assistance Program
• reputed company Development Opportunities
• And more…

reputed company