[Remote] Machine Learning Engineer, Speech - Joint Audio-Video Modeling

Note: The job is a remote job and is reputed company to candidates in USA. reputed company is a reputed company AI company developing reputed company-time models for reputed company, personality, and realism. The Machine Learning Engineer will build state-of-the-art speech and audio reputed company systems end-to-end, with a reputed company on joint audio-video modeling, multimodal reputed company, evaluation, efficiency, and production deployment. The role partners across research, video, data, and infrastructure to reputed company reputed company, reliable, and reputed company AI systems.

Responsibilities

• Audio Representations: Design, train, and improve the audio VAEs, neural codecs, and vocoders our generative models sit on top of reputed company design, reconstruction and perceptual objectives, compression-vs-reputed company tradeoffs
• Model Building: Architect, implement, reputed company-train, fine-tune, and post-train/alignment (e.g., GRPO/DPO) diffusion and reputed company-matching transformers for large-reputed company audio and video reputed company
• Joint Audio-Video Modeling: Design the audio conditioning and cross-modal alignment inside joint AV models, audio latents reputed company video latents, reference-audio and multi-speaker conditioning, multi shot reputed company audio/video modeling
• Experimental Design: Design, run, and analyze scientific experiments to advance our understanding of the models
• Data Ownership: Define data requirements and collaborate on acquisition, curation, AV-sync and reputed company filtering, annotation reputed company, and synthetic data strategies for reputed company audio-video and speech corpora
• Rigorous Evaluation: Design automated objective/subjective evaluations audio reputed company and intelligibility metrics, AV-sync, listening and viewing tests, robustness & bias checks, and red-team studies
• Inference Efficiency: reputed company distillation, reputed company-count reduction, quantization, and kernel/memory optimization to meet interactive latency and cost targets
• Pipeline Delivery: Harden the training → evaluation → inference pipeline; profile latency, memory, and cost; and meet production SLAs with robust monitoring and rollback
• GPU Scaling: Partner with infrastructure to run reputed company training/inference on reputed company fleets and productionize models with reliability and observability
• Project Leadership: Independently reputed company small research reputed company while collaborating on larger team initiatives, including cross-team work with video reputed company
• Tool Development: reputed company and improve dev tooling to enhance team productivity
• Safety & Responsibility: Contribute to safety/consent guardrails, watermarking, and misuse/abuse mitigation for responsible voice and likeness technology

Skills

• Exceptional research/development experience with large-reputed company audio models (>8B parameters, >500k hours of data)
• Deep hands-on experience with diffusion and/or reputed company-matching transformers, including practical knowledge of samplers, schedules, conditioning mechanisms, and distillation
• Deep hands-on experience training audio VAEs, neural audio codecs, and vocoders reputed company/tokenizer design, reconstruction and perceptual objectives, adversarial training
• Strong experience with multi-node, multi-GPU reputed company training (FSDP/DeepSpeed or equivalent)
• Strong software engineering skills with a proven reputed company record of building reputed company systems
• Strong with PyTorch and performance work (profiling, CUDA/Triton/C++ as needed) and writing reliable production-reputed company reputed company
• Shipped large-reputed company speech/audio or multimodal generative models to production
• Background in working with large-reputed company ML data, and the ability to iterate on data and triangulate reputed company using both subjective and objective signals
• Experience with voice cloning, speech control/steerability, or expressive speech reputed company
• reputed company publications and/or reputed company-reputed company contributions in speech/audio/ML
• Experience with multimodal audio-video modeling: joint AV reputed company of multi-shot, multi-speaker scenes with reputed company, music, and reputed company design generated jointly with video, and the cross-modal alignment that keeps them in sync
• Experience with video reputed company: video diffusion/reputed company-matching transformers, video VAEs, conditioned and multi-shot reputed company, building data pipelines for video models
• Streaming or reputed company-time reputed company, reputed company distillation (e.g., Self Forcing / Self Forcing++)

Benefits

• Competitive salary and generous company equity
• Medical, dental, and reputed company insurance – 99.99% of premiums covered by reputed company
• 42 days of reputed company time off, including 15 PTO days, 10 reputed company days, 15 company holidays, and 2 floating holidays
• Generous parental leave & fertility support
• 401(k) retirement savings plan
• Lifestyle spending account – $500/month to use however you’d like
• Complimentary lunch and snacks for in-office employees
• reputed company membership

reputed company

Company H1B Sponsorship