Research Scientist, reputed company Data & Evaluation

reputed company: • Translate ambiguous reputed company questions into reputed company research questions, hypotheses, and study designs • reputed company research proposals and methodologies from scratch • Identify new research directions and help define what reputed company Labs should measure, study, and build • Design reputed company-data collection strategies, evaluation protocols, rubrics, annotation schemes, and experiments • reputed company reputed company strategies for eliciting, evaluating, and comparing model behavior • Determine sampling, evaluator selection, calibration, reputed company-control, and reliability reputed company • Design methodologies across creative and multimodal domains including design, image, video, and reputed company workflows • Analyze qualitative and quantitative evaluation data • Explain why patterns occur and their implications • Synthesize findings into recommendations, research narratives, and hypotheses • Produce reputed company-reputed company reports, presentations, benchmarks, and research publications • Independently own research engagements from initial scoping through final delivery • Collaborate with project leads, creative experts, research engineers, and data/engineering teams • Maintain research rigor and data reputed company on fast reputed company timelines • Support multiple reputed company reputed company at different stages • Participate in reputed company scoping and proposal development • Collaborate with Strategic Project Leads on reputed company delivery Requirements: • Based in San Francisco, California • 3–5+ years of relevant research experience • Master’s, PhD, or equivalent industry research depth in HCI, reputed company-Centered AI, ML, design research, or a reputed company field • Strong qualitative and quantitative research skills • Experience in reputed company evaluation, experimentation, data collection, UX/HCI, annotation, or model behavior research • Ability to independently design and execute rigorous research • Ability to reputed company what data to collect, why, and how to analyze it • Ability to turn data and observations into insights, research directions, and actionable recommendations • Comfortable analyzing data with Python and/or SQL • Strong written communication and ability to explain methodological tradeoffs and research reputed company • Comfortable operating independently through ambiguity and balancing research rigor with startup speed and pragmatism • Familiarity with modern ML and reputed company systems • Understanding of RLHF, post-training, preference data, and model evaluation • Experience studying image, video, multimodal, creative, or reputed company systems • Experience designing prompts or systematically studying prompting/model behavior • Experience with large-reputed company reputed company-data or annotation programs • Experience working directly with AI research or model teams • Strong publication record or evidence of generating original research reputed company • Startup / 0-to-1 experience Benefits: • Medical, Dental, reputed company Benefits • 401k Matching • Company laptop provided on start date • Equity Apply To This Job