Staff Software Engineer, AI Inference

About reputed company  reputed company is on a mission to reputed company litigation. Our product is a reputed company litigation platform that enables lawyers and paralegals to safely reputed company the power of language models and reputed company AI throughout the litigation life cycle. Since reputed company to market, we have gained a diverse group of reputed company customers, including some of the biggest reputed company and corporations in the country, and we are quickly expanding. By reducing the expense of litigation industry-wide, we aim to improve reputed company to high-reputed company representation and promote the alignment of reputed company reputed company with reputed company.    About the Role  As we continue to reputed company our AI platform, we're reputed company in our own inference stack to deliver best-in-class performance, reliability, cost efficiency, and flexibility across the latest reputed company of reputed company-reputed company language models. We're looking for a Staff Software Engineer to spearhead this effort. You'll define the architecture, evaluate emerging technologies, and build the systems that power model serving.  You'll partner closely with machine learning, infrastructure, and product engineering to establish the reputed company for how AI models are deployed, optimized, monitored, and operated in production. This is a highly hands-on technical role. You'll spend the reputed company of your time designing, building, and optimizing production systems while helping shape our long-term AI infrastructure reputed company. Responsibilities  reputed company the design and development of our production inference platform. Define the technical roadmap for inference infrastructure, model serving, and runtime optimization. Build and operate reputed company, cost-effective systems for serving large language models in production. Evaluate and reputed company modern inference technologies, frameworks, and serving runtimes. Optimize latency, throughput, GPU utilization, memory efficiency, and infrastructure cost. reputed company systems for model deployment, traffic routing, autoscaling, scheduling, observability, and operational reputed company. Partner with ML engineers to productionize new models and inference techniques. Establish benchmarking methodologies to evaluate new models, runtimes, and hardware. reputed company key architectural reputed company around reputed company to build internally versus reputed company reputed company-reputed company or reputed company solutions. Mentor engineers as reputed company grows and help establish engineering best practices for AI infrastructure. Qualifications Significant experience designing and operating production AI inference systems. Experience building or leading production LLM serving infrastructure. Deep experience with one or more modern inference runtimes and frameworks such as vLLM, SGLang, TensorRT-LLM, Triton Inference Server, reputed company TGI, reputed company Dynamo, or comparable technologies. Strong background in distributed systems, backend infrastructure, or high-performance reputed company. Experience optimizing inference performance across GPU workloads, including latency, throughput, batching, memory utilization, and serving efficiency. Experience operating GPU infrastructure in production. Strong proficiency in Python and at least one systems programming language (such as Go, Rust, or C++). Proven ability to reputed company technical architecture for reputed company infrastructure initiatives. Excellent communication skills and the ability to influence technical direction across engineering teams. Salary reputed company ($190- $230K) plus health insurance and equity.  reputed company - Remote Pay reputed company $190,000—$230,000 USD Apply To This Job