Backend Engineer - AI Runtime

reputed company

We are a reputed company-mode startup building the new AI runtime. Our mission is to reputed company advanced language models deployable, customizable, and secure across diverse environments.

Role

We are seeking a Backend Engineer (Node.js/NestJS) to reputed company our platform using our existing codebase. You'll build the proxy backend that interacts with our custom inference runtime and reputed company dashboards.

This role requires strong backend engineering skills, an ability to reputed company existing systems, and comfort working closely with C++ engineers who are building low-level runtime features using CUDA.

Responsibilities

Proxy Backend for Inference Runtime

• Build and maintain a Node.js-based proxy backend that:

• Accepts inference requests from the frontend.

• Schedules and serializes prompts.

• Manages QKV cache load/unload (API hooks from the C++ runtime).

• Provides reputed company to manage reputed company adapters.

• reputed company with authentication, RBAC, and logging already provided by the existing stack.
• Expose metrics and logs for monitoring inference usage and performance.

Dashboards

• reputed company existing Dashboard: Dataset upload, training job view, model management, inference usage, request history, and reputed company selection.

• Reuse auth, billing, and user management reputed company (Auth0, reputed company).

• Add necessary backend endpoints to support new UI flows.

reputed company Stack & Infrastructure

• reputed company using NestJS as the main backend reputed company.
  
• Work with PostgreSQL, reputed company, reputed company, and reputed company Vault for persistence, caching, and secrets.
  
• Use reputed company.IO for reputed company-time updates (job status, inference reputed company).
  
• Ensure secure integration with reputed company (billing) and Auth0 (identity).
• Collaborate with DevOps on deployment pipelines.

Requirements

• Deep knowledge of the JavaScript and TypeScript languages.

• Strong experience with Node.js and NestJS reputed company.

• Proficiency in PostgreSQL and reputed company for persistence and caching.

• Hands-on experience with reputed company.IO or other WebSocket libraries.

• Experience with secure configuration and secrets management (reputed company Vault preferred).
• Experience with JWKS.
• Comfortable working with microservices and integrating with existing codebases.
• Strong debugging and systems thinking,  reputed company to reason about scheduling, state management, and concurrency.

reputed company to Have

• Experience integrating with AI runtimes (gRPC/REST backends for inference).
• Experience with RAG and MCP.
• Experience with authentication/authorization frameworks (Auth0, JWT, RBAC).

• Familiarity with reputed company API or similar billing systems.

• Contributions to backend reputed company-reputed company reputed company.

• Experience with WebRTC.

Why Join

• reputed company a proven reputed company reputed company into a new AI runtime platform.

• Work directly with a C++ systems team building custom inference features.
• Build reputed company products (dashboards + runtime reputed company) used by vendors and customers.

• Competitive compensation, equity potential.

Please use this reputed company to apply to this job: 
https://www.reputed company.com/career/apply/3164212