Senior LLM Engineer — Build a Private AI Assistant (RAG, FastAPI, Streamlit, ChromaDB, OpenAI)

Senior LLM Engineer Needed — Build a Private AI Assistant (RAG, FastAPI, Streamlit, ChromaDB, OpenAI) Project Overview I’m looking for a senior-level AI engineer with real experience designing and implementing LLM-powered applications, especially those involving Retrieval-Augmented Generation (RAG), vector databases, multi-prompt agent behavior, and clean production-grade Python architectures. The goal is to build my internal private AI assistant (“TomGPT”) that will run locally and serve as: •A Tax Planning Advisor • A Profitability & Business Advisory Assistant • A Content Creation Assistant for my CPA practice This project requires someone who understands how to build modular LLM systems, not someone who glues together LangChain tutorials. ________________________________________ What I Need Built A complete private AI system with: 1. Backend (FastAPI) • /chat endpoint that: o Loads mode-specific system prompts o Performs vector retrieval (Chroma) o Constructs messages for the LLM o Calls OpenAI models (GPT-4.x / 5.x class) o Returns assistant responses 2. Frontend (Streamlit) • Password-gated access • Mode selector (Tax / Profit / Content / General) • Full chat interface with history in session state • Fast, responsive UI 3. Document Knowledge Base (RAG) • Document ingestion pipeline: o PDF/DOCX text extraction o Chunking (configurable) o Embedding (OpenAI) o Storage in ChromaDB with metadata • Runtime retrieval: o Query embedding o Top-k similarity search o Automatic context injection 4. Mode-Based Agent Behavior Load prompts from external files (4 modes): • Tax Planner • Profitability Coach • Content Writer • General Advisor The backend should orchestrate prompts cleanly, not hard-code them. 5. Security & Config • Password protection for the UI • .env for secrets • No API keys exposed to frontend 6. Documentation A professional-quality README explaining: • How to run the system • How to add documents • How to create new modes • How to change models • Optional: how to run everything via Docker ________________________________________ Tech Stack Requirements Required Experience You must be strong in: • Python (senior-level) • FastAPI (production-quality routes & architecture) • Streamlit (clean user interface) • OpenAI API (chat + embeddings) • Vector DBs (Chroma, Pinecone, Qdrant, etc.) • RAG design patterns: o chunking strategies o embedding management o context window optimization o metadata filters • Prompt architecture & multi-agent patterns Strongly Preferred • Experience with Ollama or other local models • Docker • Building similar “private GPT” solutions • Understanding of tax or financial domain (not required, but helpful) Not Interested In • Beginners • People who only use LangChain without understanding what happens under the hood • No-code tools (e.g., Bubble, WordPress plugins) • “Chatbot builders” with no real backend knowledge If you cannot explain embeddings, chunking, and RAG tradeoffs clearly, please do not apply. ________________________________________ Deliverables • Fully working FastAPI backend • Fully working Streamlit frontend • Ingestion script • Vector DB setup (Chroma) • Mode-based prompt system • Clean, simple project structure (folders provided upon hire) • Excellent documentation ________________________________________ Budget & Timeline • Budget: $2,000–$3,500 (fixed price or milestone-based) • Timeline: 2–3 weeks I’m willing to pay top rate within the budget for senior talent who can build this cleanly, modularly, and efficiently. ________________________________________ To Apply (Important) Please include the following in your proposal: 1. A short summary of your experience building LLM/RAG systems. 2. One example of an LLM app you built (no NDAs needed—just describe architecture & decisions). 3. Confirmation that you are comfortable with: o FastAPI o Streamlit o Chroma or similar o RAG design 4. Your estimated timeline and approach to this project. Shortlisted candidates will be asked one technical question about embeddings and chunking to verify expertise. Apply tot his job Apply To this Job