Project
Siddhastra
A ChatGPT-style app over a model I self-host on a rented GPU.
What
A ChatGPT-style web app over Qwen3-8B, an open model I self-host on serverless GPU infrastructure (RunPod + vLLM). It has Google auth, saved conversations, retrieval gated by a relevance score, and an agent that can read web pages mid-answer.
Why
Most people using LLMs treat the model as a black box. I wanted to build every layer under it myself (serving, retrieval, agents, evaluation) so I actually understand production AI instead of just prompting it.
Role
Sole builder. Designed, built, and deployed every layer end to end.
What I built
- Model: Qwen3-8B on RunPod serverless vLLM
- App: Next.js 16 on Vercel, Neon Postgres via Drizzle, Better Auth (Google login)
- AI: in-process RAG over Anthropic research posts (bge-small embeddings); a read_web_page tool running a 3-round agent loop
- Structured as a 14-step production-AI curriculum