# Shaurya Gulati > AI engineer working on agents, retrieval, and evaluation. Carnegie Mellon MISM-BIDA graduate ('25), most recently Founding AI Engineer at Lunon, where I built the multi-agent pipeline and analyst copilot behind automated commercial due diligence. I care about systems that cite their sources, tests that catch agents misbehaving, and products that earn their place in a workflow. Canonical site: https://shauryagulati.dev Contact: i.shauryagulati@gmail.com · GitHub: https://github.com/Shauryagulati · LinkedIn: https://www.linkedin.com/in/shauryagulati/ --- # About Shaurya Gulati I'm Shaurya, an AI engineer working on agents, retrieval, and the unglamorous machinery that makes them trustworthy: evals, citations, observability, and the failure paths nobody designs until production forces them to. I finished my Master's at Carnegie Mellon (Information Systems Management, Business Intelligence & Data Analytics, Dec '25). While there I built hybrid RAG systems over regulatory text as a research assistant and TA'd the graduate course on operationalizing AI. From there I went straight into the deep end as Founding AI Engineer at Lunon, an AI-native company automating commercial due diligence for private equity. I built the multi-agent pipeline with its quality gates, a 20-tool analyst copilot, and the observability to know what all of it was doing. The through-line in my work is verification. Retrieval systems that cite the page they decided from. Extraction pipelines with per-field precision tracked across prompt changes. An open-source framework, rag-verdict, that tests what agents do instead of averaging how their answers score. Systems that think are only useful if you can check their work. This site is a working example of all of it. The terminal is real. Type shaurya into it and you're talking to an agent that has read everything here, cites its sources, and was designed to fail gracefully. That's the job, in miniature. --- # Suture An AI-native operations layer for a cardiology practice: the full referral-to-outreach loop, built to be audited. stack: Python, FastAPI, PostgreSQL, pgvector, LLM extraction, Hybrid RAG ## Problem A cardiology practice runs on paper-shaped work. Referrals arrive as faxes and PDFs, prior authorizations mean reading payer policy documents, and patient outreach is manual phone-tag. Automating any of it with LLMs raises the stakes instead of lowering them: a hallucinated field in a referral or a leaked record is a patient-safety and compliance incident, not a bug. The system had to be useful and provably careful at the same time. ## Approach Suture handles the full loop of intake, review, prior-auth, and outreach, with a human kept deliberately in it. Documents flow through an LLM extraction pipeline backed by a real eval harness: per-field precision and recall, tracked across every prompt change, so quality regressions get caught before they ship. Prior authorization runs on a 3-stage hybrid-RAG engine (structured rules plus semantic retrieval) over payer policy documents. Extracted data goes to human review before anything acts on it, and outreach goes out multi-channel only after approval. The security posture is fail-closed: tenant isolation enforced at the ORM layer, where a missing tenant context raises and cross-tenant reads return 404, plus field-level PHI encryption and a PHI-safe audit log. ## Outcome A working AI platform for a domain where mostly right is not a passing grade. Extraction with measured per-field accuracy, retrieval that cites the payer policy it decided from, and an architecture where the dangerous failure modes are structurally impossible instead of merely unlikely. repo: https://github.com/Shauryagulati/suture-ai --- # rag-verdict Pytest for RAG agents: an open-source framework that tests what your agent does, not just how its answers score. stack: Python, pytest, LLM-as-judge, CI, Open source (MIT) ## Problem RAG evaluation tools like RAGAS and DeepEval mostly average quality scores over answers. That misses the failures that actually embarrass you in production: the retrieval tool that silently never fired, the citation pointing at a document that does not exist, the agent that confidently answers questions its corpus knows nothing about. Scores don't catch behavior. Tests do. ## Approach rag-verdict treats a RAG agent like code under test. Behavioral probes check whether tools fire when they should, whether every citation resolves to a real document, and whether the agent refuses out-of-corpus questions instead of improvising. A pluggable adapter (Python or HTTP) connects any RAG system in any language. An LLM-as-judge layer with structured output handles the fuzzy cases, and it degrades gracefully when no API key is present, so the deterministic checks still run in CI. Results come back as PASS, FAIL, or WEAK verdicts rather than a decimal nobody can act on. ## Outcome An MIT-licensed framework that turns 'our agent seems fine' into a test suite with verdicts. CI-friendly, language-agnostic, and opinionated about what matters. It's the distilled lesson of every RAG system I've built: retrieve before you generate, cite what you retrieved, and test the behavior, not the vibes. repo: https://github.com/Shauryagulati/ragverdict --- # Regulatory RAG at CMU A hybrid retrieval system over 1,000+ regulatory documents. 94% accuracy with grounded citations. stack: E5 embeddings, ChromaDB, BM25, BGE reranking, LangChain, RAGAS ## Problem Policy and corporate stakeholders navigating regulation across U.S. states and Europe face thousands of pages of interlinked legal text, and legal text is exactly where LLM hallucination costs the most. As an AI Research Assistant at Carnegie Mellon, my brief was a system whose answers could be trusted enough to act on: grounded, cited, and measured. ## Approach The retrieval core is hybrid. Dense E5 embeddings in ChromaDB fused with sparse BM25, then re-ranked with a BGE cross-encoder, because regulatory queries mix precise statutory language with fuzzy intent and neither dense nor sparse retrieval wins alone. Feeding it is a PDF ingestion pipeline with page parsing, header detection, and overlapping chunking, with data-lineage tracking so every vector traces back to its exact source page. The project grew out of my earlier multi-agent work, a Planner-Retriever-Synthesizer-Reviewer pipeline evaluated with RAGAS at 0.78 faithfulness and 0.74 answer relevancy, which taught me where agent complexity pays off and where retrieval quality is the whole game. ## Outcome Evaluated across 50+ query sets: 94% accuracy with grounded citations over 1,000+ regulatory documents. The deeper outcome is a working philosophy: separate the roles, verify the output, track lineage from answer back to page. I've carried it into every retrieval system since, including the one inside this site's terminal. repo: https://github.com/Shauryagulati/REG-RAG --- # This Site, and the Agent Inside It The portfolio is its own case study: a terminal with a RAG agent over everything on it, built to run at $0. stack: React Router, React Three Fiber, TypeScript, Cloudflare Workers AI, RAG ## Problem A portfolio that claims AI engineering should have to prove it on the page. Most sites settle for a chat widget bolted into a corner: no context, no character, and it breaks the moment an API key expires or a free tier runs dry. The bar here was a working AI product with real constraints. Zero monthly cost, no broken states, and an interface that says something about how I think. ## Approach Everything starts from one content layer. The same typed modules generate the pages, a virtual file system for the built-in terminal, the agent's retrieval corpus, and an llms.txt for AI crawlers. One source of truth, four surfaces. The terminal is summonable from anywhere as a floating window, and typing shaurya boots an agent REPL the way a CLI tool would. Retrieval is deliberately lexical, since tf-idf over a small corpus beats an embedding pipeline at this scale, and generation runs on Cloudflare Workers AI's free tier behind a rate-limited function. If the model is unreachable, the agent falls back to retrieval-composed answers instead of failing. The demo can't die. The hero object is the same philosophy applied to 3D: a retro CRT rasterized to live text every frame, made of the same material as the terminal. ## Outcome A fully static site with one serverless function, streaming cited answers, and no failure mode a visitor can see, at $0/month. Every architectural decision is inspectable; the site you are reading is the system diagram. It encodes my working rules for agent products: retrieve before you generate, cite what you retrieved, and design the degraded path before the happy one. --- # Eyeline A free, open-source macOS teleprompter that docks under the notch. Read your script while holding eye contact. stack: Swift, macOS, Homebrew, Open source ## Problem Every video call with a script has the same tell: eyes drifting to a second window. Teleprompter apps exist, but they're paid, cloud-connected, or float awkwardly over your content. The fix wanted to be invisible, literally at the camera. ## Approach Eyeline docks the script directly under the MacBook notch, the closest pixels to the lens, so reading looks like eye contact. It's 100% local (nothing leaves the machine), free, open-source, and ships through its own Homebrew tap for a one-line install. ## Outcome A small, finished product with a real user problem at its center. It's also proof that I ship outside the Python/ML lane: native Swift, macOS platform quirks, packaging, and distribution, end to end. repo: https://github.com/Shauryagulati/eyeline --- # Air Quality Prediction, End to End A complete MLOps pipeline on the UCI air-quality dataset: experiment tracking, streaming ingestion, drift monitoring. stack: Python, XGBoost, MLflow, Kafka, Docker, Evidently AI ## Problem Most ML course projects end at a notebook with a good metric. Real systems fail after that point: untracked experiments, data that drifts, models that quietly degrade in production with nobody watching. ## Approach I built the whole lifecycle for an air-quality prediction model on the UCI dataset. XGBoost models with every experiment tracked in MLflow, real-time data ingestion through Kafka, and a prediction API packaged in Docker. Monitoring dashboards built with Evidently AI watch for data drift and performance degradation, so the system tells you when it is going stale instead of waiting to be caught. ## Outcome A reproducible, modular pipeline with continuous evaluation baked in. Ingestion, training, serving, and monitoring as one system rather than a notebook and a prayer. It was my first full answer to what production ML means beyond the model file. --- # Question Answering with BERT on Azure A fine-tuned BERT QA system, benchmarked against classical baselines and deployed for real-time inference on Azure ML. stack: Python, BERT, Hugging Face, Azure ML, SQuAD ## Problem Matching a user's question to the sentence that actually answers it is deceptively hard. Lexical overlap fails on paraphrase, and deep models are easy to demo but hard to serve with acceptable latency. ## Approach I fine-tuned a BERT-based question-answering system on SQuAD to match queries against candidate answer sentences, using semantic similarity over sentence embeddings, and benchmarked it against Jaccard and TF-IDF baselines to prove the deep model earned its cost. The model was deployed on Azure Machine Learning with a scoring pipeline optimized for both latency and accuracy. ## Outcome Full-stack ML deployment, demonstrated end to end: fine-tuning, packaging, serving, and real-time inference. The baseline-comparison discipline, measure the simple thing before shipping the complex one, is a habit I've carried into every system since. --- # Lunon Founding AI Engineer · Feb 2026 – May 2026 - Built the 5-phase core AI pipeline for automated commercial due diligence (private equity): a multi-agent system with quality gates and automated correction loops. - Built the retrieval stack: document ingestion, hybrid search (pgvector + lexical with RRF fusion), query decomposition, and cross-encoder reranking. - Built the analyst copilot: a ~20-tool agent with streaming chat, a propose-then-confirm flow for edits, and citation verification against the source corpus. - Traced and fixed a quality-gate failure loop blocking a client demo; added warn-mode so runs completed instead of stalling. - Set up Langfuse observability for tracing and cost/latency monitoring across the agent pipeline. --- # Carnegie Mellon University AI Research Assistant · Jun 2025 – Dec 2025 - Engineered a hybrid RAG system across 1,000+ regulatory documents: dense retrieval (E5 embeddings, ChromaDB) fused with sparse BM25, re-ranked with a BGE cross-encoder. - Built a PDF ingestion pipeline with page parsing, header detection, and overlapping chunking, with data-lineage tracking for accurate vector-store population. - Evaluated across 50+ query sets: 94% accuracy with grounded citations. --- # Carnegie Mellon University, Heinz College Graduate Teaching Assistant, Fundamentals of Operationalizing AI · Aug 2025 – Oct 2025 - Ran recitations and Q&A on deploying and scaling AI systems in practice; guided students through implementation challenges and graded coursework. --- # YMGrad Software Development Engineer · Jun 2022 – Jul 2024 - Built analytics APIs and optimized Redis caching and MySQL indexing, cutting load times ~25% and query time ~30%. - Automated operational reporting workflows, reducing manual support overhead ~15%. --- # Education ## Carnegie Mellon University Master's in Information Systems Management, Business Intelligence & Data Analytics (Aug 2024 – Dec 2025) Coursework: Computational Data Science, Machine Learning, Applications of NLP and LLMs, Distributed Systems, Applied AI. ## Chandigarh University B.E. Computer Science, specialization in Artificial Intelligence & Machine Learning (2018 – 2022) Undergraduate researcher in sentiment analysis and computer vision; published at international conferences. --- # Publications - Enhancing Sentiment Analysis in Short Texts with POS-Embedded LSTM Models - Paint/Writing Application through Webcam using MediaPipe and OpenCV - Virtual Assistant with Sentiment Analysis - Can Artificial Intelligence Work Wonders in Healthcare? --- # Skills - AI / LLM: Agents, RAG, Claude Agent SDK, OpenAI API, LangChain, MCP, LLM-as-judge, Evals, Prompt engineering & caching - Retrieval: pgvector, FAISS, Hybrid search, BM25, RRF fusion, BGE cross-encoder reranking, Query decomposition - Backend: Python, FastAPI, PostgreSQL, Redis, SQLAlchemy, Alembic, asyncio, SSE streaming, SQL / MySQL - Observability / Infra: Langfuse, OpenTelemetry, Docker, GitHub Actions, Git --- # Resume: Shaurya Gulati ## Education - Carnegie Mellon University: Master's in Information Systems Management, Business Intelligence & Data Analytics (Aug 2024 – Dec 2025). Coursework: Computational Data Science, Machine Learning, Applications of NLP and LLMs, Distributed Systems, Applied AI. - Chandigarh University: B.E. Computer Science, specialization in Artificial Intelligence & Machine Learning (2018 – 2022). Undergraduate researcher in sentiment analysis and computer vision; published at international conferences. ## Publications - Enhancing Sentiment Analysis in Short Texts with POS-Embedded LSTM Models - Paint/Writing Application through Webcam using MediaPipe and OpenCV - Virtual Assistant with Sentiment Analysis - Can Artificial Intelligence Work Wonders in Healthcare? ## Skills - AI / LLM: Agents, RAG, Claude Agent SDK, OpenAI API, LangChain, MCP, LLM-as-judge, Evals, Prompt engineering & caching - Retrieval: pgvector, FAISS, Hybrid search, BM25, RRF fusion, BGE cross-encoder reranking, Query decomposition - Backend: Python, FastAPI, PostgreSQL, Redis, SQLAlchemy, Alembic, asyncio, SSE streaming, SQL / MySQL - Observability / Infra: Langfuse, OpenTelemetry, Docker, GitHub Actions, Git PDF: https://shauryagulati.dev/resume.pdf --- # Never Let Synthesis Outrun Retrieval 2026-07-10 Every RAG demo I have ever watched fails the same way, and almost nobody in the room notices. Someone asks a question, the system produces a confident, fluent, well-structured answer, and everyone nods. Then you check what the retriever actually returned, and it's two half-relevant chunks and a fragment from the wrong document. The language model wrote the answer anyway. It always will. That's what it's for. This is the core failure mode of retrieval-augmented systems: synthesis outrunning retrieval. The model's ability to produce a plausible answer grows completely independent of whether the system found the right information. Worse, better models make the problem harder to see. A weak model on top of bad retrieval produces obviously bad answers. A strong model on top of bad retrieval produces beautiful ones. I learned this on legal text, which is the least forgiving corpus there is. At Carnegie Mellon I built a RAG system over more than a thousand regulatory documents, for people who needed to act on the answers. Regulation punishes paraphrase: a summary that sounds right and cites nothing is worse than no answer, because someone might use it. So the architecture bent everything toward retrieval. Hybrid search, because statutory language is exact while the questions about it never are. Cross-encoder reranking, because the difference between the right section and the almost-right section is the whole job. Data lineage on every chunk, so any answer could be traced back to the page it came from. We evaluated across more than fifty query sets and measured 94 percent accuracy with grounded citations, and the number I trusted most wasn't the accuracy. It was the citation rate. Then I took the lesson into production at Lunon, building due-diligence automation for private equity, where a wrong answer isn't embarrassing, it's expensive. The rule got mechanical there. The analyst copilot verified citations against the source corpus before showing them. The pipeline had quality gates that could fail a run outright rather than let a weak intermediate result flow downstream into something persuasive. When a gate misfired and blocked a client demo, the fix wasn't to remove the gate. It was a warn-mode: complete the run, flag the doubt, keep the human informed. The system stayed honest either way. Why do teams keep breaking this rule? Because synthesis improves your demo overnight and retrieval improves it over months. Swap in a better model and everything instantly sounds smarter, so that's where the effort goes. Retrieval work is unglamorous: chunking strategy, index quality, reranking, evaluation sets. Nobody claps for a better chunker. But the model is rented; your retrieval layer is the actual product. The discipline, as I practice it now: make citations load-bearing, so the system refuses rather than answers uncited. Measure retrieval separately from generation, because a blended score hides exactly the failure you care about. And test behavior, not vibes. I built rag-verdict, an open-source testing framework, around precisely these checks: did the retrieval tool fire, does every citation resolve to a real document, does the agent refuse questions its corpus can't answer. PASS or FAIL, not a decimal. Systems that think are only useful if you can check their work. Synthesis is the voice. Retrieval is the knowledge. Keep the voice on a leash exactly as long as the knowledge. --- # Contact - email: i.shauryagulati@gmail.com - linkedin: https://www.linkedin.com/in/shauryagulati/ - github: https://github.com/Shauryagulati --- This site has an interactive terminal (press /) with a built-in RAG agent. Type "shaurya" in it to ask questions about this content.