Regulatory RAG at CMU — Shaurya Gulati

problem

Policy and corporate stakeholders navigating regulation across U.S. states and Europe face thousands of pages of interlinked legal text, and legal text is exactly where LLM hallucination costs the most. As an AI Research Assistant at Carnegie Mellon, my brief was a system whose answers could be trusted enough to act on: grounded, cited, and measured.

approach

The retrieval core is hybrid. Dense E5 embeddings in ChromaDB fused with sparse BM25, then re-ranked with a BGE cross-encoder, because regulatory queries mix precise statutory language with fuzzy intent and neither dense nor sparse retrieval wins alone. Feeding it is a PDF ingestion pipeline with page parsing, header detection, and overlapping chunking, with data-lineage tracking so every vector traces back to its exact source page. The project grew out of my earlier multi-agent work, a Planner-Retriever-Synthesizer-Reviewer pipeline evaluated with RAGAS at 0.78 faithfulness and 0.74 answer relevancy, which taught me where agent complexity pays off and where retrieval quality is the whole game.

outcome

Evaluated across 50+ query sets: 94% accuracy with grounded citations over 1,000+ regulatory documents. The deeper outcome is a working philosophy: separate the roles, verify the output, track lineage from answer back to page. I've carried it into every retrieval system since, including the one inside this site's terminal.