Lessons Learned Building a Production RAG Pipeline for ChatNHS

When building ChatNHS, we were tasked with creating a specialized text-based chatbot for the hydropower sector in Nepal. The challenge? An absolutely massive amount of highly noisy, unstructured data.

A standard LLM would immediately hallucinate when asked complex engineering or operational questions. Fine-tuning was too rigid. The only path forward was a highly customized Retrieval-Augmented Generation (RAG) pipeline.

The Data Ingestion Engine

If you just split documents by 500 characters, you slice critical data in half. We built an ingestion engine that processes data through embedding models into precise chunks with perfect overlapping and metadata tagging. This ensures that when the AI retrieves information, it never pulls an incomplete or out-of-context answer.

Intent Checking & Reranking

To completely eliminate hallucinations, we implemented user intent checking before the query even hits the core retrieval system. Once the documents are fetched, they go through a reranking phase, guaranteeing the AI only generates answers from the absolute best, most relevant documents.

Context Windows & Hybrid Search

We heavily optimized the context window. Instead of dumping the entire chat history (which leads to slow responses and hallucinations), we dynamically bind the context. The chatbot seamlessly handles follow-ups, utilizes hybrid search (combining vector and keyword search), and can fall back to web search or general model answers when appropriate.

By keeping the stack tight React, Django, and our custom RAG engine, we delivered a lightning-fast, highly accurate AI solution tailored exactly to the needs of the hydropower sector.