RAG In Production: What We Learned Building Enterprise Search.RAG In Production: What We Learned Building Enterprise Search.

A retrieval-augmented search demo takes an afternoon. Making it answer correctly for thousands of real documents takes much longer. On our enterprise search build, the first version found the right passage about 65 percent of the time. These are the changes that moved it past 90.

Measure before you tune

Before changing anything we wrote an evaluation set: real questions from the client's team, each paired with the passage that answers it. Every change after that was scored against the same set, so we always knew whether we had made things better or just different.

Chunk by meaning, not by length

Fixed-size chunks split tables, lists and procedures in half. Splitting on headings and paragraphs, and keeping each chunk's section title with it, gave the model context that fixed windows threw away.

Hybrid retrieval

Vector search is good at meaning and bad at exact terms like product codes and policy numbers. Keyword search is the opposite. Running both and merging the results fixed most of the misses our evaluation set exposed.

• Vector and keyword search run side by side
• Results merged and re-ranked before they reach the model
• Filters for department and document date applied before ranking

Citations build trust

Every answer links to the passages it came from. Users check the source when it matters, and when the model has nothing to cite it says so instead of guessing.

• AI
• RAG
• Search

Written by

Haseeb

AI and ML Developer, Dev N Scale

Haseeb builds the retrieval, chat and automation features in our AI products, and the evaluation sets that keep them honest.