Building Retrieval-Augmented Generation (RAG) in C#
A C# Corner walkthrough builds a working RAG pipeline in a small console app on Microsoft.Extensions.AI, then goes past the demo: hybrid search, reranking, GraphRAG, agentic retrieval and evaluation. It is a compact map of the design decisions that decide whether a grounded assistant is trustworthy.
Key points
- Chunk a few hundred tokens at a time with 10-20% overlap, split on headings where possible, and inspect the extracted text before blaming retrieval.
- Documents and queries must use the same embedding model; switching models means re-embedding the entire corpus.
- Start with a database that already supports vectors (pgvector, SQL Server, Azure SQL) instead of standing up a separate vector database.
- Hybrid search (vector + BM25 merged with RRF) plus a cross-encoder reranker is the highest-leverage retrieval upgrade.
- Filter by tenant, language and user permissions at retrieval time, and tell the model to answer only from the supplied context.
- Evaluate retrieval recall@k separately from generation faithfulness, using a test set of 50-200 real questions.
C# Corner builds a small console app that answers questions from a folder of Markdown files. It uses Microsoft.Extensions.AI's IEmbeddingGenerator and IChatClient, so OpenAI, Azure OpenAI or Ollama can be swapped without touching the RAG logic; paragraphs are chunked with overlap, embedded in batches, and scored with TensorPrimitives.CosineSimilarity over a plain list.
The advice around the demo is the valuable part. Keep vectors in a database you already run, such as PostgreSQL with pgvector or SQL Server, before buying a dedicated vector store; apply tenant and user permission filters before chunks reach the model; and instruct the model to say it does not know when the context is silent. From there: hybrid search with RRF, rerankers, GraphRAG, agentic retrieval as an MCP tool, and measuring recall@k separately from faithfulness.
Real-world use
- An internal assistant that answers from wikis, PDFs and runbooks, with citations back to the source page.
- A support bot searching past tickets where each customer's results are filtered by tenant and role before the model sees them.
- A "chat with your contracts" tool for a legal or procurement team that must show the passage an answer came from.
- An agentic workflow that turns search and order lookups into callable tools so multi-step questions can be answered.
Why it matters
RAG is now the default way to put an LLM on private data, and most failures come from ingestion and retrieval rather than the model. The article's checklist — clean text, permission filters, reranking, separate retrieval metrics — is what separates a demo from something you can ship.