Does RAG fully eliminate hallucinations?

No, though it cuts them substantially. Grounding responses in retrieved text, insisting on citations and telling the model to decline when context is absent make incorrect answers both less frequent and easier to spot.

What vector database is best for RAG?

pgvector within PostgreSQL is enough for most teams to begin with; dedicated stores like Pinecone, Weaviate or Qdrant help at bigger scale or with heavy filtering. Chunking and re-ranking affect retrieval quality more than the choice of database.

Can a RAG system honour document permissions?

Yes. Save access metadata alongside each chunk and filter retrieval using the signed-in user's permissions, so the model never receives content that user may not read.

How long does building a production RAG system take?

A focused proof of value over one document set can be delivered in a few weeks; hardening for production — connectors, permissions, evaluation and monitoring — usually takes a further one to three months.

Retrieval-Augmented Generation (RAG) Guide for Business

Retrieval-augmented generation (RAG) explained: how it works, when to pick it over fine-tuning, and how to build a RAG system that survives production.

Retrieval-augmented generation (RAG) allows a large language model to answer from your own documents and data. When a question arrives, the system fetches the most relevant passages and hands them to the model as context, so responses stay grounded, up to date and traceable to a source — with no need to retrain the model.

Retrieval-augmented generation in plain terms

A large language model knows only what its training data contained. Your policies, contracts, product catalogue and last week's tickets are unknown to it, and if asked regardless it may give an answer that sounds certain but is wrong. RAG fills that gap by inserting a retrieval step ahead of generation.

As soon as a user submits a question, the system looks through a knowledge store for related passages, inserts the strongest matches into the prompt and tells the model to answer from that context. Reasoning and writing come from the model; the facts come from the retrieval layer.

The stages of a RAG pipeline

Production RAG is made of two parts: an ingestion pipeline that gets your content ready, and a query pipeline that responds to questions.

RAG or fine-tuning: how to decide

Reach for RAG when the challenge is knowledge: answers rely on facts that change, are private or need citing. Refreshing a RAG system is just a matter of re-indexing a document, and access control can be applied per user.

Reach for fine-tuning when the challenge is behaviour: a consistent output format, a domain's tone and vocabulary, or a narrow task a smaller, cheaper model should perform dependably. Many strong systems blend the two — a model fine-tuned to do the task well, grounded through RAG in the latest facts.

Advanced RAG techniques worth knowing

Basic top-k vector search is fine for a demo. Real-world document collections generally demand more:

Evaluating RAG quality

Before tuning anything, assemble an evaluation set of real questions with known correct answers and sources. Score retrieval quality (was the right passage returned?) independently of answer quality (is the answer correct, complete and faithful to the context?).

Treat faithfulness — answers backed by the retrieved text — as a core metric, and log every production query along with its retrieved context so you can diagnose failures. Frameworks like Ragas and LLM-as-judge scoring help automate the work, but a golden set reviewed by humans stays the reference point.

Why RAG systems fail, and how to prevent it

The majority of RAG issues are really data issues: badly parsed PDFs, chunks that split tables down the middle, outdated indexes or absent permissions metadata. Next most frequent is retrieval that surfaces believable but incorrect passages, which the model then repeats with confidence.

Defend against these with document-quality checks during ingestion, re-ranking, an instruction to reply "I don't know" when context falls short, and per-user access filtering applied at retrieval time instead of after generation.

Planning an assistant over private data? Our team delivers RAG and knowledge systems from start to finish — our AI services page explains how we approach it.

Ingest: collect documents from file shares, wikis, ticketing tools, databases and email, and keep everything synchronised.
Parse and chunk: pull out clean text (tables and scanned pages included), then divide it into passages that are small enough for precise retrieval yet big enough to retain meaning.
Embed and index: turn every chunk into a vector embedding and save it in a vector database together with metadata such as source, date and access permissions.
Retrieve: for every question, perform vector search — typically paired with keyword search — and filter by what the user is permitted to see.
Re-rank: apply a re-ranking model to push the most relevant passages to the top and discard weak matches.
Generate and cite: the LLM answers using the retrieved passages and supplies citations so users can verify the source.
Hybrid search (vector plus BM25 keyword) so exact terms such as product codes and clause numbers are still found.
Contextual chunking that keeps each passage linked to its headings and document title.
GraphRAG or knowledge graphs when answers hinge on relationships between entities rather than individual passages.
Text-to-SQL for questions answered by databases instead of documents.
Query rewriting and multi-step retrieval for complicated or multi-part questions.
Caching frequent questions and retrieved context to lower latency and token cost.