Designing an MCP server for private documents: why intent-based retrieval matters

The Model Context Protocol (MCP) gives LLM clients a standard way to call tools and read context from your systems. That’s powerful, but “expose my documents over MCP” hides most of the real design work. Here’s how I think about building a document server that returns the right context, not just some context.

What an MCP server actually is

An MCP server advertises tools (actions a model can call) and resources (data it can read) to any compatible client. The same server can be reached over stdio (local, launched by the client) or HTTP with SSE (remote, shared). Designing for both transports from the start means one retrieval layer can serve a desktop assistant today and a team-wide service tomorrow.

Because the protocol is vendor-neutral, a well-built server also stays provider-agnostic: the model on the other end could be from Anthropic, OpenAI or a local runtime, and your retrieval logic doesn’t change.

The pipeline behind the tool

A document-search tool is only as good as what sits behind it:

1. Ingest each format with a parser that respects its structure (PDF pages, DOCX headings, CSV rows, JSON paths) rather than flattening everything to text.
2. Chunk recursively along natural boundaries (sections, paragraphs, sentences) with a little overlap so meaning isn’t cut in half.
3. Embed each chunk into a vector, for example 384-dimension sentence embeddings, and store them in a vector database such as LanceDB with source metadata.
4. Watch the document folder so edits, additions and deletions update the index automatically. Stale context is a silent failure mode.
5. Retrieve at query time and return passages with citations back to the model.

Where most servers stop, and where precision comes from

The naive version runs a single top-k vector search for every question. That works for “what does the contract say about termination?” and fails for “summarize the Q3 report” or “compare these two proposals”, which need different retrieval behaviour.

Intent-based retrieval adds one cheap step before searching: classify what the user is trying to do, then pick a strategy.

Intent	Strategy
Fact lookup	Small top-k, high similarity threshold, return exact passages
Summarize	Retrieve broadly across a single document, then order by position
Compare	Retrieve per document, return side-by-side with labels
Explore	Larger k, diverse results, include neighbouring chunks

A sketch of the tool handler shows how little code the idea needs:

async function searchDocuments(query: string, filters?: Filters) {
  const intent = await classifyIntent(query); // 'lookup' | 'summarize' | 'compare' | 'explore'
  const plan = strategies[intent];            // k, threshold, grouping, ordering

  const hits = await vectorStore.search(await embed(query), {
    k: plan.k,
    minScore: plan.minScore,
    filter: filters,
  });

  return plan.shape(hits).map((h) => ({
    text: h.text,
    source: h.metadata.file,
    location: h.metadata.page ?? h.metadata.section,
    score: h.score,
  }));
}

The classifier can be a few-shot prompt to a small model or even rules for a narrow domain. What matters is that retrieval is no longer one-size-fits-all.

Pitfalls worth designing around

• Chunk size is a product decision. Too small loses context; too large dilutes relevance. Tune it against real questions.
• Always return provenance. File, page and section let users (and models) verify an answer, and make failures debuggable.
• Mind permissions. A server that can read everything will happily leak everything. Scope access per client and per folder.
• Measure. Keep a small set of real questions with known answers and re-run it whenever you change chunking, embeddings or strategies.

Done well, an MCP server turns a pile of private documents into a dependable, vendor-neutral knowledge layer. If you’re exploring one for your team, I’m happy to talk through the architecture, so get in touch.