How to convert PDFs to Markdown for RAG (with page numbers, clean chunks and a missing-page check)

If you're building retrieval-augmented generation (RAG) over PDFs, the conversion step decides a lot more than people expect. Retrieval can only find what made it out of the PDF. Answers can only cite a page if you kept the page number. And a scanned page that converts to nothing will never show up as a gap: it just silently isn't there.

This guide converts a real PDF to Markdown, splits it into chunks that keep their heading and page number, cleans up the usual PDF debris and checks for missing pages. Everything uses free, open-source tools. At the end there's a hosted option for when you'd rather call an API.

The test file throughout is the original RAG paper, Lewis et al. 2020 (19 pages, arXiv PDF).

Why Markdown, and not plain text?

Plain-text extraction gives you one long string. Markdown keeps the structure that makes chunking and retrieval work:

• Headings (## 2 Methods, ### 2.1 Models) give you natural chunk boundaries, and a heading is useful metadata to embed alongside the chunk.
• Tables stay as rows and columns instead of a jumble of numbers.
• Lists stay as lists, so a step-by-step procedure doesn't merge into one paragraph.

LLMs also read Markdown well, so the same text works for embedding and for the context you pass to the model.

Step 1: Pick a converter

There's no single best PDF-to-Markdown tool. These are the ones worth trying, all free to run yourself:

Tool	Good at	Watch out for
PyMuPDF4LLM	Fast, page-by-page output, headings from font size, tables	AGPL licence (commercial licence available)
Docling (IBM)	Layout analysis, complex tables, reading order	Heavier install, slower on CPU
Marker	Equations, multi-column papers, OCR	Needs a GPU to be quick. Check its licence for commercial use
MinerU	Scientific papers, formulas, tables	Large models to download
MarkItDown (Microsoft)	Office files and HTML	Its PDF converter returns plain text, with no headings

Start with the lightest one that handles your documents, and test it on your own PDFs, not someone else's benchmark. A tool that's great on arXiv papers can struggle with scanned invoices or two-column brochures.

We'll use PyMuPDF4LLM because it's quick and returns one result per page, which makes page numbers easy.

Step 2: Convert the PDF page by page

# pip install "pymupdf4llm<2" langchain-text-splitters
import pymupdf4llm

pages = pymupdf4llm.to_markdown("rag.pdf", page_chunks=True)
print(len(pages))                          # 19
print(pages[0]["metadata"]["page_number"]) # 1
print(pages[0]["text"][:300])

A note on that version pin: at the time of writing, pip install pymupdf4llm installs version 2.0, which is an empty placeholder package. The PyMuPDF team moved the code into PyMuPDF 2.x itself. Pinning <2 gets you the 1.x release with the API used here.

On our test machine (a small cloud VM, CPU only), the whole 19-page paper took about 2.7 seconds. page_chunks=True returns a list of dictionaries, one per page, each with the page's Markdown in text and its 1-based page_number in metadata. Headings came out as ### **2 Methods** and #### **2.1 Models**. PDFs don't actually store headings, so every tool infers them from font size and weight. Spot-check them on your own files.

Step 3: Chunk by heading, keep the page number

Splitting every 1,000 characters regardless of structure works, but it cuts sections in half and loses the heading. A better default is to split on headings first, then split any section that's still too long:

import re
import pymupdf4llm
from langchain_text_splitters import MarkdownHeaderTextSplitter, RecursiveCharacterTextSplitter

pages = pymupdf4llm.to_markdown("rag.pdf", page_chunks=True)

header_splitter = MarkdownHeaderTextSplitter(
    headers_to_split_on=[("#", "h1"), ("##", "h2"), ("###", "h3"), ("####", "h4")],
    strip_headers=False,
)
size_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=100)

chunks, current = [], {}
for page in pages:
    page_no = page["metadata"]["page_number"]  # 1-based in pymupdf4llm 1.x
    for section in header_splitter.split_text(page["text"]):
        # a page that starts mid-section keeps the previous heading
        current = section.metadata or current
        heading = next((current[k] for k in ("h4", "h3", "h2", "h1") if k in current), None)
        for piece in size_splitter.split_text(section.page_content):
            chunks.append({
                "text": piece,
                "page": page_no,
                "heading": re.sub(r"[*_]", "", heading).strip() if heading else None,
                "source": "rag.pdf",
            })

print(len(chunks), "chunks")

This gave 110 chunks. Each one carries page, heading and source. Store those as metadata in your vector database so an answer can say "Lewis et al., page 3, section 2 Methods" instead of just quoting text.

Splitting page by page means a section that runs across a page break becomes two chunks. For citations that's usually what you want. If you'd rather have whole sections, join the pages first and insert a marker such as <!-- page 3 --> at each boundary, so you can still work out which page a chunk starts on.

Step 4: Clean up the PDF debris

Look at a few chunks before you embed anything. In ours, the chunk for "2 Methods" ended with a footnote and a lone 2, which is the page number from the footer. Common debris in PDF text:

• Page numbers, running headers and footers repeated on every page.
• Ligatures: many PDFs store "fi" and "fl" as single characters (fi, fl). The text looks fine on screen, but "fine-tuned" won't match a search for "fine-tuned", and embeddings treat them differently.
• Words broken across lines with a hyphen, such as knowl- + edge.

PyMuPDF4LLM already joined the broken words and expanded the ligatures in our paper. Not every extractor does: pdfminer-based tools often keep both. If yours doesn't, a few lines fix most of it:

import re
import unicodedata

def clean(text: str) -> str:
    text = unicodedata.normalize("NFKC", text)       # "fi" -> "fi"
    text = re.sub(r"(\w)-\n(\w)", r"\1\2", text)      # "knowl-\nedge" -> "knowledge"
    text = re.sub(r"\n\s*\d{1,3}\s*$", "", text)      # lone page number at the very end
    return text

The hyphen rule is a trade-off. In this paper it would also glue genuine compounds that happen to break at the line end, like knowledge- + intensive and cross- + encoder. For retrieval that's a smaller problem than half-words, but don't use this rule on text you'll show to users as-is.

Step 5: Check for pages that came out empty

This is the step most pipelines skip. A scanned page, or a page where the text was converted to vector shapes, has no text layer. Every converter returns nothing for it, and nothing raises an error. Check each page's text length and whether the page contains images:

import pymupdf  # installed with pymupdf4llm

def suspicious_pages(path, min_chars=50):
    doc = pymupdf.open(path)
    flagged = []
    for page in doc:
        if len(page.get_text().strip()) < min_chars:
            reason = "image only (scanned?)" if page.get_images() else "blank or near-blank"
            flagged.append((page.number + 1, reason))
    return flagged

print(suspicious_pages("rag.pdf"))  # [] for this paper

Our paper came back clean. For a pile of real-world documents, route any flagged file to OCR (Docling and Marker both have it, as does Tesseract via ocrmypdf) or to a human, instead of embedding a document with holes in it. A word count per page that's far below the rest of the document is another cheap warning sign.

Step 6: Embed and store

From here it's standard RAG: embed text, keep page, heading and source as metadata, and pass the metadata through to your prompt so the model can cite. Re-run Steps 4 and 5 whenever you add documents, because a new batch often brings a new kind of mess.

The hosted option: an API instead of a conversion server

If you'd rather not run and maintain this (or you need to convert Word, PowerPoint and Excel files and shared Google Drive, Dropbox or SharePoint folders, not just PDFs), we built Document to Markdown on Apify. You send a URL, it returns Markdown, chunks with page and heading, and a per-document quality report covering empty and scanned pages. OCR for scanned pages is optional.

We ran it on the same paper:

# pip install apify-client
from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")
run = client.actor("flyingkiwi/document-to-markdown").call(run_input={
    "urls": ["https://arxiv.org/pdf/2005.11401"],
    "chunkSize": 1000,
    "chunkOverlap": 100,
})

for doc in client.dataset(run.default_dataset_id).iterate_items():
    print(doc["title"], doc["pageCount"], len(doc["chunks"]), doc["quality"]["possibleContentLoss"])

(That's apify-client 3.x, where call() returns a Run object. With the older 1.x and 2.x clients, use run["defaultDatasetId"].)

Or with plain HTTP, which waits for the run and returns the results in one request (fine for small jobs; long runs should use the asynchronous endpoints):

curl -X POST "https://api.apify.com/v2/acts/flyingkiwi~document-to-markdown/run-sync-get-dataset-items" \
  -H "Authorization: Bearer $APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"urls": ["https://arxiv.org/pdf/2005.11401"], "chunkSize": 1000, "chunkOverlap": 100}'

The run took about 7 seconds. It returned the title "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks", 19 pages, 9,692 words and 96 chunks averaging about 730 characters. Every chunk had a heading and a page number:

{
  "index": 17,
  "page": 4,
  "heading": "3 Experiments",
  "charCount": 753,
  "text": "### 3 Experiments\n\nWe experiment with RAG in a wide range of knowledge-intensive tasks. For all experiments, we use\na single Wikipedia dump for our non-parametric knowledge source. ..."
}

The quality report showed 19 of 19 pages with text, no scanned pages and possibleContentLoss: false. The cost was $0.006: $0.003 to start the run and $0.003 for the document (up to 20 pages, then $0.001 per extra page).

To be straight about how it compares with the DIY route on this paper: PyMuPDF4LLM's output was cleaner on the text itself. Document to Markdown left the ligatures (fi) and the line-end hyphens in place, so run the clean() function from Step 4 on its output too. Where it helps is everything around the conversion: one call for mixed file types and whole shared folders, chunks and page numbers built in, the empty-page check on every document, and nothing to install or keep updated.

Which route should you take?

• A few hundred PDFs, and you're comfortable with Python: go DIY. PyMuPDF4LLM (mind the AGPL licence) or Docling, plus the chunking and checks above, will do the job for free.
• Papers full of equations, or heavily laid-out PDFs: try Marker or MinerU and compare them on your own files.
• Mixed Office files and shared folders, called from an app or an agent, without running a server: a hosted converter saves the maintenance. Check whichever one you pick for empty pages anyway.

We (flyingkiwi) built and maintain Document to Markdown. The numbers in this post come from runs on 2 November 2026 (NZ time): PyMuPDF4LLM 1.28.2 and langchain-text-splitters 1.1.3 locally, and Document to Markdown on Apify.

Tool: https://apify.com/flyingkiwi/document-to-markdown