Why does PDF extraction sometimes fuse two words together?

PDF text is positioned glyph runs, not words with explicit boundaries. The space between two words is a visual gap, not always an encoded character. An extractor that reads runs in order without reasoning about that gap can fuse adjacent words whose horizontal spacing was tight.

Will layout-aware extraction always fix this?

It fixes most cases, since it reasons about horizontal gaps rather than raw draw order. It can't recover a case where the PDF genuinely has zero space in its underlying content stream. Such a case relies entirely on visual alignment, which an extractor has to infer. Those cases need a post-process regex as a backstop.

Will the camelCase regex fix ever break real text?

Occasionally. A genuine CamelCase identifier or acronym followed by a capital letter can get an unwanted space inserted. It's a heuristic, not a certainty. Spot-check a sample of the output before trusting it at scale.

Two words fused together with no space, over and over.

PDF text is a sequence of positioned glyph runs, not words. There's often no space CHARACTER between them at all, only a visual gap an extractor has to reason about.

looks-like

Revenue grew 34% driven byEMEA demand.The board approved a newpricing tier.

Three fusions, and only two of them leave a detectable trace. byEMEA has a lowercase-to-uppercase boundary, and demand.The has a period followed by a letter. But newpricing is all lowercase mid-sentence, so no pattern can see it.

why-it-happens

PDF text is stored as a sequence of positioned glyph runs, not words. A producer draws each run at an (x, y) coordinate and relies on visual spacing to imply a word boundary. There's no space character between "quick" and "brown" unless the layout explicitly encoded one. An extractor that concatenates runs in draw order without reasoning about the horizontal gap between them fuses adjacent words whose gap happens to be small.

This shows up most in PDFs generated by layout engines that letter-space or kern aggressively, or that split one visual word across multiple glyph runs. That's common with justified text or certain font-subsetting tools. Each run individually has no space character, and none was ever encoded for the extractor to find.

confirm-it

Paste the text into the chunk previewer. Five or more fused-word occurrences trips the missing-spaces signal specifically. Scan for missing spaces

fix-it-yourself

Re-run extraction with layout preserved

pdftotext's -layout flag reasons about column positions and horizontal gaps rather than raw run order. This recovers spaces a naive concatenation drops.

bash

pdftotext -layout input.pdf output.txt

Insert spaces heuristically as a post-process

These two patterns, a lowercase-to-uppercase boundary and a period followed immediately by a letter, are exactly the two the chunk previewer counts for this signal. So what the tool flags is what this repairs. Two honest limits apply. It will occasionally split a genuine CamelCase identifier or an acronym. And it cannot see an all-lowercase fusion mid-sentence, like the newpricing case above, because nothing in the text marks where the boundary was.

python

import re
text = re.sub(r"([a-z])([A-Z])", r"\1 \2", text)
text = re.sub(r"\.([A-Za-z])", r". \1", text)

what-txtfetch-does

Apache Tika's PDF text extraction reasons about glyph position, not just draw order. So it reconstructs word and line spacing far more reliably than a naive run-concatenation extractor. This is the same class of layout-aware reading pdftotext -layout does.

It isn't perfect on every producer. A PDF that genuinely encodes zero space characters between visually-adjacent runs, relying entirely on gap detection, can still fuse words in txtfetch's output. That's because there's no space in the underlying content stream for any extractor to find.

what-it-costs-you-downstream

A fused run like "newpricing" tokenizes and chunks as one unrecognizable unit. It won't match a lexical search for either of its component words. It embeds as noise rather than as the concept those words actually represent.

faq

Why does PDF extraction sometimes fuse two words together?: PDF text is positioned glyph runs, not words with explicit boundaries. The space between two words is a visual gap, not always an encoded character. An extractor that reads runs in order without reasoning about that gap can fuse adjacent words whose horizontal spacing was tight.
Will layout-aware extraction always fix this?: It fixes most cases, since it reasons about horizontal gaps rather than raw draw order. It can't recover a case where the PDF genuinely has zero space in its underlying content stream. Such a case relies entirely on visual alignment, which an extractor has to infer. Those cases need a post-process regex as a backstop.
Will the camelCase regex fix ever break real text?: Occasionally. A genuine CamelCase identifier or acronym followed by a capital letter can get an unwanted space inserted. It's a heuristic, not a certainty. Spot-check a sample of the output before trusting it at scale.

related-reading

Characters & encoding

Fix the text you already have.

The free cleaner repairs this damage in your browser. Nothing leaves the page.

Clean up your text →

Get an API key →