HN Reader

Ask HN: Any PDF Benchmarks?

2 nnurmanov 1 6/13/2025, 7:16:51 AM
I am testing several PDF parsing libraries, and unfortunately, most of them have issues. For example, many struggle with non-English languages, while others can’t reliably handle tables. Having a standardized PDF benchmark would be incredibly helpful — at the very least, it would allow us to identify obvious shortcomings without needing to install and test each library individually.

Is anyone building such a benchmark?

Comments (1)

nnurmanov · 1d ago
For example, markitdown from MS can't recognize Cyrillic text, when I started researching into it, I found that they use pdfminer.six under the hood and there is an unresolved issue with supporting languages.

Docling is OK with tables, but fails with cyrillic text;

marker-pdf is OK with tables, but it also fails with cyrillic text;

What other pdf parser libraries exist? I am looking for preferably on-premise solutions, but if I won't find a reliable and accurate solution, I might consider cloud based solutions as well.