ColBERT vs. ColPali: From Text-Only to Multimodal Late Interaction Retrieval

What's the difference between ColBERT and ColPali?
ColBERT and ColPali are both new retrieval models based on the late interaction mechanism, but how do they differ?
ColBERT: Contextualized Late interaction over BERT
ColPali: Contextualized late interaction over PaliGemma
ColBERT: single modality (text-only)
ColPali: multimodal (text and images) using VLMs
ColBERT: documents as text only
ColPali: documents as images (screenshots of PDFs)
ColBERT: https://lnkd.in/gaV3mRev
ColPali: https://lnkd.in/gTf68AgY

This post is factual and educational — a concise technical comparison between two retrieval models. It contains no promotional or opinion-based language; it's a straightforward definitional breakdown.

Confirmed Accurate

• ColBERT stands for "Contextualized Late Interaction over BERT," introduced in a 2020 Stanford paper (Khattab & Zaharia) as a passage retrieval model using a late interaction mechanism.
• ColPali stands for "Contextualized Late Interaction over PaliGemma," using PaliGemma (a vision-language model) as its backbone, analogous to how ColBERT uses BERT.
• ColBERT is a single-modality (text-only) model, since BERT itself is text-only.
• ColPali is multimodal, encoding documents as images (e.g., screenshots of PDF pages) rather than extracted text, using a vision-language model architecture.
• Both models are correctly described as belonging to the "late interaction" family of retrieval architectures.

Not Fully Accurate / Promotional

• None identified. The post makes no exaggerated, promotional, or opinion-based claims — it's a neutral technical summary.

Not Independently Verifiable

• No personal experience, benchmark results, or user-specific claims are made in the post, so there is nothing in this category to flag.
• Note: the links included in the original post appear malformed/broken (garbled with stray characters), so their destinations can't be verified — but this doesn't affect the accuracy of the technical claims themselves.