fairdataihub/poster-sentry

PosterSentry: Multimodal Scientific Poster Classifier

Model Description

PosterSentry is a lightweight, CPU-optimized multimodal classifier that determines whether a PDF is a scientific poster or a non-poster (paper, proceedings, newsletter, abstract book, etc.).

Part of the quality control pipeline for posters.science, a platform for making scientific conference posters Findable, Accessible, Interoperable, and Reusable (FAIR).

Developed by the FAIR Data Innovations Hub at the California Medical Innovations Institute (CalMI²).

Version

Version	Date	Notes
1.2.0	2026-09-01	Stacked head: the 512-d text embedding is scored by a stage-one logistic regression whose poster probability becomes a single text-score feature for the final classifier over [text_score + 15 visual + 15 structural] = 31 features. Held-out accuracy 92.9%. Head weights now store a zero first column so softmax reproduces sklearn's sigmoid exactly; earlier heads doubled the logit, which sharpened reported confidences (decisions at 0.5 were unaffected). Flat heads remain loadable by the package.
1.1.0	2026-08-28	Feature extraction moved off the AGPL-licensed PyMuPDF to pdfplumber (text and structure) and pypdfium2 (page rendering), both permissively licensed, so the whole stack is MIT-compatible. The head was retrained on the re-extracted features; held-out accuracy is 89.2%.
1.0.0	2026-08-18	Head trained on the human-validated corpus: 3,381 documents labeled by a three-reviewer survey (Krippendorff's alpha 0.79) with blinded adjudication of the 439 contested documents.

Earlier unversioned weights (April 2026) were trained on heuristically labeled data and are superseded; they remain available in the repository history.

Related Models & Tools

Resource	Description	Link
PosterSentry	Multimodal poster classifier (this model)	fairdataihub/poster-sentry
poster-sentry	Installable classifier package	GitHub
poster-sentry-training	Training code and replication	GitHub
poster-sentry-training-data	Human-validated training dataset (3,381 samples)	HuggingFace
poster-sentry-evaluation-paper-code	Reproducible analysis for the paper	GitHub
Llama-3.1-8B-Poster-Extraction	Poster → structured JSON extraction	fairdataihub/Llama-3.1-8B-Poster-Extraction
poster2json	Python library for poster extraction	PyPI · Docs · GitHub
poster-json-schema	DataCite-based poster metadata schema	GitHub
Platform	posters.science	posters.science

Pipeline Position

PosterSentry sits at the front of the posters.science pipeline. It screens incoming PDFs before the expensive Llama-based extraction:

PDF Input
   │
   ▼
┌──────────────┐     ┌───────────────────────────────────┐     ┌──────────────┐
│ PosterSentry │ ──► │ Llama-3.1-8B-Poster-Extraction    │ ──► │ poster2json  │
│ (classify)   │     │ (extract structured metadata)      │     │ (validate)   │
└──────────────┘     └───────────────────────────────────┘     └──────────────┘
   poster? ✓              raw text → JSON schema                  FAIR output

Architecture

Two-stage (stacked) logistic regression. Stage 1 scores the 512-d text embedding alone; its poster probability becomes the single text_score feature of stage 2, which classifies 31 features:

Channel	Features	Dimension	Signal
Text score	Stage-1 LogisticRegression over the model2vec (potion-base-32M) embedding	1	Semantic content, summarized
Visual	Color stats, edge density, FFT spatial complexity, whitespace	15	Visual layout
Structural	Page count, area, font diversity, text blocks, density	15	PDF geometry

Stage 2 is trained on inner 5-fold out-of-fold text scores so it never sees an in-sample-optimistic text score. Both stages (weights and scalers) live in one numpy .npz head (about 20 KB). Inference is pure numpy, no torch required at prediction time. Every stage-2 coefficient is a named, interpretable feature; the raw 542-d concatenation of earlier releases underperformed it by about six points out of fold because the 512 standardized text dimensions crowded out the reliable engineered signal.

PDF backend (speed vs license)

Feature extraction reads PDFs through a selectable backend:

Backend	Libraries	License	Speed	Notes
pdfplumber (default)	pdfplumber + pypdfium2	MIT + BSD-3-Clause	~2.0 s/doc	The stack the released model is trained on; reproduces the paper.
pymupdf	PyMuPDF	AGPL-3.0	~1.2 s/doc (about 1.75x faster)	Faster, but AGPL. Install with pip install poster-sentry[pymupdf]. Its extracted features differ slightly, so retrain with this backend for best accuracy or accept minor prediction drift.

Select the backend with --backend, the backend= constructor argument, or the POSTER_SENTRY_BACKEND environment variable. The default is pdfplumber so results match this model.

On a 40-document corpus sample, the dominant cost in the default stack is pdfplumber's pure-Python text and structure parsing (about 780 ms and 760 ms per document); page rendering with pypdfium2 adds about 490 ms. A pypdfium2-based text variant cuts the text step to about 75 ms and reaches roughly 1.3 s/doc, but it changes the extracted text, so it would require its own retrain; pymupdf is the recommended fast path when AGPL is acceptable.

Performance

Evaluated against human-validated labels (three-reviewer survey with blinded adjudication of contested documents):

Metric	Value
Held-out accuracy (508 documents)	92.9% (95% CI 90.3 to 94.8)
Macro F1 (held-out)	0.929
Precision / Recall / F1 (poster)	0.932 / 0.925 / 0.929
Precision / Recall / F1 (non-poster)	0.926 / 0.933 / 0.930
Nested out-of-fold accuracy (all 3,381 documents, 5-fold)	94.4%
Panel reliability with the model as a fourth rater	alpha 0.793 -> 0.797 (unchanged)
Calibration (out-of-fold)	Brier 0.045, ECE 0.017
Classifier speed	under one second per file on a CPU (PDF parsing is the bottleneck)

Errors concentrate where the human panel itself divided: out-of-fold agreement is 96.5% on documents the panel rated unanimously and 81.1% on documents decided two to one.

Top Features by Importance

Standardized logistic regression coefficients of the trained head (positive pushes toward poster):

Rank	Feature	Coefficient	Signal
1	page_count	-3.87	More pages pushes away from poster
2	size_per_page_kb	+2.49	Dense, high-resolution single pages
3	line_count	+2.00	Posters pack many short text lines
4	file_size_kb	-1.74	Multi-page documents are bigger overall
5	mean_g	+1.30	Colorful, non-white pages
6	img_width	+1.21	Large rendered width
7	page_width_pt	+1.16	Posters are physically wide
8	edge_density	+0.83	Visually busy layouts
9	is_landscape	+0.75	Many posters are landscape
10	color_diversity	+0.75	Posters are visually rich

Every coefficient of the final classifier is a named feature; the whole text channel enters as the single text_score (+0.64, rank 12 of 31), whose value concentrates on the documents where geometry misleads.

Training Data

Trained on 3,381 documents with human-validated labels, zero synthetic data:

Class	Count	Label provenance
Poster	1,686	Three-reviewer survey; unanimous panel label or blinded adjudication
Non-poster	1,695	Three-reviewer survey; unanimous panel label or blinded adjudication

Three reviewers independently rated all 3,570 candidate documents (inter-rater Krippendorff's alpha 0.79); the 439 documents without a unanimous panel were settled in a blinded adjudication review. After removing 182 near-duplicate documents and 7 with unavailable PDFs, the remaining 3,381 form the training corpus. Applied to the full corpus of 30,139 readable repository PDFs labeled as posters, PosterSentry classifies 80.6% as posters: roughly one in five records labeled as posters is something else.

Training data: fairdataihub/poster-sentry-training-data

Usage

Python API

from poster_sentry import PosterSentry

sentry = PosterSentry()
sentry.initialize()

# Classify a PDF (uses text + visual + structural features)
result = sentry.classify("document.pdf")
print(f"Is poster: {result['is_poster']}, Confidence: {result['confidence']:.2f}")
# {'is_poster': True, 'confidence': 0.97, 'path': 'document.pdf'}

# Batch classification
results = sentry.classify_batch(["poster1.pdf", "paper.pdf", "newsletter.pdf"])

Installation

pip install git+https://github.com/fairdataihub/poster-repo-qc.git

# Or install from source
git clone https://github.com/fairdataihub/poster-repo-qc.git
cd poster-repo-qc
pip install -e ".[train]"

Training

python scripts/train_poster_sentry.py --n-per-class 2000

Training completes in ~40 minutes on CPU (PDF rendering is the bottleneck, not the classifier).

Model Specifications

Attribute	Value
Embedding backbone	minishlab/potion-base-32M (model2vec StaticModel)
Embedding dimension	512
Visual features	15 (color, edge, FFT, whitespace)
Structural features	15 (page geometry, fonts, text blocks)
Final-classifier input dimension	31 (text score + 15 visual + 15 structural)
Classifier	Two-stage (stacked) LogisticRegression + StandardScaler
Head file size	20 KB (.npz, both stages)
Precision	float32
GPU required	No (CPU-only)
License	MIT

System Requirements

• CPU: Any modern CPU (no GPU needed)
• RAM: ≥4GB
• Python: ≥3.10
• Dependencies: numpy, model2vec, scikit-learn, pdfplumber, pypdfium2, Pillow

Citation

@software{poster_sentry_2026,
  title = {PosterSentry: Multimodal Scientific Poster Classifier},
  author = {O'Neill, Jamey and Portillo, Dorian and Zeinali, Nahid and Soundarajan, Sanjay and Blake, Gerard and Sarin, Parth and Buttrick, Adam and Patel, Bhavesh},
  year = {2026},
  version = {1.1.0},
  url = {https://huggingface.co/fairdataihub/poster-sentry},
  note = {Part of the posters.science initiative}
}

License

This model is released under the MIT License.

Acknowledgments

• FAIR Data Innovations Hub at California Medical Innovations Institute (CalMI²)
• posters.science platform
• MinishLab for the model2vec embedding backbone
• HuggingFace for model hosting infrastructure
• Funded by The Navigation Fund (10.71707/rk36-9x79), "Poster Sharing and Discovery Made Easy"