MDPI ROR Search

An open-source CLI tool for matching raw affiliation strings against the Research Organization Registry (ROR) using local NER and embedding models.

Features

• ror-dump: Downloads the latest ROR data dump from Zenodo, computes dense embeddings, and writes a compact Parquet file ready for search.
• mdpi-ror-search search: Loads the Parquet file into an in-memory numpy vector store and returns all matches above a configurable similarity threshold.
• mdpi-ror-search topk: Same as above but returns the top-K matches per affiliation.

---

User Guide

Installation

uv sync

Python 3.12 is required.

Configuration

All settings - model names, batch sizes, thresholds, device, and Zenodo record ID - are loaded exclusively from environment variables or a .env file. Copy the provided example and fill in your values before running any command:

cp .env.example .env

Edit .env with your local settings. This is the single place to change configuration; there are no CLI flags for these values.

Key variables:

Variable	Description
NER_MODEL_NAME	HuggingFace model ID for the NER step
EMBEDDING_MODEL_NAME	HuggingFace model ID for dense embeddings
EMBEDDING_BATCH_SIZE	Batch size for embedding inference
NER_BATCH_SIZE	Batch size for NER inference
ZENODO_RECORD_ID	Zenodo record to fetch; update when a new ROR release is published

For the full list of available settings and their defaults see mdpi_ror_search/src/config.py.

NER model requirements

The NER model must output entity groups named ORG, SUB, CITY, and COUNTRY.

Quick start

1. Build the Parquet file

Download the latest ROR dump from Zenodo and embed it:

uv run ror-dump --output-dir ./data

Or use an existing JSON file:

uv run ror-dump --json-path ./ror-data.json --output-dir ./data

2. Search affiliations

Return all matches above the configured similarity threshold:

uv run mdpi-ror-search search \
    --parquet-path ./data/ror_collection.parquet \
    --affiliations-file ./data/affiliations.txt \
    --output ./data/results_search.json

Return the top-3 matches per affiliation:

uv run mdpi-ror-search topk \
    --parquet-path ./data/ror_collection.parquet \
    --affiliations-file ./data/affiliations.txt \
    --top-k 3 \
    --output ./data/results_topk.json

Pass --affiliation "some string" (repeatable) instead of --affiliations-file to supply affiliations directly on the command line.

Memory requirements

A full ROR dump produces approximately 550 000 rows. At 768 embedding dimensions the numpy matrix alone requires ~1.7 GB of RAM. Use a smaller embedding model (e.g. 384 dimensions) to halve this footprint to ~850 MB.

---

Contributing

Development setup

Install all dependencies and pre-commit hooks:

make install
make install_pre_commit

Quality checks

Run tests, ruff, and deptry:

make check

---

License

Apache 2.0