Download PaperAtlas

The files below are the July 2026 release, the version described in the manuscript. Releases are versioned by date and earlier releases are kept. What PaperAtlas is · browse the atlas.

Files

Build it yourself

Every large artifact is derived from public data. The whole chain runs from the PubMed Central open-access bulk distribution plus the PMCID list above, using the code in the repository. Only the embedding step wants a GPU; everything else is CPU work.

1. Get the source XML. Download the PubMed Central open-access bulk packages, about 641 GB, from ftp.ncbi.nlm.nih.gov/pub/pmc/oa_bulk. This is the only bulk download you need; nothing here is scraped from the web.

2. Parse the corpus. About 20 minutes on 10 workers, roughly 900 papers per second, writing ~16 GB across 22 shards. Resumable per shard.

python3 parse_v2.py --ids computational_ids.txt.gz \
                    --pubmed-root ./pmc \
                    --out ./corpus_v2 --workers 10

3. Build the structured store. This produces the 3.0 GB DuckDB file: papers, parsed references, the internal citation graph, accessions and repository URLs.

python3 build_db.py --corpus ./corpus_v2 --out paperatlas.duckdb

4. Chunk for retrieval. Section-aware passages, 30,913,457 of them at a median of 306 tokens.

python3 chunk.py --corpus ./corpus_v2 --out chunks

5. Embed and index. Start Qdrant, then load in tiers so the index answers queries before the whole corpus is in. This is the long step: 10.56 billion tokens to embed.

python3 index.py create
python3 index.py load --tier tier1     # every paper's profile, then Methods and Results
python3 index.py load --tier tier2
python3 index.py load --tier tier3
python3 verify_complete.py             # checks the collection against the chunk files

Request the large datasets

This keeps the corpus from being redistributed in ways the source licences do not permit, and lets us tell you if a newer release would suit you better. We do not pass your details to anyone.

Licence and reuse

Derived records — tool names, artifact types, cluster assignments, summaries, accessions and the citation graph — are released under CC BY 4.0. Please cite the paper below.

Verbatim passage text carries the licence of its source article and is redistributed only where that licence permits. Of the 1,074,191 papers in the corpus, 77.42% carry terms permitting commercial reuse, 18.94% carry non-commercial terms, and 3.64% (39,120 papers) carry no reuse licence and are excluded from redistribution entirely. Every record ships with its licence class, so you can filter to the terms you need. The per-paper assignment is in the supplementary tables bundle.

Datasets recovered from the corpus

The same extraction pass recovered 499,531 dataset accessions and 152,097 code repositories, each linked to the paper that produced it. Most point to public datasets that carry no index of their own and are cited only a handful of times. They are included in the structured companion store and in the atlas bundle, and are released for reuse, benchmarking and model training under the same CC BY 4.0 terms as the rest of the derived records.

Citation

Shah N, Vippagunta A, Bhargava Y. PaperAtlas: an automatically constructed atlas of computational biology tools from 6.4 million open-access articles. 2026.