Doc2LoRA provides decodable representations of scientific ideas

A map built for search

An embedding places a paper as a point in a vector space, turning a body of literature into a map of science. Papers on similar topics cluster together, making it easy to find related work or recommend what to read next. Most embeddings are optimized for search. Reading how papers cluster on this map helps us understand the current landscape of scientific knowledge production.

Beyond search, this map invites generation: mixing papers through simple vector operations creates new points on the map, mirroring combinatorial novelty, the recombination of existing ideas into new ones. The key question we ask is: what idea does a new mixed point represent?

Standard embeddings can't answer this directly because the components of the vector have no clear interpretable meaning. Usually, we check which papers are closest and try to infer the meaning, but this heuristic relies on the premise that nearby papers accurately represent the interpolated area, which does not always hold. Crucially, the most interesting regions—combinations that haven't been tried—are precisely where there are no actual papers.

Every point becomes a model

We use Doc-to-LoRA, a hypernetwork that reads a document and creates a LoRA adapter: a small module that changes the output of a language model as if it had read the document.

While large language models have many parameters, the embedding is compact as it contains only the information about the document. It offers idea arithmetic: any point—single document, group average, or an interpolation—can be turned into a LoRA adapter and loaded into a language model for direct question answering.

For example: we embedded two recipes, Italian cacio e pepe and Japanese kake udon. The Italian adapter produces “Pecorino Romano” as the cheese. Halfway between the two, the model describes “Kake Udon with Creamy Parmesan Sauce”—mixing noodles from one and cheese from the other.

Map generation results

We evaluated on American Physical Society physics papers labeled by PACS, a hierarchical topic scheme for organizing fields and subfields.

Naming groups of papers

For each topic, we averaged paper embeddings and asked the model to generate a 2-3 word label. We compared these outputs to official PACS names and five baselines (including keyword extraction and vector-to-text decoders). Doc2LoRA's labels best matched the official names by word overlap and were preferred in language model panel comparisons.

Shorter vectors, broader ideas

Averaging divergent papers shortens the embedding vector. Shorter vectors tend to be more generic, broader categories, and the level of generality can be controlled by shrinking the vector length (right). Shrinking a Wikipedia article's vector shifts its label from Citric acid cycle metabolism to Cellular respiration. Near zero, the model defaults to its base answer.

Reading the space between two papers

We interpolated between two physics papers and, at each point, asked the model to write an abstract. Move the slider to shift the weight from paper A to paper B. Doc2LoRA's abstract follows the weight and combines terms from both papers near the switch, while a language model given both papers and the weight in its prompt blends them at every weight.

Adding g greatly improves search

Doc2LoRA is designed for text generation, not ranking. Adding a small reversible transform, g, trained on 42,332 citation pairs from OpenAlex, improves its search results to the level of the commonly used text embeddings. g is the only trained part and preserves reversibility, so all points can be decoded as before.

We evaluated Doc2LoRA (with and without g) against five common text encoders on 14 benchmarks covering four tasks: next-paper prediction, topic classification, collaboration prediction, and author disambiguation. Without g, Doc2LoRA ranks last. With g, it performs on par with EmbeddingGemma and GTE, ahead of Instructor and SPECTER2, and behind SBERT.

Where this goes next

Traditional maps of science show only published work. Here, we built a new map that also covers unexplored areas—spaces between fields and combinations of ideas that haven't been tried. We can target gaps directly: What problem would a paper here address? What methods and experiments could bridge the divide?

This enables new questions: Can we trace a path across disciplines and see how concepts connect? Can we limit the map to papers up to a certain date, generate ideas for the "knowledge holes," and later see if those predictions appear in new research?

We must emphasize that our map reveals potential ideas, but verifying their validity is left for future investigation. The map does not assess how promising an idea is. That judgment is up to people. Yet, we believe that the Doc2LoRA embedding is a powerful tool for generating novel research questions and invites scientific curiosity.

Citation

@misc{mansuri2026doc2lora,
  title         = {Doc2LoRA Provides Decodable Representations of Scientific Ideas},
  author        = {Mansuri, Chand Sahil and Zachariah, Joel and Kojaku, Sadamori},
  year          = {2026},
  eprint        = {2609.38374},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CL},
  url           = {https://arxiv.org/abs/2609.38374}
}