[AARR] Self-Retrieval: Building an Information Retrieval System with One Large Language Model

Self-Retrieval is a novel architecture for end-to-end information retrieval that utilizes large language models (LLMs). It improves the efficacy of downstream applications and outperforms previous retrieval methods. The proposed system integrates LLMs into storing the corpus to be retrieved by internalizing the documents and creating a natural language index. Self-Retrieval consists of three steps: indexing, retrieval, and self-assessment. This design allows a single LLM to entirely execute the retrieval task. Compared to sparse and dense retrieval baselines, self-retrieval shows an average improvement of 11% in MRR@5. Further investigation is needed to understand the scaling law heading the link between document size and model parameters.