RankVicuna: Zero-Shot Listwise Document Reranking with Open-Source Large Language Models

In this paper, the authors propose RankVicuna, an open-source model for document re-ranking that achieves comparable performance to proprietary models like GPT-3.5 and GPT-4 while being significantly smaller in size (7 billion parameters compared to 175 billion). The model is deterministic, ensuring consistent output format and rankings across different runs. RankVicuna uses a teacher-student paradigm for data augmentation, generating query-document pairs from a larger model and shuffling the input order of documents to provide more examples for training. The authors also highlight the importance of prompt engineering in achieving stable results. Overall, this paper showcases the potential of open-source large language models for document re-ranking tasks and emphasizes the role of data augmentation and prompt engineering in improving model performance.