CVPR 2024 Datasets and Benchmarks – Part 2: Benchmarks

In this article, three innovative benchmarks from CVPR 2024 are highlighted to showcase their potential impact on computer vision research. These benchmarks include ImageNet-D for testing the robustness of image classifiers against real-world perturbations, Polaris for assessing the ability of vision-language models to follow natural language instructions in interactive environments, and VBench for evaluating text-to-video generation models across multiple dimensions. Each benchmark presents unique challenges and opportunities for researchers, pushing the field towards more robust models. The article also discusses the design process, evaluation metrics, and potential impact of these benchmarks on future research directions.