CVPR 2024 Datasets and Benchmarks – Part 1: Datasets

The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) is a significant platform for introducing new datasets and benchmarks in computer vision research. These datasets and benchmarks play a crucial role in advancing deep learning techniques by providing diverse, large-scale data and standardized tasks or challenges to evaluate model performance. Datasets are collections of data samples like images, videos, or annotations used to train and evaluate deep learning models. They provide the raw data that models learn from during the training process. Datasets often include labeled or annotated data to provide ground truth information for supervised learning tasks. Examples of famous datasets introduced at previous CVPRs include ImageNet (2009), Cityscapes Dataset (2016), Kinetics Dataset (2017), and nuScenes (2020). Benchmarks, on the other hand, are standardized tasks or challenges used to evaluate and compare the performance of different models or algorithms. They typically consist of a dataset, a well-defined evaluation metric, and a leaderboard ranking the performance of different models. Benchmarks allow researchers to gauge performance against state-of-the-art approaches and track progress in a specific task or domain. Examples include the 2009 Pedestrian Detection Benchmark and 2012’s KITTI Vision Benchmark Suite. The importance of datasets and benchmarks in deep learning cannot be overstated, as they serve as the foundation for training models and objectively assessing their performance. New datasets are essential to exploring novel tasks, accommodating emerging model architectures, and capturing diverse real-world scenarios. As models achieve higher – and eventually human-level – performance on established benchmarks, researchers create new, more challenging ones to push the boundaries of what's possible. At CVPR 2024, several new datasets and benchmarks have been introduced, each with the potential to advance progress in computer vision and deep learning. Three notable datasets are Panda-70M, 360+x, and TSP6K. These datasets cover a spectrum of computer vision tasks, including multimodal, multi-view, room layout, 3D/4D/6D, robotic perception datasets, and more. Panda-70M is a large-scale video captioning dataset that enables more effective pretraining of video-language models, driving progress on video understanding tasks. It overcomes the limitations of ASR annotations and provides a solution to manual annotations' cost and scalability issues. The 360+x dataset is a panoptic multimodal scene understanding dataset that covers multiple viewpoints (panoramic, third-person, egocentric) with multiple modalities (video, audio, location, text). It captures real-world complexity and provides a more realistic challenge for research. TSP6K is a dataset specifically designed for traffic scene parsing and understanding, focusing on semantic and instance segmentation in urban traffic scenes. These datasets address limitations in existing resources, provide rich and diverse data, and establish new performance benchmarks. They are poised to drive research forward and contribute to real-world applications like content creation, smart environments, and intelligent traffic management.