The NeurIPS 2024 Preshow: Using Knowledge Graphs to Diagnose and Debias Visual Datasets

The ConBias framework, presented in a recent paper at NeurIPS 2024, is a novel approach to diagnose and mitigate object co-occurrence biases in visual datasets. These biases can negatively impact the performance of deep learning models by introducing spurious correlations between class labels and concepts that are causally unrelated. The authors argue that representing the dataset as a knowledge graph of object co-occurrences allows for a structured and controllable method to diagnose and mitigate these biases. The ConBias framework involves three steps: concept graph construction, concept diagnosis, and concept debiasing. Concept graph construction maps the co-occurrence relationships between objects (concepts) and class labels, while concept diagnosis uncovers imbalanced concept combinations across classes. Finally, concept debiasing rectifies these imbalances by generating new images with under-represented concept combinations using a text-to-image generative model. The framework has been demonstrated to consistently improve the generalization performance of classifiers on multiple datasets, outperforming traditional augmentation techniques and state-of-the-art methods for data debiasing. Its effectiveness lies in its targeted approach, which focuses on identifying and addressing biases within the dataset, rather than relying on external language models that may introduce new biases.