Conquering ControlNet

ControlNet has emerged as a successful project in machine learning (ML) in 2023. It provides an easy and interpretable way to influence the outputs of diffusion models through input maps, such as Canny edge maps, segmentation masks, pose keypoints, and scribbles. One of its key features is accessibility; ControlNet models are only 1.45GB in size, making them trainable at home on a single GPU in just 600 GPU hours. Despite the success of ControlNet 1.0, it suffered from bugs that were resolved in version 1.1. The major change was improving data quality, emphasizing the importance of high-quality data for state-of-the-art performance. This blog post provides a step-by-step guide on how to clean and curate high-quality data for training ControlNet models from scratch.