From DAN to Universal Prompts: LLM Jailbreaking

Since its release in late 2022, users have been attempting to "jailbreak" OpenAI's ChatGPT by crafting prompts that trick the model into providing unsafe or controversial responses. This has led to the emergence of prompt engineering as a field and the development of more sophisticated jailbreaking techniques. Researchers have proposed methods for generating universal adversarial prompts, which can be used to consistently elicit harmful or biased responses from language models. The introduction of blackbox jailbreaking presents an even greater challenge, as it does not require access to the target model's architecture or parameters. These developments highlight the need for updated security measures and continuous improvement in AI safety mechanisms.