Security researchers at the company Irregular have concluded that generative AI struggles significantly when tasked with creating robust passwords. The investigators put Claude, ChatGPT, and Gemini through rigorous testing. All models received the identical instruction: to produce a 16-character password incorporating uppercase letters, digits, and special characters. Initially, the outcomes appeared promising; online complexity checkers rated them as “very strong passwords,” even suggesting “hundreds of years” would be needed to crack them. However, this proved to be a façade. The core issue lies in the fact that these checkers fail to account for the inherent patterns language models tend to establish. Malicious actors, conversely, can easily exploit these patterns. According to Irregular’s findings, all tested AI models consistently generated passwords featuring recurring structures, most noticeably at the beginning and the conclusion of the string. For instance, across 50 separate prompts given to Claude (specifically the Opus 4.6 model), the researchers only received 30 distinct passwords. Of those, 18 were completely identical. Furthermore, nearly every generated sequence started and ended with the same characters. Not a single one of the 50 instances contained any repeating characters, which strongly suggests predictability over true randomness.
Comparable patterns were identified when testing OpenAI’s GPT-5.2 and Gemini 3 Flash. Even when the Nano Banana Pro model was explicitly instructed to “write a random password on a sticker,” Gemini’s characteristic templates persisted.
The Register replicated the experiment using Gemini 3 Pro. The model provided three output options: one optimized for “high complexity,” one emphasizing “symbols,” and a “random alphanumeric” one. The first two adhered to discernible patterns, whereas the third appeared genuinely random. Interestingly, Gemini explicitly advised users not to employ these generated passwords for sensitive accounts, instead recommending the use of dedicated password managers such as 1Password or Bitwarden.
Irregular proceeded to quantify the entropy (the measure of randomness) within these generated passwords. For 16-character strings produced by the LLMs, the entropy measured approximately 20–27 bits. A truly random password of the same length should exhibit an entropy level closer to 98–120 bits.
In practical terms, this low entropy implies that these AI-created passwords could theoretically be brute-forced within mere hours, even using older computing hardware. An additional related vulnerability is that these predictable patterns can reveal when passwords were generated by AI tools. Searches for these characteristic symbol sequences on platforms like GitHub already lead to documentation, instructions, and test projects containing such strings. Irregular posits that as the popularity of “vibe coding” and automated code generation increases, this vulnerability is likely to worsen. If AI ends up writing the majority of software code—a future previously suggested by Anthropic CEO Dario Amodei—then the weak passwords created by these models could become pervasive across numerous projects.
About the Author
