Large language models (LLMs) are more likely to criminalize users that use African American English, the results of a new study reveal. The study, conducted by Cornell University, delved into the “covert racism” embedded in these deep learning algorithms used to generate human-like texts.
Language Dialects and AI Bias
The study found that the dialect of the language a person speaks can significantly influence the AI’s perception of their character, employability, and even criminal tendencies. This bias was particularly evident when the AI detected dialects, showcasing a concerning trend in how language models interpret different linguistic variations.
Image for illustrative purposes
Impact of Large Language Models
Notable examples of large language models include OpenAI’s ChatGPT and GPT-4, Meta’s LLaMA2, and French Mistral 7B. These models have been under scrutiny for their predisposition to associate African American English with negative attributes, such as criminal behavior, without any explicit mention of race.
The study employed matched guise probing, prompting the LLMs with both African American English and Standardized American English to identify characteristics of the speakers. Shockingly, the GPT-4 technology was more inclined to “sentence defendants to death” based on the English commonly used by African Americans, highlighting a grave concern regarding the algorithm’s decision-making processes.
Unveiling Covert Prejudice
Researcher Valentin Hofmann from the Allen Institute for AI emphasized that the size of the LLMs plays a role in their understanding of African American English. While larger models may avoid overtly racist language, they still exhibit covert prejudice, perpetuating harmful stereotypes and biases.
Hofmann cautioned against interpreting the reduction in overt racism as a sign of progress, warning that the underlying racial bias in LLMs remains a pressing issue. The study revealed that conventional methods of training these models, such as providing human feedback, do little to address the covert racial biases ingrained in the algorithms.
Addressing the Challenge of AI Bias
As AI continues to permeate various sectors, including business and law, the need to address and rectify bias in language models becomes paramount. The study’s findings underscore the urgency of implementing robust measures to mitigate racial prejudice in AI systems and promote fairness and equity in algorithmic decision-making processes.
In conclusion, the study sheds light on the intricate relationship between language, bias, and AI, urging stakeholders to confront and combat the insidious manifestations of prejudice embedded in large language models. As technology advances, the ethical implications of AI bias must be addressed to ensure a more equitable and inclusive future.
