AI models found to show language bias by recommending Black defendants be ‘sentenced to death’
Artificial Intelligence (AI) has become an integral part of various technologies, but a recent study sheds light on a concerning issue - language bias in AI models. The dialect of the language one speaks can significantly influence the outcomes predicted by AI, including character judgments, employability, and even criminal sentencing.
Unveiling Covert Racism in AI Models
A pre-print study conducted by Cornell University delved into the realm of large language models (LLMs) and their tendency towards covert racism. Large language models like OpenAI’s ChatGPT and GPT-4, Meta’s LLaMA2, and French Mistral 7B were scrutinized for their responses based on different dialects of English.
Researcher Valentin Hofmann from the Allen Institute for AI highlighted a troubling discovery - the GPT-4 technology was more inclined to “sentence defendants to death” when presented with prompts in African American English, without any explicit mention of the speaker’s race.
The study employed matched guise probing, revealing that AI models made assumptions about the speakers’ characteristics based on the type of English they used. Shockingly, the algorithms associated African American English speakers with less prestigious jobs, showcasing a clear bias in their assessments.
The Impact of AI Bias on Society
As AI systems incorporating LLMs are increasingly utilized in critical areas like business and jurisdiction, the presence of such language bias raises significant concerns. The study emphasized that while overt racism in LLMs might be diminishing, covert racial bias still persists, posing a challenge in combating discriminatory practices.
Hofmann cautioned against misinterpreting the reduction of overt racism as a resolution of racial bias in AI models. The study indicated that conventional methods of providing feedback to AI models do not effectively address covert racial bias, leading to a mere superficial concealment of underlying prejudices.
Image for illustrative purposes
Addressing the Root of Bias in AI
The study’s findings underscore the urgent need for a deeper understanding of how language bias manifests in AI models and the implications it holds for individuals affected by such biases. By acknowledging and actively working to mitigate these biases, the tech industry can strive towards more equitable and unbiased AI technologies.
In a world increasingly reliant on AI-driven solutions, it is imperative to address and rectify biases embedded in these systems to ensure fair and just outcomes for all individuals, regardless of their background or dialect. The journey towards unbiased AI is a collective responsibility that demands continuous scrutiny and corrective measures to build a more inclusive technological landscape.
