Large-scale language models like ChatGPT often use Wikipedia content for training, so they risk amplifying this bias.
New York, New York — Over the past two decades, Wikipedia has evolved into an essential information resource for millions of users around the world, attracting over 4 billion visits per month. However, its importance extends beyond direct human readers, as Wikipedia content is also routinely used to train large-scale language models (LLMs) such as ChatGPT. Because Wikipedia’s reach and influence is immense, the accuracy and neutrality of its content is crucial. There is a risk that biases present in Wikipedia content will be absorbed into the underlying parameters of modern AI systems, further perpetuating and amplifying biases.
In a new report for the Manhattan Institute, David Rozado details findings from a groundbreaking analysis of political bias in Wikipedia content. In his assessment, Rozado uses a computational approach to assess the sentiment and emotional tone associated with politically significant terms in Wikipedia articles (such as the names of recent U.S. presidents, U.S. congressional representatives, U.S. Supreme Court justices, or prime ministers of Western countries). The results show that Wikipedia entries tend to associate negative sentiment with terms expressing a right-wing political orientation more than with left-wing terms. Furthermore, terms suggesting a center-right political position are more frequently associated with sentiments of anger and disgust than with terms suggesting a center-left position. Conversely, terms associated with left-wing ideology are more frequently associated with sentiments of joy than with terms associated with right-wing ideology.
Rozado found that some of the politically biased sentiment associations embedded in Wikipedia articles also showed up in OpenAI’s language models, suggesting that bias in Wikipedia’s content may seep into widely used AI systems. Given Wikipedia’s important and valuable role as a public resource, Rozado points out areas where Wikipedia could improve how it presents political information by upholding and strengthening Wikipedia’s principles of neutrality and impartiality.
Click here to see the report.
to donate
Are you interested in supporting Manhattan Institute’s public interest research and journalism? As a 501(c)(3) nonprofit organization, donations to support the research of MI and its researchers are fully tax-deductible as required by law (EIN #13-2912529).
