What policy does ChatGPT represent? Research shows that AI sits on the left side of the political spectrum.
The release of OpenAI’s ChatGPT catapulted the large language model technology into the public eye. It is already leading to changes in society, for example in the education system: schools and universities try to manage the possibilities of such systems – or to prohibit them.
Faced with the prospect that ChatGPT and other language models will be used by more and more people and will change the Internet searches of Google or Bing in the future, researchers are questioning the policy direction of ChatGPT.
“If anything is to replace the current stack of Google search engines, it’s future iterations of AI language models like ChatGPT that people will interact with on a daily basis for decision-making tasks,” the researcher said. David Rozado. “Language models that claim political neutrality and correctness (as ChatGPT does) while displaying political bias should be cause for concern.”
How political can AI be?
Rozado’s research shows which way OpenAI’s chat AI political pendulum is swinging. Shortly after ChatGPT was released at the end of November 2022, there were already several articles addressing AI trends: “ChatGPT Isn’t Politically Neutral”, “OpenAI Chatbot Spits Out Biased Musings, Guardrails Despite” , “OpenAI’s ChatGPT Bot Recreates Racial Profiling”
In fact, it wasn’t all that surprising: biases in AI models have long been a problem, including in recent generative text-image models like Stable Diffusion and DALL-E, which have been shown to they reinforced racial stereotypes.
Rozado subjected ChatGPT to various political bias tests three times, first on December 5 and 6, 2022, and then about two weeks later and after on January 9.
During the first test, he concluded that ChatGPT could be clearly classified as belonging to the “Establishment Liberals” group. However, a repeat of the tests brought ChatGPT closer to the political center, leading Rozado to believe that OpenAI was moderating ChatGPT at the political center.
“Something seems to have changed in ChatGPT and its political orientation test answers have shifted to the center in three out of four tests,” Rozado wrote in his newsletter. “It’s in stark contrast to just two weeks ago.”
Previously, in the Political Compass Test, ChatGPT had spoken out against the death penalty and free markets, but in favor of abortion, more taxes on the rich, government subsidies, welfare benefits for those who refuse to work and immigration, among others. .
According to Rozado, the most likely explanation for ChatGPT’s political orientation is that the system was trained on a large body of textual data from the Internet written by professionals, the majority of whom are politically left-leaning. Other possibilities for human influence exist in fine tuning and human evaluation of machine responses (RLHF).
Rozado tentatively drew a positive conclusion after December’s investigation into how OpenAI apparently corrects political tendencies in ChatGPT. “I appreciate the system which strives to be neutral and tries to provide arguments for both sides on different issues.”
But this tentative conclusion is now apparently premature.
New survey: ChatGPT is clearly on the left
After the post-January 9 analysis, Rozado revises this hypothesis: according to the new test, 14 out of 15 political orientation tests revealed left-wing views.
According to Rozado, his December 6, 2022 and December 24, 2022 analyzes of ChatGPT’s political biases were preliminary and based on limited data. The results are now more robust, he said, and he can say with more certainty that ChatGPT “indeed exhibits a preference for left-leaning responses to questions with political overtones.”
“Ethical AI systems should try not to favor certain political beliefs over others on largely normative issues that cannot be decided with empirical data,” Rozado says. “Most certainly, AI systems should not claim to provide neutral, factual information while displaying a clear political bias.”
Its conclusions are supported by a study conducted by German researchers on the political orientation of ChatGPT.
“Inviting ChatGPT with 630 political statements from two leading voting advice apps and the nation’s independent political compass test in three pre-recorded experiments, we uncover ChatGPT’s pro-environmental and libertarian ideology,” their post reads. published in early January 2023.