ZDNET’s key takeaways
- Americans use AI to search for information – including elections.
- Voters ask chatbots for basic election information and candidate recommendations.
- AI systems can hallucinate and add layers of bias to political information.
Most search tools now put a layer of AI between you and the information you’re looking for, especially Google.
Also: Tired of AI Overviews? I found 9 Google Search alternatives
What voters asked chatbots
The researchers asked nearly 2,500 people in 35 states with Senate races to query ChatGPT, Claude, Grok, or Gemini about the election. Participants in the study, which ran for a few weeks in October leading up to the Nov. 3 midterm elections, were adults who already use chatbots weekly.
Most users asked chatbots for basic facts about the election, including who was on the ballot. But 36% asked a chatbot to help them pick a candidate to vote for, often by describing their priorities and asking the chatbot to assess which candidates aligned with them.
The former query has slightly less room for error, because the internet has a ton of data on the basic facts of an election, such as who is running where. Chatbots interpreting candidate positions, on the other hand, is a bit more dicey, and relies significantly on the sources a chatbot uses (a statement from a candidate’s own website, for example, is more reliable than a social media clip of an opponent characterizing them in a certain way).
Also: How to keep your chatbot conversations as private as possible
In one example, a user asked ChatGPT to give them the “pros and cons” of each candidate in Virginia, an inherently subjective question.
When it came to recommending candidates, chatbots varied widely.
“When users asked who to vote for but had not yet said anything about themselves, only Grok regularly recommended a candidate, in 14% of chats,” the researchers wrote. “Once users shared their views, ChatGPT recommended a candidate in 79% of chats and Claude in 40%, with Grok and Gemini in between.”
AI and elections
Pew Research found that 42% of US adults surveyed use chatbots to search for information. That’s a broad umbrella that can include election information — but Pew also found that only 8% of US adults reported using chatbots to find election-related information in this year’s election.
What those users are looking for matters, though. Pew found that of that group, 76% were inquiring about where candidates stand on certain issues, and 71% were fact-checking information they’d heard about candidates or issues.
Also: OpenAI will merge ChatGPT’s chat and work tabs
Given that chatbots are still not reliable summarizers, the nature of these questions poses a misinformation risk if users don’t also check a chatbot’s linked sources. An overwhelming majority of users don’t, at least when it comes to Google’s AI Overviews. At least one study found AI platforms gave users “outdated or inaccurate” information that could mislead voters in some cases by missing a deadline or failing to mention a step about voting logistics.
This election cycle was <a href="https://airisk.mit.edu/ai-incidents/1702-russia-linked-matryoshka-influence-operation-reportedly” rel=”nofollow noopener” target=”_blank”>particularly plagued by AI-generated deepfakes and misinformation as tools have become much more sophisticated since the 2024 election. OpenAI and Anthropic reportedly set up election “war rooms” to monitor misuse and information accuracy, according to exclusive reporting by Politico and Semafor.
While an August study from the Brennan Center for Justice found that chatbots did push back surprisingly well against certain falsehoods related to election disinformation campaigns, it also found that AI models easily generated “misleading election-related images and video” when asked.
“Many of their safeguards for preventing this were ineffective, inconsistently applied, or functionally absent,” noted the study, which tested ChatGPT, Gemini, Grok, Meta AI, Runway, and Flux.2.
Also: 50 AI agents tried to build a thriving Minecraft civilization – it didn’t end well
Apart from these factors, however, there is a broader theme in AI use that is equally consequential for civic engagement: cognitive offloading. The political media landscape can already be dense and hard to parse for a voter trying to stay informed. Layering a chatbot — which has opaque training data sets, filters, and other settings determined at the discretion of AI labs — on top of that information search process can further muddy the waters while appearing like a convenient and neutral summarizing tool.
Why this study matters
While some studies test how AI chatbots respond to election questions by using standardized questions, this approach let users themselves engage with chatbots as they normally would. While the Yale researchers’ queries and standardized study queries show a lot of similarities, the distinction is valuable for evaluations.
“This is a small step toward evaluations that measure AI’s political influence on the questions users actually ask,” the researchers noted.
Outside of users self-reporting what they asked a chatbot, only AI labs have records of what users are really asking about elections. In October, Anthropic analyzed 250,000 conversations with Claude and determined people “rarely ask Claude whom or what to vote for,” finding that “0.51% of all conversations are specifically about elections or voting,” though it expected election-related queries to increase closer to Nov. 3.
Also: 6 AI agent workflows I use to hand off everyday tasks
“More often, we observed people asking Claude questions that could help them reach their own conclusions, like where candidates stand on a particular issue. Requests that violated our election integrity policies were even less common, appearing in about 0.002% of conversations,” Anthropic’s report said.
The Yale researchers said the next step for a study like theirs is determining how accurate and unbiased models are in their responses, as well as what models learn about users, and whether their recommendations actually match users’ needs.
“In multi-turn chats, users often answered the chatbot’s questions, and those answers led to its recommendations. The questions a model asks shape what users tell it and what it recommends,” the researchers explained.
