AI is making it harder to get quality data… but could also be the solution, research reveals

Researchers are struggling to write surveys that get enough quality data, according to a study of researchers and insights professionals we ran using Checkbox and Prolific. AI is a factor in data quality, but could also be a solution.

Everyone is talking about how AI is making work easier, but for researchers, it may be doing the opposite. Market researchers, insights professionals, and survey designers are facing ever-growing challenges, with data quality being one of them – and AI is having a real impact on this. 

Sophisticated operations, at times referred to as "click farms" or AI-agents, use generative AI and connected devices to flood surveys with machine-generated answers. As mentioned in a report by Ayda and RedDem, traditional safeguards such as IP screening and CAPTCHAs are no longer sufficient to detect bots that rotate IP addresses, spoof devices, or mimic human behavior using AI-generated text.

According to recent data, over a third of businesses have encountered AI-related fraud in their research processes, from fabricated survey responses to bot infiltration.

Our study reveals similar results, with a large number of respondents reporting "responses completed using AI" as one of the major data quality issues they frequently face, together with "misunderstanding the question" and "incomplete responses".

AI-generated responses, if left undetected, may have real consequences on business decisions and research outcomes. Beyond inaccurate results, they threaten the trust researchers place in their data. When it becomes harder to distinguish between authentic human responses and AI-assisted ones, researchers are forced to question not just individual answers, but the reliability of entire studies.

Tackling fraudulent responses in a world of AI

Some organizations have already taken steps to recognize AI-generated or fraudulent responses. When asked about these measures, most participants rely on attention check questions and pattern recognition in answers. 

However, not everyone has adopted techniques to identify AI respondents yet, with a quarter that have no need or no technique to identify them.

One respondent said, "We don't really have anything to track AI responses to user responses, though we check responses manually."

In fact, we found that 18% of people who cited AI responses as a data quality issue currently have no way to identify them. This stat highlights a gap between the scale of the problem and the tools researchers are using currently, or the recognized need for them, leaving many teams exposed to undetected AI interference in their data.

Could AI be the solution for the problem caused by AI?

With new data quality issues come new technologies to resolve these problems. AI response detection tools are already a reality, with more and more researchers showing interest in using AI-powered fraud and bot detection technologies.

Among our respondents, 60% said they were "Very interested" or "Extremely interested" in adopting AI-powered fraud and bot detection in their workflow.

Unsurprisingly, those encountering AI responses as a data quality issue are far more likely to be interested in AI-powered fraud and bot detection than those who aren't. In fact, more than half (54%) of the respondents facing this data quality issue said that they were extremely interested in this emerging technology.

On the flip side, some respondents raised concerns about adopting AI – especially in fields that require strict data security and compliance: "Data privacy is my largest concern, with more platforms moving to integrate AI. We collect data from children, and are therefore not legally allowed to share this data. If it is not 100% clear that a platform is not using our data to train AI, we cannot use it."

Research is changing fast, and the rise of AI is forcing teams to rethink how they collect and validate data. As new risks emerge, researchers must evolve their methods to protect the integrity of their studies and the trust placed in their insights. While AI continues to push the boundaries of automation, human responses remain the foundation of meaningful, reliable research.

For help creating your own surveys, try Checkbox, which is fully customizable and has plenty of question type options. Request a Checkbox demo today.

300 insights professionals and researchers were surveyed for this data. Not all questions were compulsory.