Co-written by Adam Bai, Chief Strategy Officer of Glimpse, an AI-powered human response platform, and Vignesh Krishnan, Founder and CEO of Research Defender, a research fraud detection and Data Quality tool. Or, the story of Glimpse was tricked by Generative AI in response to a survey about Generative AI–while working for one of the world’s leading Generative AI companies.Read more to find what happened and learn some concrete tips to defend your own research against similar threats . . .
Or, the story of Glimpse was tricked by Generative AI in response to a survey about Generative AI–while working for one of the world’s leading Generative AI companies.
Read more to find what happened and learn some concrete tips to defend your own research against similar threats . . .
One of us (Adam) leads Strategy and Client Services for Glimpse, a small but rapidly growing AI-powered research start-up, focusing on the language, emotion, and sentiment of the audiences we recruit for client organizations.
So when a sequence of extremely thoughtful responses to a recent survey question started rolling in, we were understandably pleased. Especially because our client, a highly respected research firm, was running the study on behalf of their client, one of the world’s largest tech companies.
The study was about the perceptions of Generative AI amongst 600 IT decision makers in the United States and this was the final question:
“Do you think Generative AI will impact your industry? If so, how?”
So this response, for example, seems pretty great at first glance:
“Yes as we run a contact center with a multiple level switch. AI could help filter chats to few automated chats, and phone trees with voice response. It can also build bots for adherence and performance contacts to agents on thresholds.”
At Glimpse we pride ourselves on data quality and integrity and employ both automated filters and trained human eyes to ensure our clients only pay for meaningful answers from real people.
The response passed all of our tests.
The only problem? It was created by Generative AI (most likely ChatGPT)–in response to a study about Generative AI for one of the most important organizations engaged in Generate AI research!
For the first (and hopefully last!) time in the history of Glimpse, a client discovered a fraudulent response written by a machine.
In retrospect, the irony is pretty funny.
At the time, we were too busy trying to figure out what hit us and how to deal with it.
Just like anything else that happens primarily over the Internet, the market research industry has its fair share of bad actors and fraud. The emergence of digital surveys and programmatic recruitment techniques led to unprecedented speed, agility, and cost savings across the entire research ecosystem.
But it also led to an escalating arms race between bots and survey farms trying to capture the incentives paid to respondents, on one hand, and brands and research companies, on the other. After all, it’s tough to tell if the person who chose, ‘C,’ on a digital survey is, in fact, a person. (That’s one reason why Glimpse clients include at least one open-ended question in each study.)
The consequences are potentially enormous. Organizations end up making important decisions based on invalid data, fake opinions, and random responses.
Glimpse’s trusted partner, Research Defender, was founded by an industry veteran (the co-author, Vignesh) to directly combat this threat. Research Defender helps clients like Glimpse to eliminate fraudsters and bad actors and to measure respondent engagement in real-time to weed out inattentive respondents.
According to Research Defender’s data, their service eliminates substantial amounts of fraud across all of their clients:
Bad Actors (bots/click farms/international fraud) Around 10 to 15%
Duplicates From 2% to 20%
Text Fails Around 5%
Activity Fails From 7% to 10%
And now we face the possibility of real human respondents using Generative AI to respond to open-ended questions. At the recent Quirks market research industry event in Chicago, a presentation about this emerging threat attracted the largest audience and a lot of urgent questions.
So is it time to give up?
Nope! Here’s why . . .
When Glimpse’s client detected Generative AI-written responses to their Glimpse survey, we moved quickly to rescue the study and to find a long term solution.
Here’s what we did:
● Manually fed each response into a Generative AI detector (itself powered by Generative AI) and eliminated the fraudulent ones, a process soon to be totally automated
● Eliminated the sample suppliers who were providing the problematic responses
● Implemented more advanced copy/paste detection to determine if a response was created elsewhere and then pasted into Glimpse’s text chat-like survey
● Implemented a second proprietary digital fingerprint in additional to the one provided by Research Defender
● Activated more sophisticated Research Defender filtering
And here are Research Defenders’s general tips for responding to the Generative AI threat:
Copy-and-paste behavior, engagement, and speed/minute are still important indicators of fraud. Most importantly, we should follow the behavioral data where it takes us. Even though we’re only months into the Age of Generative AI, we already know a lot about how people tend to take surveys. We can use the ‘wisdom of the crowds’ to track behavior changes that might be consequential.
As always when we confront emerging technologies, we should totally abandon our existing practices. For example, basic checks like depreciation, printing, and repeated respondents can still be effective. After all it’s professional survey takers who will be most prone to using tools like ChatGPT or Bard at scale.
In the Glimpse story, in fact, we saw that many of the AI-generated responses could be traced to a single set of users (at a ratio of almost six or seven to one). We now know how to detect users and responses like this far more effectively.
Many platforms are able to perform contextual analysis, which as the name suggests, allows us to look at the context of what is being described and determine if the response appropriately aligns with the question. Again here, we can look either at pre-programmed answers for every record (not very scalable), or at comparative answers (more scalable).
There’s another irony here: Glimpse recently released an industry leading GPT-4 integration. Yes, the very technology we described as a threat above.
Glimpse users can now summarize and capture the essence of open-ended responses with the touch of a button. And they can filter by any audience segment or demographic/behavioral attribute.
They can discover instantly how Korean women between 25-35 talk about electric vehicles. Or how male gamers in California with graduate degrees feel about Valorant.
We think this is a game-changer for advertising, PR, creative agencies, CPG and technology companies, and market research firms who rely on human insights to make informed decisions about their products, services, and campaign concepts.
And GPT has another important application in this context: It is precisely technologies like GPT that will enable all of us, including Glimpse and Research Defender, to detect fraud in a more accurate and sophisticated manner.
When it comes to market research, there has never been an emerging technology that represents such extraordinary opportunity and extreme threat at the same time as ChatGPT (and other Large Language Model-based applications).
The challenge, as always, is working with companies like Glimpse to seize the opportunity while working with companies like Research Defender to combat the threat.
Integrating AI into something as deeply personal as meditation highlights the transformative potential of AI, signaling its profound impact across industries in the years to come. The key challenges for brands will be finding the right balance between technology and tradition. Some consumers may embrace AI for its accessibility and customization, while others will resist the erosion of human connection.
As industry giants enforce full-time office mandates and the rest of the corporate world feels pressure to follow, a staggering 28% of employees are ready to quit rather than give up the flexibility. This shift highlights not only the changing dynamics of work but also the need for companies to stay ahead of trends affecting employee engagement and digital transformation.
America just had a big night of presidential debate, but do the issues discussed reflect what voters really think? Our recent survey, conducted before the debate, uncovers surprising trends that are already shaping the upcoming election.
