validation study
Comparing AI and human panel
In 2024, we compared results from our AI panel with archive data and with human respondents in dozens of comparison projects, both at our partner universities and current enterprise clients.
Tl,dr:
Our AI respondents are just as accurate as the real ones. There were no large differences in responses from an AI panel and human panel.
Academic and commercial
In 2024, we ran dozens of comparison projects, academic and commercial. Read about comparing our AI panel in Skoda Auto or Raiffeisenbank.
We were invited to present this study at two academic conferences - ISPIM in Tallinn, June 2024 and ESA is Porto, August 2024.
status quo
Online panels are biased.
Online panels are often used to benchmark AI panels. However, online panels suffer from low representativeness as indicated by the latest research.
LLM for synthetic samples
ChatGPT: Inaccurate & safety risk
ChatGPT was built to mimic human language, not run heavy computations needed for accurate synthetic or silicon samples.
Training data: No niche groups
ChatGPT ignores the variance found in society = at best, you end us with the “average US citizen on the internet”, not your target group.
The world of ChatGPT is too flat and too skewed to correspond with our world. No prompting can change that.
Cao et al., 2023
Lakmoos AI can zoom onto niche groups.
Sampling: No random distribution
Even if ChatGPT had data with variance you need, it would not choose the right respondents for your survey.
ChatGPT is not capable of random distribution = no random sampling = offers random insights regardless the data it knows.
Liu, 2024
Lakmoos AI can draw random samples.
Adding data: Safety threat
Adding your distribution and your data to ChatGPT (e.g. RAG approaches) helps, but presents a huge data security threat.
ChatGPT uses your private data to train itself and by extension shows your cards to your competitors.
Falade, 2024
Lakmoos AI keeps client data private.
status quo
Neuro-symbolic AI
Frankly, it is just a fancy term for a lot of maths and probabilistic reasoning. It means that we combine different types of analyses to achieve the optimal result. When use asks, the questions travels to
analytical core where we compute the answer which
is interpreted in our LLM (large language model like ChatGPT)
This approach combines the best of both worlds: computing power and readability of results.
Seth et al., 2024
“
“The collaboration between human researchers and AI tools will lead to a new era of market research—more efficient, accurate, and insightful than ever before.”
Said our CEO Kamila for Planung & analyse: How AI can improve market research
Imagine the scope of our proprietary technology
For the U.S. with 332M inhabitants, we have a model with 332M digital respondents. For 10M in Czechia, we have a model with 10M AI respondents.
Each digital respondent is formed by hundreds of thousands of datapoints, ranging from age to risk aversion.
We combine hundreds of data sources some updating in real time, some only private for a particular client.
In 2024 alone, we answered tens of thousands of questions in our AI panel.
Plugging Generative AI (GenAI)?
GenAI is the type of AI most people are familiar with.
ChatGPT and other LLM (large language models) trained for text-based tasks
Midjourney for generating photos
HeyGen for generating videos
Lakmoos AI does not use LLM to compute the answer, but it uses its own LLM to translate numbers into a human-sounding response.
Not using LLM for data = accuracy
Proprietary LLM = data security
Adding LLM as the last step = readable results
Rule of thumb
If you need to handle numerical tasks, choose predictive AI.
If you need to handle text-based tasks, choose a LLM, generative AI.
If you need to handle images or video, use generative AI.
If you need all of the above, choose neuro-symbolic AI.
chatgpt is not enough
AI beyond ChatGPT
We are not a ChatGPT wrapper. Two reasons: accuracy & data security.
ChatGPT was never meant to simulate data. It was built to perform text-based tasks. Asking it to sample data is like asking an Excel spreadsheet to write a poem.
Asking ChatGPT for public opinion is like asking a neighbor - it’s not representative of a wider group. Why will it not give accurate answers?
No random sampling
Even if ChatGPT had relevant data, it cannot draw a random sample. The sample will always be skewed in an unpredictable direction.
Cultural differences
ChatGPT performs better in US context and average population, not other countries and niche groups. You rarely define products for the average US citizen, thus their best guess is not relevant to your target group.
Safety threat
ChatGPT will learn use your data to train itself. Most companies do not fancy sharing their data with other parties and their competitors.
Lakmoos AI computes the public opinion. It does not hallucinate, it does not guess, it does not try to sound human.
Analytical core
Our answers are computed with a series of analyses that optimise for accuracy.
Quantification
Lakmoos AI does not give you a random opinion, we give you the distribution. We quantify how many people agree with what statement. Also, we provide you with the “why”: motivations, worries and needs behind the % distributions.
Data security
Each enterprise has their own private data layer in a dedicated instance or on premise. Their data stays theirs. We do not train our models on those.