Artificial intelligence has become a ubiquitous term in marketing technology. Vendors, consultants and even media outlets commonly describe their measurement solutions as “AI-powered,” yet the underlying reality is often less compelling than the marketing suggests. This divergence between hype and substance carries real risks for brands relying on measurement tools to inform multimillion-dollar decisions.
At its core, marketing measurement, whether through media mix modelling (MMM) or attribution, aims to answer a fundamental business question: Which marketing investments actually drive profit? Yet many solutions touted as AI-driven fail to provide stable, causal insights that marketers can trust.
1. The Limits of AI Hype in Measurement
a) LLMs Aren’t Built for Causal Inference
Large language models (LLMs) such as ChatGPT are excellent at producing text that sounds confident and knowledgeable. However, they are fundamentally generative tools trained to predict plausible language sequences, not to deduce causal relationships between marketing inputs and business outcomes. That means relying on them alone for decisions like media budget allocation can be dangerously misleading.
For example, an LLM might confidently recommend increasing spend on a particular channel simply because its training data contains patterns that appear successful, without understanding whether that spend actually caused the uplift rather than merely correlated with it. This is a critical distinction in marketing measurement that goes beyond pattern recognition.
b) AI Washing in MarTech
This trend reflects a broader industry phenomenon known as AI washing — the practice of overstating the role or depth of AI in a solution to make it seem more advanced than it actually is. It’s similar to “greenwashing” in sustainability communications and can mislead buyers about what a tool can genuinely accomplish.
2. Where AI Can Be Valuable in Measurement
Although LLMs have limitations, if we define AI broadly to include advanced machine learning methods, such techniques are already entrenched in measurement practice. For instance, Hamiltonian Monte Carlo (HMC) and other Bayesian approaches help estimate complex models more efficiently.
Here’s where AI genuinely contributes:
- Enhanced Analytics Interpretation: AI tools can summarise complex model outputs in accessible language, helping non-technical stakeholders understand assumptions and results.
- Data Quality Checks: Machine-assisted systems can flag anomalies or inconsistencies in datasets that might otherwise go unnoticed, improving the robustness of insights.
- Workflow Efficiency: Automated processes can speed up repetitive tasks, allowing analysts to spend more time on strategy and validation.
However, these contributions enhance the process, rather than replace the statistical foundations needed for reliable causal measurement.
3. The Real Risks of Unvalidated AI Models
When an AI-claimed measurement model isn’t properly validated, the consequences extend beyond academic concerns, they affect business performance and profitability. For example, a national airline using an unvalidated MMM tool might misallocate millions of pounds of media budget because the model’s outputs lack a strong causal foundation.
This risk is compounded when internal teams assume the model has been rigorously tested simply because it’s marketed as “AI-powered.” Without rigorous checks, models can hide flawed assumptions and statistical errors “in plain sight”.
4. How to Evaluate Marketing Measurement Tools
So, given the hype, how should marketers judge a measurement solution’s true value? The article proposes an internal validation framework that emphasises evidence over claims:
a) Dedicated Experimentation
Set aside a budget for controlled experiments designed to test model predictions. True validation needs empirical evidence from real marketing interventions, not just model outputs.
b) Forecast Reconciliation
Compare model forecasts with actual business outcomes on a consistent basis. If reality diverges significantly from predictions, the model may lack a causal signal.
c) Quality Metrics from Vendors
Ask vendors for out-of-sample forecast accuracy checks, parameter recovery tests and stability measures. These help gauge how reliable and robust a model’s outputs are.
Ultimately, every model — AI-enhanced or not — must prove its ability to distinguish investments that deliver incremental revenue from those that simply correlate with demand that would have occurred anyway.
5. A Strategic Perspective on AI in Measurement
The most effective marketing teams view AI as a complementary tool, one that supports deeper analysis and clearer interpretation but doesn’t replace sound experimental design and statistical reasoning. When AI supports work that demonstrably improves ROI, it delivers real value. When it masks opaque methodologies, it becomes expensive “theatre” rather than insight.
Frequently Asked Questions
1. What is the main limitation of AI in marketing measurement?
AI tools like LLMs are not inherently designed to perform causal inference, meaning they can struggle to link marketing actions directly to business outcomes.
2. How does “AI washing” affect marketers?
AI washing involves overstating the role of AI in a product, potentially obscuring methodology and reducing transparency, which can mislead buyers about the model’s effectiveness.
3. Can machine learning still benefit media mix models?
Yes, advanced algorithms can improve estimation speed, detect data anomalies and summarise results more clearly, but they don’t replace rigorous causal modelling.
4. What is forecast reconciliation in measurement?
It’s the process of comparing a model’s predictions with actual results consistently to ensure the model is capturing causal relationships rather than noise.
5. Why is experimentation still essential?
Experiments provide empirical evidence of cause and effect, validating whether a model’s recommendations actually improve business outcomes.