The Complexity Of Marketing Assessment How Attribution Mmm And Experiments Unveil Their Strengths
Management Summary
Authors: Alexandra Lanz & Buster Grunau
The challenge of marketing assessment
In today’s data-driven world, measuring and evaluating marketing success is crucial. But many marketers face the challenge of identifying what really works. Offline channels without direct measurement options, different attribution models and a variety of analysis tools can lead to confusion. There is no “one truth” when evaluating marketing success. Different methods have different strengths, weaknesses and areas of application. The choice of the appropriate evaluation system depends on the specific question and goals.
Three dimensions of marketing assessment
| methodology | Database | Application | |
|---|---|---|---|
Attribution |
Rule based, machine learning | User level (cookies, logins, IDs) |
Very granular Real-time optimization |
MMM |
Static modeling | Aggregated historical data (sales, spend, etc.) |
Cross-channel budget allocation Long-term, strategic decisions |
Experiments |
Causal inference | Test & Control groups |
Evidence of direct input Methodologically the most robust |
Differences in methodology, database and application
- Attribution: This system uses rule-based models and/or machine learning to analyze user-level touchpoints. It provides granular insights and real-time optimization, but relies heavily on tracking data and does not provide direct proof of causality.
- Marketing Mix Models (MMM): MMM uses statistical modeling based on aggregated historical data. They are good for cross-channel budget allocation and long-term strategic decisions, but are less granular and deliver slower results.
- Experiments: This relies on causal inference through test and control groups. Experiments provide the most robust evidence of causality, but can be laborious and are not always practical.
Strengths and weaknesses
Each of the three approaches – attribution, marketing mix models (MMM) and experimentation – has specific strengths and weaknesses. Strategic questions, e.g. about budget planning or channel evaluation as a whole, can be answered well with an MMM. Detailed analyzes of creatives, for example, can be answered with the attribution of any advertising tool. What must not happen, however, is that attributed numbers from different advertising tools are compared without reflection. Fundamental things are often ignored: How is attribution done, with which lookback window? What is the basis for measurement and how dependent are we on 3rd-party cookies for measurement? Are conversions deduplicated? etc.
The ideal strategy is therefore to combine cross-channel analytics such as MMM, experiments and attributions to get a comprehensive picture of marketing performance and make the right decisions.
Strengths and weaknesses of the models, source: e-dialog
The Measurement Triangle
None of the rating systems is better on its own – it’s the combination that makes it. This enables cross-channel, validated evaluation of marketing and the identification of potential experiments.
The Measurement Triangle – No “one truth”
The Measurement Triangle, source: e-dialog based onGoogle
Use experiments for calibration
In order to optimize attribution or marketing mix modeling (MMM), it is worth calibrating them through experiments such as lift studies. Such tests create real comparison groups – with and without advertising exposure – and thereby enable a causal assessment of the advertising effect. In contrast to purely observational models, experiments provide reliable information about what advertising actually does. These real measured effects can be used to adjust attribution or MMM. The result: valid, realistic models that make marketing decisions more informed and budget management more efficient.
Conclusion
The complexity of marketing assessment requires a holistic approach. By combining attribution, MMM and experimentation, marketers can get a more complete picture of their marketing performance and make informed decisions. It is important to understand the strengths and weaknesses of assessment systems and use them strategically to achieve maximum benefit.
