Many companies are considering econometric studies using marketing mix modelling (MMM) to evaluate their marketing effectiveness. I’ve discussed why this technique is becoming more popular in this related article titled: The Renaissance of Marketing Mix Modelling, but for firms exploring this option as a way of measuring marketing effectiveness, what do they need to consider?
Marketers will always want a view of the effectiveness of their activities. The modern MMM landscape is filled with choices, each with benefits and trade-offs. Despite this approach becoming more accessible, businesses must exercise caution and strategic deliberation when choosing the right approach to build these models, as the complexity of the modern marketing landscape demands a well-informed and tailored methodology.
As the ability to source data, find appropriate talent, and have requisite computational power has become easier, many companies will be able to build MMMs within their own analytics or data science functions. There are several advantages to this: external solutions are likely to be more expensive, there’s often deep domain knowledge specific to the firm held by modellers and analysts within, and it’s a way of ensuring that key, required questions can be answered by the model. Most companies however cannot afford to keep an econometrician on the books solely for updating a MMM four times a year, which could lead to well-meaning amateurs building models in an area where experience is important. If MMMs are to be used by a company in the way that maximises their utility (i.e., high-level budget setting), this needs to be treated far more seriously than a temporary project.
A key benefit of working with an agency or consultancy is to benefit from their experiences of building similar models within your sector, as well as possibly borrowing learnings from outside of it.
Specialist econometric consultancies are likely to have multiple clients facing very similar business problems and will possibly have tools or solutions going beyond MMM or building on top of it. Still, care should be taken that they aren’t applying “cookie-cutter” techniques and take the time to fully understand the intricacies of your business. Meanwhile, many media agencies may have the same capability of an econometrics provider, but with the added advantage of the measurement and media teams being able to work closely and collaboratively. This can lead to multiplicative benefits, but steps should be taken to ensure the independence of those teams (such as what we implement at Jaywing), and that they aren’t “marking their own homework”.
Some advice on choosing which agency to work with can be found in this guide from WARC.
Along with deciding who will build the MMM, companies also must decide how. MMMs can be built from scratch in, for example, python or R. This has the greatest degree of flexibility in terms of the model structure and transformations but also presents the greatest barrier to entry for companies. This approach shouldn’t really be considered, except by companies either willing to hire an econometrician or where the right combination of modelling skills and market understanding is already present.
Another option is to take advantage of pre-existing, open-source tools such as Robyn from Meta or PyMC-Marketing from PyMC Labs. These tools can provide a structure for modelling and increase the speed at which a reasonable model can be built (at the cost of flexibility). There has been some criticism that Robyn might be biased towards online channels, but this likely arises out of a small bias towards short-term impacts rather than long-term, rather than outright favouritism or deception.
The final option would be for companies to license or adopt a proprietary tool for their modelling. Although these tools are effective at building many models quickly, and even helping the modeller structure these models in a specific way, they may be expensive for a company looking to step into MMM for the first time, or that faces little need for complexity (i.e., they operate across few regions or have few products). These tools are worthwhile when building MMMs at scale, say across significant numbers of brands and product lines, but could be a questionable overhead otherwise for many single-sector businesses.
The resurgence of MMM powered by shiny modelling techniques is a great option for many firms to use. Irrespective of whether firms go with internal or external builders, however, there are other techniques, such as A:B testing (either within a population or on a matched population) that are still very valuable.
Hypothesis testing is the best way to determine an answer to a specific question (i.e., incrementality of a single channel). The outputs of these tests can be used in conjunction with a MMM – either to ground estimates when building a new model (e.g., setting Bayesian priors) or as validation/monitoring for existing ones (see this WARC article on calibration for more details).
Multi-touch attribution (MTA) is unlikely to also disappear completely. Although the solution is faced with data challenges, many of these are surmountable and MTA can still provide useful, tactical insight in a granularity, both in terms of populations and campaign details, that are not possible otherwise.
The “gold standard”, therefore, is a suite of solutions – including MTA, MMM, and A:B Testing – being used and operated by a curious and creative team of marketers, willing to use both artistic flair and rigorous modelling and analysis to achieve business objectives.
At Jaywing, we feel that there’s still significant benefit coming from MTA, especially with respect to online advertising. This can work very well from a bottom-up approach, in conjunction with MMM working from top-down. A:B testing then serves to validate both models, and provide a source of truth when answering extremely key one-time questions e.g. whether to launch a new channel in a particular area or not.
The key things that you can be doing in preparation for a MMM is to begin to collect data and to run tests and experiments.
Having your marketing cost and impressions/reach data in one place will make the data collection stages of a model build much easier and potentially drive down costs. This might require leveraging an analyst within the organisation to properly store this data, but even in the absence of this, the raw data will likely be more useful than not.
Tests and experiments are useful on their own merit but are invaluable when used to help ground and calibrate an MMM. Without these tests, modellers are relying on their experience and measures of statistical fit when building a model, rather than a channel-level comparison with the ground truth.
It may also be necessary to build out a business plan and indicate how an econometrics project can show value. When evaluating the prospect of a MMM in the context of your marketing budget, it’s useful to remember that even a modest increase in efficiency can pay for a MMM project in of itself. A 20% efficiency gain on a £250k marketing budget, for example, will likely be enough to cover the costs of an entry-level MMM – estimated efficiency gains vary with agencies and providers quoting (for example), 10-38%, 20%, 20%, and 30%.
The world of marketing mix modelling (MMM) offers many options for companies seeking to measure their marketing effectiveness. As we've explored in this article, the choices you make can significantly impact the success of your MMM initiative. Whether you're debating between in-house modelling, partnering with a consultancy, or leveraging an agency, or if you're torn between building from scratch, using open-source tools, or adopting proprietary solutions – each path has its own merits and considerations. There’s no silver-bullet answer, but to have a unified view of all marketing, consistent and reliable data is a must, and a commitment to experimentation and testing can minimise downside risk for relatively little cost. The key is to remain adaptable and informed.
Ready to take your marketing effectiveness to the next level with econometric studies? Let's connect and discuss how we can tailor a solution to fit your unique needs.
Contact us today to schedule a consultation and unlock the insights that will drive your business forward.