measurement · 9 min read
How to measure marketing ROI when every system reports a different number
Marketing ROI arguments are almost never about the return. They are about what counted as revenue, what counted as cost, which month the money belonged to, and whether anything would have happened anyway. Settle those four and the number becomes useful; leave them open and no dashboard will help.
Written by Zubair Afzal, FounderUpdated
The actual disagreement
Nobody is arguing about the division
Return minus cost, divided by cost. That is the whole formula, and in twenty years of these arguments nobody has ever disputed it. What people dispute, without realising they are doing it, is four things underneath: what counted as return, what counted as cost, which period the money belonged to, and whether the outcome would have happened without the spending.
This is why better dashboards do not settle it. A dashboard makes a set of assumptions and then displays their consequences very quickly. If marketing is dividing revenue by media spend and finance is dividing gross profit by media plus fees plus production plus a share of salaries, the two will produce different numbers forever, and both will be correct.
So the useful work is not analytical, it is definitional, and it happens in a meeting rather than in a tool. Below is the arithmetic that makes the definitions visible, then a sequence for agreeing them, then an honest account of the point at which the calculation stops being able to tell you anything.
What return is made of
Return is a product of four numbers, and only one is marketing
- Recorded enquiries
- The only term most marketing reporting actually measures directly.
- Close rate
- Owned by sales. A change here moves reported marketing ROI on its own.
- Average order value
- Owned by pricing and product mix, and often seasonal.
- Gross margin
- Owned by finance. Using revenue instead makes every channel look profitable.
The method
Six decisions, taken in this order
The first two are conversations, not analysis, and skipping them is why the other four never settle anything.
Agree what counts as return
Revenue, gross profit or contribution after variable costs. Write down which, and whether it includes repeat purchases within a stated window. Gross profit is the honest default, because a channel that produces revenue at a margin below its own cost is destroying value while reporting a positive return on ad spend.
You get: One written definition of return, signed off by finance
Agree what counts as cost
Media, agency fees or salaried time, production, tooling and any platform minimums. The most common omission is internal hours, which makes in-house activity appear free and systematically biases every comparison against outsourced work. Decide once, apply everywhere, and note what was deliberately excluded.
You get: A cost basis, with exclusions stated
Fix the collection layer before analysing anything
Establish what is actually being counted as a conversion, whether it fires once or every time, what happens when somebody declines consent, and whether a checkout on another domain breaks the session. Analysis on top of a broken collection layer produces confident, precise, wrong answers, and the failure never announces itself.
You get: A written definition of every counted event and where it fires
Cohort the return to the spend period
Attach revenue to the month the spending happened, not the month the money arrived. Report recent cohorts as incomplete, with the expected maturation period stated. On any considered purchase this single change reverses conclusions, because a lagging channel measured on a calendar month always looks worse than a harvesting channel measured the same way.
You get: A cohort report with maturity flags on recent periods
Reconcile to the accounting system
Take the number of sales the business actually recorded and compare it with what the marketing systems claim. They will not match. The point is not to make them match but to know the size and direction of the gap, and to state it in the report. A marketing number that has never been reconciled is not evidence in a budget conversation.
You get: A stated variance between marketing and finance totals
Run one withholding test a year
Choose one meaningful line of spend. Withhold it in a defined region or segment for a defined period, keep everything else constant, and compare against a matched group. This is the only step that produces evidence about causation rather than association, and it will occasionally tell you something expensive and useful about a line nobody was willing to question.
You get: One documented test with a matched control and a written conclusion
Where the number goes wrong
Four ways a confident ROI figure misleads
All four produce a number that is internally consistent, reproducible and wrong, which is why they survive so long.
- Platform totals were added together.
- Each advertising platform reports the conversions it can associate with itself, inside its own window and under its own counting rule, with no knowledge of what the others recorded. A single purchase touched by two platforms is counted twice by design. Adding the columns produces a total that exceeds the number of sales the business made, and the resulting return is inflated by exactly that overlap.
- Revenue was used where margin belonged.
- Dividing revenue by spend makes almost every channel look successful, because it ignores the cost of delivering what was sold. On a twenty per cent margin, a campaign returning four times its ad spend is not profitable once fees and production are included. Nobody in these conversations is being dishonest; they are quoting the number their tool reports by default.
- A lagging channel was judged on a calendar month.
- Search and brand activity frequently produce revenue months after the spend, while harvesting channels produce it within days. Measured on the same calendar window, the lagging channel loses every time and gets cut, after which the harvesting channel quietly declines because the demand it was harvesting is no longer being created. Cohorting to the spend period prevents this, and very little else does.
- Nobody asked what would have happened anyway.
- Branded search is the clearest case. It converts at a high rate and costs little, so it reports beautifully, and a share of those people were going to reach you regardless. The reported figure is not wrong about what occurred, it is silent about what caused it, and only a withholding test can distinguish the two.
Methods
Four ways to answer whether it worked
These are not competing tools to choose between. They answer different questions, cost different amounts and fail in different directions, and a serious measurement setup uses several.
| Dimension | The question it can answer | What it costs to run | How it fails |
|---|---|---|---|
| Platform reporting | Did activity inside this platform coincide with recorded outcomes it could see? | Nothing extra. It is already there and already in the meeting. | Overlaps with every other platform, and cannot see anything outside itself. |
| Analytics attribution | Among the touchpoints we recorded, how should credit be divided? | Setup and maintenance, plus the discipline to tag campaigns consistently. | Silent about touchpoints it never saw, and sensitive to consent and thresholds. |
| Asking the customer | What does the buyer believe brought them here? | A form field and the patience to read free-text answers. | Unreliable individually, and skewed by recency and by what is easy to name. |
| Withholding tests | What changes when this activity stops, compared with a matched group? | Real forgone revenue for a defined period, plus the discipline to leave it alone. | Expensive, slow, and only answers the one question it was designed around. |
What to stop doing
Measurement effort that does not repay itself
Every one of these consumes real time in real businesses and none of them changes a decision.
- Trying to make platform totals reconcile to the penny. They were never designed to agree. Know the size of the gap, state it, and move on.
- Rebuilding the attribution model because the current one is unflattering. Changing the rule changes the history, and unless the targets are re-baselined at the same moment the comparison becomes meaningless.
- Reporting a return on a period that has not matured. On a considered purchase, last month is not a result yet, and publishing it as one guarantees an argument two quarters later.
- Quoting an industry return benchmark. The population, the definitions and the period behind those numbers are almost never stated, which makes them unusable even when they are honest.
- Building a bespoke attribution system before the collection layer has been checked. Sophisticated analysis of unreliable events produces confident nonsense faster than the simple version did.
- Counting a lead as a return. A lead is an input to a sales process with its own conversion rate, and treating it as revenue moves the argument to a place finance cannot follow.
Questions
What marketing and finance each ask about this
What is the formula for marketing ROI?
Return minus cost, divided by cost. The arithmetic is trivial and it is never the problem. The problem is that "return" and "cost" both have several defensible definitions and the people arguing are usually using different ones.
Write both down before calculating anything. Return is normally gross profit from marketing-attributable revenue, not revenue itself. Cost is normally media plus agency or salary plus production plus tooling, not media alone. Any pair of definitions can be defended; only an agreed pair ends the argument.
What is the difference between ROAS and ROI?
ROAS is revenue divided by advertising spend. ROI is profit minus total cost, divided by total cost. They answer different questions and they can point in opposite directions on the same campaign.
A campaign at four times return on ad spend is losing money if gross margin is twenty per cent and the fees and production are counted. Neither number is dishonest. Quoting one when the audience is asking about the other is where the trouble starts.
Why do our platform conversion totals add up to more sales than we made?
Because each platform reports conversions it can associate with itself, within its own attribution window and under its own counting rules, and it does not know what the others recorded. One purchase touched by two platforms is counted by both.
Some platforms also report modelled conversions, where an observed outcome could not be directly linked to a click and is estimated instead. That is documented behaviour rather than a fault, but it means platform totals were never designed to be added together, and the accounting system remains the only arbiter of how many sales occurred.
Our sales cycle is nine months. How do we measure anything?
By cohorting rather than by calendar. Attach revenue to the period the spend happened in rather than the period the invoice arrived, and accept that the current quarter is always incomplete and should be labelled as such.
In the meantime, watch leading indicators that move earlier and correlate with the eventual outcome: qualified enquiries, opportunities created, and stage progression. Report them as leading indicators, not as results, so nobody mistakes activity for revenue.
How do we account for word of mouth and offline influence?
You do not, in the analytics. Nothing records the conversation in a pub or the recommendation in a group chat, and any model that appears to account for them is redistributing recorded touchpoints rather than discovering unrecorded ones.
Two partial remedies are worth the effort. Ask on the form how people heard about you, and treat the answers as a directional supplement rather than data. And run occasional holdouts, because a genuine holdout captures the unrecorded influence automatically — it is included in the difference between the two groups whether or not anybody logged it.
Is turning a channel off to test it not just losing revenue?
It costs something, and that cost should be stated honestly rather than hidden. What it buys is the only evidence that separates what the spending caused from what would have happened anyway.
It is also cheaper than it sounds when scoped properly: one region, or one segment, for a defined period, rather than the whole account. Set against a budget line that runs every month indefinitely, a few weeks of partial withholding is a small price for finding out whether the line is doing anything.
Settle the definitions before the next review meeting
Most of this is resolved in one session with both sides in the room: what counts as return, what counts as cost, and how big the gap to the accounts actually is. It is not glamorous work and it ends a recurring argument.
Related
Where to go next
- sizing the budget in the first placeThat page decides what to spend. This one judges what the spending did.
- the lifecycle and measurement practiceWhere the collection layer and the CRM side of this gets fixed.
- attribution and incrementality work
- fixing what the tracking counts
- how credit gets divided between channels
- why benchmark rates are unusable
- tag campaign links so they land in the right report
Last updated