Delivery rate is not a result

Messaging dashboards lead with delivery rate, and it is almost always above 95%. That is not because the programmes are working. It is because delivery rate measures the network, and networks are good.

"Delivered" usually means a delivery receipt came back: an operator or messaging app confirmed the message reached the device. It does not mean the person saw it, read it, or did anything.

Between delivery and outcome sit at least three drops: the message is not seen; it is seen and ignored; it is read but not acted on. Each stage loses most of what the previous one had.

That is arithmetic, not pessimism — and it is why a 97% delivery rate and a 2% outcome rate routinely describe the same campaign.

Use sent, not delivered, as the denominator. Dividing by delivered quietly hides your worst failures: the messages that never arrived are exactly the ones that could not succeed.

Delivery rate is an average over statuses, and the average hides the diagnosis.

• Rejected at submission — your problem: a malformed number, a missing registration, an unapproved template. Easiest to fix, because it fails immediately and loudly.
• Accepted, then failed downstream — an unreachable or barred number, or an operator that accepted and dropped it. The cause code is the whole value here; a provider that returns "failed" without one is hard to debug against.
• Accepted, never resolved — the message left and nothing came back. Every large programme has some. Track them explicitly rather than letting them round into "delivered".

Detecting a break. A delivery rate that drops from 97% to 71% overnight is a real signal: a registration lapsed, a template was rejected, a route changed. Alert on sudden changes, not on the level.

SMS is billed per segment. A single segment holds 160 characters in the basic GSM alphabet, but only 70 when the message contains anything outside it — Arabic, Hindi, Chinese, Cyrillic, Greek, or a single emoji. Longer messages are split into segments of 153 or 67 characters.

So the same notice can cost two or three times as much in one language as in another, and one stray character (a curly quote pasted from a word processor, say) can double a message's cost. A variable in a template — a name, an address — means you often do not know the length until sending. Measure cost per outcome, not per message, and the problem shrinks to its proper size.

The hardest part is not measurement but attribution. A customer who received a payment reminder and then paid may have paid anyway.

The honest answer is a holdout group: a randomly chosen set of people who receive nothing, measured on the same outcome. If 61% of reminded customers pay on time and 55% of the holdout do, the reminder is worth about six points — not 61.

To build one that survives scrutiny:

• Randomise per person, and keep them there. Re-drawing the group for every campaign means everyone eventually receives something, and the control stops being a control.
• Exclude nobody for convenience. Pulling high-value customers out of the holdout removes exactly the people you most need to measure.
• Keep it small and permanent. A few per cent is often enough at volume. A holdout that runs only during a test tells you about the test.
• Report the uncertainty. A 3% lift measured on two hundred people is noise. Saying so is what makes the next report believable.

One page: outcome per thousand sent, with the holdout beside it; the funnel underneath; opt-out and complaint rates as a trend; one sentence on what the numbers cannot settle. If it takes more than a page, it is padded with things that move without meaning.

• Do you return a specific cause code for every failure, and can I get statuses by webhook?
• How do you report messages that were accepted but never resolved?
• Can I see segment counts and cost per message before and after sending?

• Measuring a conversational programme — the deeper level.
• Channels are not interchangeable — what silence means on each channel.
• What CPaaS actually is — accepted, delivered and acted on.