Free lift check
Is the lift your platform's AI claims real?
Platforms grade their own AI. If yours kept a holdout chosen at random before the AI acted, its counts can say whether the lift is confirmed, smaller than claimed, or, more often than vendors admit, too small to tell. Below is a worked example; use your own export or the dashboard's numbers.
Check your own holdout
A CSV with one row per group per day (date, group, customers assigned, converters), or one row per customer (user_id, group, converted). The group column names the holdout or control; everything else is the AI group.
Runs in your browser. Your file is never uploaded.
Who does the campaign help?
From a random holdout with one row per customer: each attribute (city, plan, platform, past activity) is first tested for any difference between its groups, with a correction for checking several; each group's effect is shrunk toward its attribute's average so a lucky small group is not crowned; and the rule is checked on customers it was not fitted on. "No attribute separates who responds" is a normal, useful answer.
No holdout yet? Size one
Choose who is held out at random before the campaign starts (a salted hash of the customer id: app.py holdout draws one), run it for at least two weekly cycles, start outside festival weeks, and agree the metric, window and comparison in writing first.