Chapter 8. Practice: A Four-Week Attention Ledger and a Pre-Registered Trial

Part 1. The attention ledger

The models of this book become useful only when they meet numbers. This part describes a four-week ledger that a creator, or a brand working with creators, can keep with nothing more than a spreadsheet and the analytics every platform already provides.

What to record

One row per post or live session:

Column	Meaning
date, format	Day, and short video, long video or live
make_min	Minutes spent making it, including planning and editing
impressions	Times it was shown
views_3s	Views that lasted past the first seconds (the platform’s own threshold)
watch_s	Average watch time, in seconds
clicks	Clicks to the shop, product or link
orders	Orders attributed to it
repeat	Orders from customers who had bought before (if the shop can tell)

The four weeks

Week	Task	Question it answers
1	Record only; change nothing	What does the funnel look like now?
2	Compute each link of Chapter 4’s chain	Which link is weakest?
3	Change one thing aimed at the weakest link	Did that link move, and did orders follow?
4	Compare formats per hour of making, not only per post	Where does an hour of work produce most?

Reading a week

Calculation

One invented week of a creator’s ledger, with two short videos a day and a long video every third day, summarised by format.

import random

# One invented week of a creator's attention ledger: one row per post. Real ledgers are exported
# or typed in.
rng = random.Random(8)
rows = []
for day in range(7):
    for fmt, length_s, make_min in (("short", 30, 45), ("short", 25, 40), ("long", 480, 240)):
        if fmt == "long" and day % 3:                      # a long video every third day
            continue
        imp = rng.randint(4000, 20000) if fmt == "short" else rng.randint(2000, 8000)
        v3 = int(imp * rng.uniform(0.25, 0.45))
        watch = length_s * (rng.uniform(0.35, 0.6) if fmt == "short" else rng.uniform(0.25, 0.45))
        clicks = int(v3 * (rng.uniform(0.004, 0.01) if fmt == "short" else rng.uniform(0.015, 0.03)))
        orders = sum(rng.random() < (0.08 if fmt == "short" else 0.12) for _ in range(clicks))
        rows.append(dict(fmt=fmt, make_min=make_min, imp=imp, v3=v3, watch=watch, clicks=clicks, orders=orders))

print("format  posts  hook rate  watch time  clicks/1k views  orders/1k impressions  orders per hour made")
for fmt in ("short", "long"):
    r = [x for x in rows if x["fmt"] == fmt]
    s = lambda k: sum(x[k] for x in r)
    print(f"{fmt:6s} {len(r):6d} {s('v3') / s('imp'):10.1%} {s('watch') / len(r):9.0f} s "
          f"{1000 * s('clicks') / s('v3'):16.1f} {1000 * s('orders') / s('imp'):22.2f} "
          f"{s('orders') / (s('make_min') / 60):21.2f}")
# format  posts  hook rate  watch time  clicks/1k views  orders/1k impressions  orders per hour made
# short      14      34.9%        13 s              6.8                   0.19                  3.03
# long        3      31.0%       174 s             18.2                   0.77                  0.75

In this invented week, long videos turn far more viewers into buyers: about four times as many orders per thousand impressions, and nearly three times as many clicks per thousand views. Short videos still produce more orders per hour of making, because they take so much less time. Neither number alone decides the strategy; a creator who measures only one of them will optimise for the wrong thing. And neither number says anything about what happens after the first order, which is the question the trial below is designed to answer.

Note

Attribution, deciding which post caused which order, is imperfect on every platform, and the same customer may see several posts before buying. The ledger is for comparing a creator’s own formats and weeks, not for proving cause and effect. That needs a randomised trial.

Part 2. A pre-registered trial

The question

Does a stream of long-form content that builds understanding and trust (“depth”) produce more repeat purchases over 90 days than a stream of frequent short content optimised for immediate response (“bursts”)? Chapter 4’s hypothesis predicts that depth helps the later links of the funnel, and especially repeat purchase. The common practice of many sellers assumes the opposite, or that frequency is what matters.

Design

Element	Specification
Population	First-time buyers at participating online shops that sell repeat-purchase products, who have agreed to receive follow-up content
Allocation	Individual randomisation 1:1 at the first purchase, stratified by shop, computer-generated and concealed
Arm A: depth	One long-form piece a week for 12 weeks (explainer video or article about the product category, use and care)
Arm B: bursts	Four short pieces a week for 12 weeks (short videos, offers and reminders)
Equalised	Total offers and discounts identical in both arms; same channels; same opt-out
Primary outcome	Repeat purchase at the same shop within 90 days of the first purchase, from shop records
Secondary	Number of repeat orders; revenue per customer; opt-out rate; complaints
Blinding	Customers are not told which arm they are in; analysts blinded to arm labels until the code is frozen

Sample size

The smallest difference worth acting on is 3 percentage points in the 90-day repeat-purchase rate, from an assumed 20 per cent in one arm. With two-sided α = 0.05 and 90 per cent power:

Calculation

from math import sqrt, ceil
from statistics import NormalDist

z = NormalDist().inv_cdf
# 90-day repeat purchase: assumed rate in one arm, smallest difference worth having
p1, p2 = 0.20, 0.23
alpha, power = 0.05, 0.90
pbar = (p1 + p2) / 2
n = ((z(1 - alpha / 2) * sqrt(2 * pbar * (1 - pbar)) + z(power) * sqrt(p1 * (1 - p1) + p2 * (1 - p2))) / (p2 - p1)) ** 2
print(f"first-time buyers needed per arm: {ceil(n):,}")
print(f"total, allowing 10% lost records:  {ceil(2 * ceil(n) / 0.9):,}")
# first-time buyers needed per arm: 3,939
# total, allowing 10% lost records:  8,754

Analysis and decision rule

The primary analysis compares the repeat-purchase rates of the two arms in all randomised customers (intention to treat), with a 95 per cent confidence interval for the difference, adjusted for shop as a stratification factor. The result is read by the rule below, written before any data exist, in the style of an equivalence test (Lakens 2017).

Calculation

# smallest difference in 90-day repeat-purchase rate worth acting on (3 percentage points)
SESOI = 0.03

def verdict(lo, hi):
    """lo, hi: 95% confidence interval of (repeat rate, long-form arm) - (repeat rate, short-burst arm)."""
    if lo > 0 and hi >= SESOI:
        return "long-form depth wins"
    if hi < 0 and lo <= -SESOI:
        return "short bursts win"
    if -SESOI < lo and hi < SESOI:
        return "equivalent: no difference worth acting on"
    return "inconclusive"

for name, ci in {"A": (0.012, 0.051), "B": (-0.047, -0.008), "C": (-0.018, 0.021), "D": (-0.011, 0.042)}.items():
    print(f"result {name}: CI [{ci[0]:+.3f}, {ci[1]:+.3f}] -> {verdict(*ci)}")
# result A: CI [+0.012, +0.051] -> long-form depth wins
# result B: CI [-0.047, -0.008] -> short bursts win
# result C: CI [-0.018, +0.021] -> equivalent: no difference worth acting on
# result D: CI [-0.011, +0.042] -> inconclusive

The examples are invented to show how each kind of result will be read. An interval inside ±3 percentage points will be reported as “no difference worth acting on”, not as “no effect”; an interval that crosses a boundary will be reported as inconclusive, not as a trend.

Ethics, data protection and conflicts of interest

Both arms are ordinary, honest marketing communication; neither contains false claims or manufactured social proof. Customers can opt out of all follow-up content at any time, and opting out is itself a recorded outcome. Only data that shops already hold for their own records are used, under the shops’ privacy notices and applicable data-protection law, such as Malaysia’s Personal Data Protection Act 2010 (Parliament of Malaysia 2010). The protocol will be reviewed by an independent research ethics committee and registered publicly before any customer is enrolled; it follows the logic of the SPIRIT and CONSORT guidance for trial protocols and reports (Chan et al. 2013; Schulz et al. 2010), and the practice of online controlled experiments (Kohavi et al. 2020).

Disclosure

The author is the founder and chief strategist of Yafu Global and of Asia Influencer X, an agency for creators and live commerce in Southeast Asia, and of marketingconsultantasia.com, and so has a direct commercial interest in the result. To limit the effect of that interest, the protocol, analysis code and decision rule are published before enrolment, the analysis will be carried out by analysts independent of the author’s companies, and the results, anonymised data and code will be published whatever the outcome.

What the trial cannot show

It tests two content strategies in shops that agree to take part and products that people buy more than once. It says nothing about one-off purchases, brand awareness or long-term brand value. A null result would not show that content does not matter; it would show that, between these two strategies, the difference in 90-day repeat purchase is smaller than three percentage points.

References

Chan, An-Wen, Jennifer M. Tetzlaff, Douglas G. Altman, et al. 2013. “SPIRIT 2013 Statement: Defining Standard Protocol Items for Clinical Trials.” Annals of Internal Medicine 158 (3): 200–207.

Kohavi, Ron, Diane Tang, and Ya Xu. 2020. Trustworthy Online Controlled Experiments: A Practical Guide to a/b Testing. Cambridge University Press.

Lakens, Daniël. 2017. “Equivalence Tests: A Practical Primer for t Tests, Correlations, and Meta-Analyses.” Social Psychological and Personality Science 8 (4): 355–62.

Parliament of Malaysia. 2010. Personal Data Protection Act 2010 (Act 709).

Schulz, Kenneth F., Douglas G. Altman, and David Moher. 2010. “CONSORT 2010 Statement: Updated Guidelines for Reporting Parallel Group Randomised Trials.” BMJ 340: c332.

Fan, Y. (2026). The Economics of Attention: Simon's Scarcity Law, Power-Law Networks and Conversion Funnels in Creator Commerce [注意力的经济学] (Version 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.23689302 · Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0).