Why write the study before running it

Every hypothesis box in this book says what result would show it wrong. That is only worth something if the test is fixed before the data exist. A study whose hypotheses, measures and analyses are written down and time-stamped in advance cannot quietly change its question after seeing the answer.

This chapter is that plan for the book’s central practical claim. It is version 1.0 of the protocol. It will be registered on the Open Science Framework, with any changes made before recruitment listed openly, and the registration number will be added to the next version of this book.

The question

Chapter 3 proposed, and Chapter 5 sharpened, a comparison between two ways of raising a relationship’s drive $\mu = J – L – c$:

• raising the inflow $\Delta J$: adding a novel, shared activity each weekend, the kind of intervention studied by Aron and colleagues (Aron et al. 2000);
• closing the leak $-\Delta L$: protecting one hour each evening from screens.

The book predicts that the second is at least as effective as the first for desire and closeness, and more effective for the scattering of attention. This protocol tests that prediction.

Design

A three-arm randomised controlled trial with the couple as the unit of randomisation.

Couples are allocated 1 : 1 : 1 by computer-generated randomisation, stratified by whether they have children living at home. Participants cannot be blind to their arm; the analyst will work with arm labels hidden until the analysis code has been run on the real data and published.

Timeline. Baseline week (W0), four intervention weeks (W1–W4), follow-up at week 12 (W12).

Participants

Inclusion: couples aged 21 or over, living together for at least one year, both partners consenting separately, both with a smartphone that records screen time.

Exclusion: current couples therapy; either partner reporting fear of the other, or any threat or violence in the past year, on a short safety screen; either partner reporting a current severe mental-health crisis. Screened-out participants receive information about local support services. Couples may leave at any time without giving a reason.

Hypotheses

All hypotheses are stated so that the predicted direction is positive.

H1 (primary). At W4, arm B shows a larger increase in wanting than arm C.

H2 (primary). At W4, arm B shows a larger increase in wanting than arm A.

H3 (secondary). Arm B shows a larger decrease in evening attention entropy, and a larger increase in shared minutes, than arm C.

H4 (secondary). Arms A and B each show a larger decrease in recovery time after conflict than arm C.

H5 (secondary). Within arms A and B, the standardised increase in wanting is larger than the increase in liking (Chapter 2).

Exploratory. Whether couples with long recovery times at baseline benefit less from A and B (Chapter 6); whether the change in attention entropy statistically accounts for part of the effect in H1; whether effects persist to W12.

Measures

The wanting and liking items are new. Their internal consistency and their separation from each other will be checked in a pilot of about 40 couples, which is not part of the main analysis, and any item changes will be fixed before registration.

Sample size

The planned primary analysis compares arms at W4, adjusting for W0 (analysis of covariance), with two primary comparisons, each tested at $\alpha = 0.025$.

from statistics import NormalDist
from math import ceil

z = NormalDist().inv_cdf
alpha = 0.05 / 2               # two primary comparisons, Bonferroni
rho = 0.5                      # expected correlation of baseline and follow-up scores (ANCOVA gain)
print("effect size d   power   couples per arm   with 15% dropout")
for d in (0.40, 0.30, 0.25):
    for power in (0.80, 0.90):
        n = 2 * ((z(1 - alpha / 2) + z(power)) / d) ** 2 * (1 - rho ** 2)
        print(f"{d:13.2f}   {power:5.2f}   {ceil(n):15d}   {ceil(ceil(n) / 0.85):16d}")
# effect size d   power   couples per arm   with 15% dropout
#          0.40    0.80                90                106
#          0.40    0.90               117                138
#          0.30    0.80               159                188
#          0.30    0.90               207                244
#          0.25    0.80               229                270
#          0.25    0.90               298                351

The study is planned to detect a standardised difference of $d = 0.30$ for H1 with 90 % power, which requires 207 couples per arm, or 244 per arm (732 couples) allowing for 15 % dropout. With 207 couples per arm completing the study, H2 has about 76 % power for $d = 0.25$. The values of $d$ are assumptions, chosen to be modest because effects reported in small studies tend to be overestimated.

Analysis

Primary. For each primary hypothesis, a linear model with the couple’s W4 wanting score as outcome, arm as predictor and W0 wanting and the stratification variable as covariates, analysed by intention to treat. Missing data will be handled by multiple imputation under the assumption that they are missing at random, with a sensitivity analysis using complete cases.

Secondary and exploratory. The same model for secondary outcomes; multilevel models with partners nested in couples where individual scores are used; recovery time analysed per conflict with couple as a random effect. Secondary results will be reported with 95 % intervals and labelled as secondary; exploratory analyses will be labelled as such and not used to support or refute any hypothesis.

How the results will be read

Each hypothesis is judged from the confidence interval of its standardised difference against a smallest effect size of interest of $d = 0.20$: an effect smaller than that is treated as too small to matter to a couple. The rule is fixed in advance and written as code.

def verdict(lo, hi, sesoi=0.20):
    """Pre-registered reading of a standardised difference from its 97.5% confidence interval.
    Every outcome is coded so that the predicted direction is positive. sesoi = smallest effect size of interest."""
    if lo > 0 and hi > sesoi:
        return "supported"
    if hi < 0:
        return "refuted: effect in the opposite direction"
    if hi < sesoi:
        return "refuted: an effect worth having is ruled out"
    return "inconclusive: the study could not tell"

examples = {"B vs C, wanting": (0.12, 0.61), "B vs A, wanting": (-0.05, 0.38),
            "A vs C, wanting": (-0.08, 0.17), "B vs C, recovery": (-0.35, -0.04)}
for name, (lo, hi) in examples.items():
    print(f"{name:17s} CI [{lo:+.2f}, {hi:+.2f}] -> {verdict(lo, hi)}")
# B vs C, wanting   CI [+0.12, +0.61] -> supported
# B vs A, wanting   CI [-0.05, +0.38] -> inconclusive: the study could not tell
# A vs C, wanting   CI [-0.08, +0.17] -> refuted: an effect worth having is ruled out
# B vs C, recovery  CI [-0.35, -0.04] -> refuted: effect in the opposite direction

The example intervals are invented, to show how each kind of result will be read. In particular:

• The book’s central claim is refuted if H2’s interval lies wholly below $d = 0.20$, that is, if closing the evening leak is shown to be no better, by any margin that matters, than adding weekend novelty.
• The leak model is refuted if H1 is refuted: if a protected evening hour does not raise wanting more than keeping a diary.
• The mechanism is refuted if H3 is refuted: if the protected hour changes desire without changing the scattering of attention, the explanation in Chapters 1 and 5 is wrong even if the practice works.
• Inconclusive results will be reported as inconclusive, not as support.

Ethics, conflicts of interest and openness

The study will not recruit until an independent research ethics committee has approved it; the approval number will be given in the registration and in the next version of this book. Each partner consents separately and can withdraw without the other being told why. Safety screening and information about support services are part of enrolment.

What the study cannot show

It cannot show that the model of this book is true in general. It tests two practices, over four weeks, in couples who feel safe with each other and who volunteered for a study, most likely people who are already interested in their relationship. A positive result would support the leak model for such couples; a negative one would count against it. Either way, the result will be published, and the next version of this book will be changed to match it.