5D Library › Two-Process Sleep Architecture, Slow-Wave Glymphatic Clearance and Thermoregulatory Phase Control · 中文
Chapter 6. What a Sleep Tracker Can Know, and the Score That Keeps You Awake
Measuring against an imperfect standard
Consumer rings and wristbands estimate sleep stages from movement, heart rate and sometimes skin temperature. They are validated against polysomnography scored by trained technicians. But scoring is itself imperfect: in the American Academy of Sleep Medicine’s inter-scorer reliability programme, thousands of scorers agreed with the reference score on about 83 % of epochs overall, with lower agreement for N1 and N3 (Rosenberg and Van Hout 2013). A device compared with one scorer is compared with a noisy ruler.
The eleventh program makes the consequence explicit. True stages come from the Chapter 3 chain; two human scorers and a device each read them with their own confusion probabilities.
Calculation
# Chapter 6. How well can a wrist or ring device stage sleep, and how would we
# know? True stages come from the p05 chain (N1 and N2 merged as "light"). Two
# human scorers and a device each read the true stage with their own confusion
# probabilities (illustrative values: the scorers agree with each other about
# 80 % of epochs, of the order reported for expert polysomnography scoring;
# the device is a plausible consumer tracker). We compute accuracy and Cohen's
# kappa against the truth and against one scorer, which is all a validation
# study can do.
import math
import random
W, L, D, R = range(4)
NAMES = ("wake", "light", "deep", "REM")
SCORER = [[0.90, 0.08, 0.00, 0.02],
[0.03, 0.89, 0.04, 0.04],
[0.00, 0.20, 0.80, 0.00],
[0.02, 0.07, 0.00, 0.91]]
DEVICE = [[0.55, 0.35, 0.02, 0.08],
[0.05, 0.78, 0.09, 0.08],
[0.01, 0.40, 0.57, 0.02],
[0.04, 0.30, 0.02, 0.64]]
def truth_night(rng):
st, since, seq = 0, 0, []
for e in range(960):
t = e / 120.0
s = math.exp(-t / 2.5)
u = rng.random()
if st == 0:
st = 1 if u < 0.045 else 0
elif st == 1:
st = 2 if u < 0.10 else (0 if u < 0.12 else 1)
elif st == 2:
p3 = 0.05 * s
pr = 0.0 if since < 100 else 0.03 * (0.5 + t / 8)
st = (
3
if u < p3
else (
4
if u < p3 + pr
else (
1
if u < p3 + pr + 0.006
else 0 if u < p3 + pr + 0.009 else 2
)
)
)
elif st == 3:
po = 0.012 + 0.05 * (1 - s)
st = 2 if u < po else 0 if u < po + 0.001 else 3
else:
pe = 1.0 / (20 + 50 * t / 8)
st = 2 if u < pe * 0.8 else 0 if u < pe else 4
since = 0 if st == 4 else since + 1
seq.append({0: W, 1: L, 2: L, 3: D, 4: R}[st])
return seq
def read(seq, conf, rng):
out = []
for s in seq:
u, acc = rng.random(), 0.0
for j, p in enumerate(conf[s]):
acc += p
if u < acc:
out.append(j)
break
else:
out.append(3)
return out
def kappa(a, b):
n = len(a)
po = sum(x == y for x, y in zip(a, b)) / n
pe = sum((a.count(k) / n) * (b.count(k) / n) for k in range(4))
return po, (po - pe) / (1 - pe)
if __name__ == "__main__":
rng = random.Random(57011)
T, A, B, Dv = [], [], [], []
deep_t, deep_d = [], []
for _ in range(200):
t = truth_night(rng)
dv = read(t, DEVICE, rng)
T += t
A += read(t, SCORER, rng)
B += read(t, SCORER, rng)
Dv += dv
deep_t.append(t.count(D) / 2)
deep_d.append(dv.count(D) / 2)
print(
" agreement Cohen's kappa"
)
for name, x, y in (
("scorer A vs scorer B", A, B),
("scorer A vs truth", A, T),
("device vs truth", Dv, T),
("device vs scorer A", Dv, A),
("perfect device (= truth) vs scorer A", T, A),
):
po, k = kappa(x, y)
print(" {:<38}{:6.3f} {:6.3f}".format(name, po, k))
print("\nMinutes per night by stage (mean over 200 nights)")
print(
" last column: share of epochs of this stage the device gets right"
)
print(" stage truth device device - truth right")
for k in range(4):
tt, dd = T.count(k) / 400, Dv.count(k) / 400
hit = sum(1 for x, y in zip(T, Dv) if x == k and y == k) / T.count(k)
print(
" {:<7} {:6.1f} {:6.1f} {:+6.1f} {:.3f}".format(
NAMES[k], tt, dd, dd - tt, hit
)
)
mt, md = sum(deep_t) / 200, sum(deep_d) / 200
cov = sum((a - mt) * (b - md) for a, b in zip(deep_t, deep_d))
r = cov / math.sqrt(
sum((a - mt) ** 2 for a in deep_t)
* sum((b - md) ** 2 for b in deep_d)
)
print(
"\nNight-to-night correlation of device deep minutes with true deep"
)
print("minutes: {:.3f}".format(r))
print("\nReading: a validation study compares the"
" device with a scorer, not with")
print("the truth, so even a perfect device cannot"
" score above the scorers' own")
print("agreement with the truth. The device's nightly"
" averages can be close while")
print("it misses almost half of the deep-sleep"
" epochs: errors in both directions")
print("cancel. Caution: here the device errs"
" independently epoch by epoch, so its")
print("errors average out over a night. Real errors"
" are correlated within a night")
print("(posture,")
print(
"heart-rate patterns), which p12 allows for; the {:.3f} is an upper"
" bound.".format(r)
)
# agreement Cohen's kappa
# scorer A vs scorer B 0.788 0.671
# scorer A vs truth 0.881 0.815
# device vs truth 0.694 0.510
# device vs scorer A 0.637 0.420
# perfect device (= truth) vs scorer A 0.881 0.815
#
# Minutes per night by stage (mean over 200 nights)
# last column: share of epochs of this stage the device gets right
# stage truth device device - truth right
# wake 47.2 43.5 -3.8 0.543
# light 253.1 274.6 +21.5 0.779
# deep 75.2 69.1 -6.2 0.571
# REM 104.4 92.9 -11.5 0.644
#
# Night-to-night correlation of device deep minutes with true deep
# minutes: 0.953
#
# Reading: a validation study compares the device with a scorer, not with
# the truth, so even a perfect device cannot score above the scorers' own
# agreement with the truth. The device's nightly averages can be close while
# it misses almost half of the deep-sleep epochs: errors in both directions
# cancel. Caution: here the device errs independently epoch by epoch, so its
# errors average out over a night. Real errors are correlated within a night
# (posture,
# heart-rate patterns), which p12 allows for; the 0.953 is an upper bound.The first block shows the ruler problem. Two scorers agree with each other on 0.788 of epochs (kappa 0.671), each agrees with the truth on 0.881 (kappa 0.815). The device agrees with the truth at kappa 0.510 but with a scorer only at 0.420 — and a perfect device, identical to the truth, would agree with that scorer at 0.815 and no higher. A validation study measures the device and the scorer together, and caps every device at the scorers’ own accuracy.
The second block shows a subtler point. Nightly averages can look nearly right while epoch-level performance is poor: the device’s deep-sleep minutes average 69.1 against a true 75.2, yet it identifies only 0.571 of the true deep-sleep epochs. Misses and false alarms cancel in the total. In this model, because the device’s errors are independent from epoch to epoch, its nightly deep-sleep minutes still correlate 0.953 with the truth; the program flags that as an upper bound, since real devices make correlated errors within a night.
Evidence
What is known. Comparing seven consumer devices with polysomnography in the same nights, Chinoy and colleagues found that most performed reasonably at separating sleep from wake but that agreement on individual sleep stages was markedly weaker and varied between devices (Chinoy et al. 2021).
A score is a measurement with error
The twelfth program treats the nightly sleep score as true quality plus a measurement error that does not average away within a night.
Calculation
# Chapter 6. A nightly sleep score as a noisy measurement. True sleep quality
# q_t varies from night to night (sd 1, lag-1 autocorrelation 0.3). The
# device's score is s_t = q_t + e_t, with a measurement error e_t of sd
# sigma_e that is independent from night to night but correlated within a
# night (so it does not average away). We ask:
# - how well does tonight's score track tonight's true quality?
# - how well does the CHANGE in score from last night track the change in
# quality?
# - how well does a 7-night average track the 7-night true average?
# - of nights flagged "worse" (score down by 1 sd of its changes or more),
# how many were truly worse by at least 0.5 sd of true quality?
# 20,000 simulated nights per row, fixed seed.
import math
import random
def corr(x, y):
n = len(x)
mx, my = sum(x) / n, sum(y) / n
return sum((a - mx) * (b - my) for a, b in zip(x, y)) / math.sqrt(
sum((a - mx) ** 2 for a in x) * sum((b - my) ** 2 for b in y)
)
def series(sig_e, n, rng, rho=0.3):
q, qs, ss = 0.0, [], []
for _ in range(n):
q = rho * q + math.sqrt(1 - rho * rho) * rng.gauss(0, 1)
qs.append(q)
ss.append(q + rng.gauss(0, sig_e))
return qs, ss
if __name__ == "__main__":
rng = random.Random(57012)
print(
" last column: flagged worse nights truly worse by 0.5 sd or more"
)
print(" error sd night r change r 7-night r truly worse")
for sig in (0.5, 1.0, 1.5):
q, s = series(sig, 20000, rng)
dq = [b - a for a, b in zip(q, q[1:])]
ds = [b - a for a, b in zip(s, s[1:])]
q7 = [sum(q[i:i + 7]) / 7 for i in range(0, 20000 - 7, 7)]
s7 = [sum(s[i:i + 7]) / 7 for i in range(0, 20000 - 7, 7)]
sd_ds = math.sqrt(sum(x * x for x in ds) / len(ds))
flagged = [i for i, x in enumerate(ds) if x <= -sd_ds]
true_worse = sum(1 for i in flagged if dq[i] <= -0.5) / len(flagged)
print(
" {:6.1f} {:6.3f} {:6.3f} {:6.3f} {:6.3f}"
.format(sig, corr(q, s), corr(dq, ds), corr(q7, s7), true_worse)
)
print(
"\nReading: day-to-day changes in the score are noisier than the"
" score itself,"
)
print(
"because the change subtracts two errors while the true change is"
" damped by"
)
print(
"night-to-night correlation. Weekly averages recover most of the"
" signal."
)
# last column: flagged worse nights truly worse by 0.5 sd or more
# error sd night r change r 7-night r truly worse
# 0.5 0.893 0.859 0.928 0.922
# 1.0 0.708 0.640 0.800 0.750
# 1.5 0.552 0.487 0.640 0.645
#
# Reading: day-to-day changes in the score are noisier than the score itself,
# because the change subtracts two errors while the true change is damped by
# night-to-night correlation. Weekly averages recover most of the signal.At a measurement error equal to the night-to-night spread of true quality (the middle row), tonight’s score correlates 0.708 with tonight’s true quality — but the change in score since last night correlates only 0.640 with the true change, because the change subtracts two errors. Of the nights the score flags as clearly worse, 0.750 were truly worse by half a standard deviation or more, so one flagged night in four is noise. A seven-night average recovers more of the signal (0.800). The number most people react to — “my score dropped this morning” — is the least reliable number on the screen.
When the score itself becomes a cause
Baron and colleagues described patients whose pursuit of perfect tracker data had become part of their insomnia and called it “orthosomnia” (Baron et al. 2017). Beliefs about sleep can act on their own: in an experiment, people told that they had slept unusually well performed better on cognitive tests than people told they had slept badly, regardless of how they had actually slept (Draganich and Erdal 2014). The thirteenth program writes one possible mechanism as a feedback loop, and the box below is explicit that it is a hypothesis.
Calculation
# Chapter 6. A hypothesis, written as a model: "orthosomnia" as a feedback
# loop. True sleep quality q_t = -beta x worry_t + (independent night-to-night
# variation). The device reports s_t = q_t + e_t (error sd 1.0, as the middle
# row of p12). Worry the next night responds to a disappointing score:
# worry_{t+1} = a x max(0, 0.5 - s_t), where a is how strongly the sleeper
# reacts to the number (a = 0 means the score is not looked at). beta = 0.4:
# one unit of worry costs 0.4 sd of sleep quality. 50,000 simulated nights per
# row, fixed seed. NOT established - see the hypothesis box.
import math
import random
BETA = 0.4
def run(a, rng, n=50000):
worry, qs = 0.0, []
for _ in range(n):
q = -BETA * worry + rng.gauss(0, 1)
s = q + rng.gauss(0, 1.0)
worry = a * max(0.0, 0.5 - s)
qs.append(q)
m = sum(qs) / n
sd = math.sqrt(sum((x - m) ** 2 for x in qs) / (n - 1))
bad = sum(x < -1.0 for x in qs) / n
return m, sd, bad
if __name__ == "__main__":
rng = random.Random(57013)
print(
" reaction a mean true quality sd share of truly bad nights"
)
print(" (q < -1)")
for a in (0.0, 0.5, 1.0, 1.5, 2.0):
m, sd, bad = run(a, rng)
print(
" {:6.1f} {:+7.3f} {:5.3f} {:6.3f}"
.format(a, m, sd, bad)
)
print(
"\nReading: in this model a sleeper who reacts to the score sleeps"
" worse on"
)
print(
"average than one who ignores it, and more of the bad nights are"
" produced by"
)
print(
"the reaction to measurement error. Whether real sleepers behave like"
" this is"
)
print(
"the question; the model only shows the size such a loop could have."
)
# reaction a mean true quality sd share of truly bad nights
# (q < -1)
# 0.0 +0.007 1.000 0.157
# 0.5 -0.195 1.025 0.214
# 1.0 -0.485 1.105 0.316
# 1.5 -0.988 1.309 0.482
# 2.0 -2.314 1.887 0.740
#
# Reading: in this model a sleeper who reacts to the score sleeps worse on
# average than one who ignores it, and more of the bad nights are produced by
# the reaction to measurement error. Whether real sleepers behave like this is
# the question; the model only shows the size such a loop could have.With no reaction to the score () the share of truly bad nights is 0.157. A sleeper who reacts to a disappointing score with worry the next night sleeps worse on average: −0.195 at , −0.485 at , and the share of bad nights rises to 0.316. Between and the loop starts to run away (mean −0.988, then −2.314): a bad score causes worry, worry causes a bad night, and measurement error alone is enough to start it.
Hypothesis · not established
The author’s hypothesis (not established): in people who check a nightly sleep score, hiding the score for three weeks reduces insomnia symptoms, and the reduction is larger in those who report reacting to the score. What would show it to be wrong: no difference between score-hidden and score-visible periods (Chapter 8 tests this), or a difference unrelated to self-reported reactivity.
Summary
Sleep trackers are validated against human scorers who agree with each other on only about four epochs in five, so no device can be shown to exceed the scorers’ own accuracy, and nightly averages can look right while epoch-level staging is poor. A nightly score is a measurement with error: its change from one night to the next is noisier than the score itself, and a weekly average is far more informative. Beliefs about sleep affect next-day function on their own, and a sleeper who reacts to a noisy score can, in a simple model, turn measurement error into real bad nights.
References
Fan, Y. (2026). Two-Process Sleep Architecture, Slow-Wave Glymphatic Clearance and Thermoregulatory Phase Control: Adenosine Homeostasis, Flip-Flop Switch Bistability, Core-Distal Heat Flux and Sleep-Stage Markov Chains [双过程睡眠架构、慢波胶质淋巴清除与体温节律相位控制] (Version 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.24090162 · Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0).