What does Dense Data Workbenches conclude, in numbers, about “Practice: Rebuilding the Harbourbridge Reconciliation Back Office from Diagnosis to Redesign”?
Answer: Chapter 8 takes Harbourbridge's reconciliation back office through the whole method: a two-week planted-row audit gives a baseline miss rate of 0.683; keeping red for four reconciliation rules cuts red rows from 435 to 40 with a positive predictive value of 0.425, and the modelled miss rate falls from 0.664 to 0.089, or to 0.339 if the new rules reach only six tenths of their set sensitivity; a state machine, an amount mask, a currency default and counted, undoable batch actions cut the rework rate from 0.0064 to 0.0013; seconds per row fall by only 0.49 and cannot be pinned down in eight weeks, so they serve only as a secondary metric. Marked boxes separate published evidence, runnable calculations, sources and the author's hypotheses, each stating what would refute it; the source studied is the reporting system of the Tang finance minister Liu Yan, given in paraphrase.
Scope of the numbers: The book is not accounting, audit or legal advice. Marked boxes separate published evidence, runnable calculations, sources and the author's hypotheses, each stating what would refute it; the source studied is the reporting system of the Tang finance minister Liu Yan, given in paraphrase.
Data table
The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 0.089, 0.339, 0.425, 0.664, 40
new red rule fires anomalies PPV N1 difference beyond RM 1.00 20 9 0.450 N2 duplicate pair 9 3 0.333 N3 credit without order 8 2 0.250 N4 paid, no credit after 2 days 3 3 1.000 new: red rows 40 (share 0.033), anomalies in red 17 of 18, PPV 0.425, red per screen 0.83 icon rows 412, folded clean S7/S9 rows 180, anomalies inside folded rows 0 expected anomalies caught a day: old 6.04, new 16.40; miss rate old 0.664, new 0.089 proper checks left for icon rows after red: 20; anomalies reached through icons 0 seconds per row: old 12.34, new 11.85; change -0.49 budget 30: new view miss rate 0.283 budget 45: new view miss rate 0.089 budget 60: new view miss rate 0.089 budget 90: new view miss rate 0.089 sensitivity of the new red rules scaled by m (expected values, all red rows checked): m 1.0: expected miss rate 0.066 m 0.8: expected miss rate 0.202 m 0.6: expected miss rate 0.339 m 0.4: expected miss rate 0.476
Show the full program
# Chapter 8. p81: The redesigned view on the same day: four reconciliation-identity red rules, icons, budget.
# Rebuilds the p10 day (seed 20261112). New red rules test the reconciliation identity itself; their firing
# probabilities are settings, not measurements. The old R2, R3, R4, R5, R6 become icons in one narrow column.
# Default view "to handle": red rows first, then rows ranked by icons (R6, R2 first); fully clean rows in
# S7 or S9 fold into one summary line. Same detection and time settings as p80 for a fair comparison.
import math
import random
SEED, ROWS, SCREEN, BUDGET, P_CHECK, P_GLANCE = 20261112, 1200, 25, 60, 0.95, 0.25
OPS = ["Ah Rong", "Chee Ming", "Mei Ling", "Ka Ho"]
STATUSES = [("S1", 0.06), ("S2", 0.14), ("S3", 0.10), ("S4", 0.12), ("S5", 0.11), ("S6", 0.17),
("S7", 0.20), ("S8", 0.04), ("S9", 0.06)]
CHANNELS = [("FPX transfer", 0.46), ("card", 0.24), ("e-wallet", 0.22), ("direct bank transfer", 0.08)]
CURRENCIES = [("JPY", 0.45), ("KRW", 0.25), ("TWD", 0.15), ("MYR", 0.15)]
RM_PER_UNIT = {"JPY": 0.030, "KRW": 0.0033, "TWD": 0.14, "MYR": 1.0} # sample rates (assumed)
ANOMALIES = [("A1", 6), ("A2", 3), ("A3", 4), ("A4", 2), ("A5", 3)]
RULES = [("R1", 0.12, {"A1": 1.0, "A3": 1.0, "A2": 0.3, "A4": 0.3, "A5": 0.3}),
("R2", 0.10, {"A1": 0.5, "A2": 0.5, "A3": 0.5, "A4": 0.5, "A5": 0.9}),
("R3", None, None),
("R4", 0.08, {"A1": 0.2, "A2": 0.2, "A3": 0.2, "A4": 0.4, "A5": 0.2}),
("R5", None, None),
("R6", 0.04, {"A1": 0.3, "A2": 0.3, "A3": 0.3, "A4": 0.3, "A5": 0.3})]
def pick(rng, table):
u, acc = rng.random(), 0.0
for row in table:
acc += row[-1]
if u < acc:
return row[0]
return table[-1][0]
def make_day(seed):
rng = random.Random(seed)
rows = []
for _ in range(ROWS):
amt = round(math.exp(math.log(180.0) + 0.9 * rng.gauss(0.0, 1.0)), 2)
rows.append({"op": OPS[rng.randrange(4)], "st": pick(rng, STATUSES), "ch": pick(rng, CHANNELS),
"cur": pick(rng, CURRENCIES), "amt": amt, "an": None})
idx = rng.sample(range(ROWS), sum(n for _, n in ANOMALIES))
k = 0
for code, n in ANOMALIES:
for _ in range(n):
rows[idx[k]]["an"] = code
k += 1
for r in rows:
r["fired"] = []
for name, p0, p1 in RULES:
if name == "R3":
hit = r["amt"] >= 1000
elif name == "R5":
hit = r["ch"] == "e-wallet" and r["st"] in ("S1", "S2")
else:
hit = rng.random() < (p1[r["an"]] if r["an"] else p0)
if hit:
r["fired"].append(name)
return rows
NEW = [("N1 difference beyond RM 1.00", 0.010, {"A1": 1.0, "A3": 0.9}),
("N2 duplicate pair", 0.003, {"A2": 1.0}),
("N3 credit without order", 0.002, {"A4": 1.0}),
("N4 paid, no credit after 2 days", 0.002, {"A5": 1.0})]
def catch_and_time(rows, red, ranked, folded, budget):
checked = set(id(r) for r in red[:budget])
for r in ranked[:max(0, budget - len(red))]:
checked.add(id(r))
caught = 0.0
for r in rows:
if r["an"]:
if id(r) in checked:
caught += P_CHECK
elif id(r) not in folded:
caught += P_GLANCE
return caught, checked
rows = make_day(SEED)
n_an = sum(1 for r in rows if r["an"])
rng = random.Random(SEED + 81)
for r in rows:
r["new"] = [nm for nm, p0, p1 in NEW if rng.random() < (p1.get(r["an"], 0.0) if r["an"] else p0)]
r["tie"] = rng.random()
print("new red rule fires anomalies PPV")
for nm, _, _ in NEW:
f = [r for r in rows if nm in r["new"]]
tp = sum(1 for r in f if r["an"])
print("{:31s} {:5d} {:9d} {:.3f}".format(nm, len(f), tp, tp / len(f) if f else 0.0))
red_new = [r for r in rows if r["new"]]
tp_new = sum(1 for r in red_new if r["an"])
rest = [r for r in rows if not r["new"]]
ranked = sorted([r for r in rest if r["fired"]],
key=lambda r: (-("R6" in r["fired"]) - ("R2" in r["fired"]), -len(r["fired"]), r["tie"]))
folded = set(id(r) for r in rest if not r["fired"] and r["st"] in ("S7", "S9"))
print("new: red rows {} (share {:.3f}), anomalies in red {} of {}, PPV {:.3f}, red per screen {:.2f}".format(
len(red_new), len(red_new) / ROWS, tp_new, n_an, tp_new / len(red_new), len(red_new) / (ROWS / SCREEN)))
print("icon rows {}, folded clean S7/S9 rows {}, anomalies inside folded rows {}".format(
len(ranked), len(folded), sum(1 for r in rows if id(r) in folded and r["an"])))
red_old = [r for r in rows if r["fired"]]
old_caught = 0.0
p_chk = min(1.0, BUDGET / len(red_old))
for r in rows:
if r["an"]:
p = p_chk if r["fired"] else 0.0
old_caught += p * P_CHECK + (1 - p) * P_GLANCE
new_caught, checked = catch_and_time(rows, red_new, ranked, folded, BUDGET)
print("expected anomalies caught a day: old {:.2f}, new {:.2f}; miss rate old {:.3f}, new {:.3f}".format(
old_caught, new_caught, 1 - old_caught / n_an, 1 - new_caught / n_an))
spare = BUDGET - len(red_new)
print("proper checks left for icon rows after red: {}; anomalies reached through icons {}".format(
spare, sum(1 for r in ranked[:spare] if r["an"])))
old_sec = ROWS * 4.0 + BUDGET * 120.0 + (len(red_old) - BUDGET) * 0.5 * 15.0
new_sec = (ROWS - len(folded)) * 4.0 + BUDGET * 120.0 + (len(ranked) - max(0, spare)) * 0.5 * 15.0
print("seconds per row: old {:.2f}, new {:.2f}; change {:+.2f}".format(
old_sec / ROWS, new_sec / ROWS, (new_sec - old_sec) / ROWS))
for budget in (30, 45, 60, 90):
c, _ = catch_and_time(rows, red_new, ranked, folded, budget)
print("budget {:3d}: new view miss rate {:.3f}".format(budget, 1 - c / n_an))
print("sensitivity of the new red rules scaled by m (expected values, all red rows checked):")
for m in (1.0, 0.8, 0.6, 0.4):
found = 0.0
for code, n in ANOMALIES:
p_red = 1.0
for _, _, p1 in NEW:
p_red *= 1.0 - m * p1.get(code, 0.0)
p_red = 1.0 - p_red
found += n * (p_red * P_CHECK + (1 - p_red) * P_GLANCE)
print(" m {:.1f}: expected miss rate {:.3f}".format(m, 1 - found / n_an))Single source and citation
Fan, Y. (2026). Cognitive Unloading of Dense Data Workbenches, Conditional-Format Salience and Zero-Training Error-Proof Form Engineering (Version 0.1.0), Chapter 8: Practice: Rebuilding the Harbourbridge Reconciliation Back Office from Diagnosis to Redesign. 5D Library. DOI pending
DOI pending · monograph page goes live after publication · 5DL-B-138 · v0.1.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026138,
author = {Fan, Yafu},
title = {Cognitive Unloading of Dense Data Workbenches, Conditional-Format Salience and Zero-Training Error-Proof Form Engineering},
year = {2026},
publisher = {5D Library},
edition = {0.1.0},
note = {DOI pending}
}Other chapter answers in this book
- Chapter 1 · Method: Three Losses, Four Metrics, Evidence Grades and the Liu Yan Reporting System
- Chapter 2 · Scan Paths and the Skeleton of a Table: Column Order, Alignment, Frozen Panes and Density
- Chapter 3 · Preattentive Salience: Let the Anomalies Jump Out, and Only the Anomalies
- Chapter 4 · Alarm Thresholds: Signal Detection, ROC, the Cost Matrix and a Per-Screen Alarm Cap
- Chapter 6 · Zero-Training Error-Proof Forms: Constraints, Defaults, Inline Validation, Confirm-or-Undo and Batch Guards
- Chapter 7 · The Workbench Playbook: Reconciliation, Orders, Stock and Customer Records, and the Distillation Spec