What does Dense Data Workbenches conclude, in numbers, about “Practice: Rebuilding the Harbourbridge Reconciliation Back Office from Diagnosis to Redesign”?

Answer: Chapter 8 takes Harbourbridge's reconciliation back office through the whole method: a two-week planted-row audit gives a baseline miss rate of 0.683; keeping red for four reconciliation rules cuts red rows from 435 to 40 with a positive predictive value of 0.425, and the modelled miss rate falls from 0.664 to 0.089, or to 0.339 if the new rules reach only six tenths of their set sensitivity; a state machine, an amount mask, a currency default and counted, undoable batch actions cut the rework rate from 0.0064 to 0.0013; seconds per row fall by only 0.49 and cannot be pinned down in eight weeks, so they serve only as a secondary metric. Marked boxes separate published evidence, runnable calculations, sources and the author's hypotheses, each stating what would refute it; the source studied is the reporting system of the Tang finance minister Liu Yan, given in paraphrase.

Scope of the numbers: The book is not accounting, audit or legal advice. Marked boxes separate published evidence, runnable calculations, sources and the author's hypotheses, each stating what would refute it; the source studied is the reporting system of the Tang finance minister Liu Yan, given in paraphrase.

中文版

Data table

The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 0.089, 0.339, 0.425, 0.664, 40

new red rule                    fires  anomalies  PPV
N1 difference beyond RM 1.00       20          9  0.450
N2 duplicate pair                   9          3  0.333
N3 credit without order             8          2  0.250
N4 paid, no credit after 2 days     3          3  1.000
new: red rows 40 (share 0.033), anomalies in red 17 of 18, PPV 0.425, red per screen 0.83
icon rows 412, folded clean S7/S9 rows 180, anomalies inside folded rows 0
expected anomalies caught a day: old 6.04, new 16.40; miss rate old 0.664, new 0.089
proper checks left for icon rows after red: 20; anomalies reached through icons 0
seconds per row: old 12.34, new 11.85; change -0.49
budget  30: new view miss rate 0.283
budget  45: new view miss rate 0.089
budget  60: new view miss rate 0.089
budget  90: new view miss rate 0.089
sensitivity of the new red rules scaled by m (expected values, all red rows checked):
  m 1.0: expected miss rate 0.066
  m 0.8: expected miss rate 0.202
  m 0.6: expected miss rate 0.339
  m 0.4: expected miss rate 0.476
Show the full program
# Chapter 8. p81: The redesigned view on the same day: four reconciliation-identity red rules, icons, budget.
# Rebuilds the p10 day (seed 20261112). New red rules test the reconciliation identity itself; their firing
# probabilities are settings, not measurements. The old R2, R3, R4, R5, R6 become icons in one narrow column.
# Default view "to handle": red rows first, then rows ranked by icons (R6, R2 first); fully clean rows in
# S7 or S9 fold into one summary line. Same detection and time settings as p80 for a fair comparison.
import math
import random

SEED, ROWS, SCREEN, BUDGET, P_CHECK, P_GLANCE = 20261112, 1200, 25, 60, 0.95, 0.25
OPS = ["Ah Rong", "Chee Ming", "Mei Ling", "Ka Ho"]
STATUSES = [("S1", 0.06), ("S2", 0.14), ("S3", 0.10), ("S4", 0.12), ("S5", 0.11), ("S6", 0.17),
            ("S7", 0.20), ("S8", 0.04), ("S9", 0.06)]
CHANNELS = [("FPX transfer", 0.46), ("card", 0.24), ("e-wallet", 0.22), ("direct bank transfer", 0.08)]
CURRENCIES = [("JPY", 0.45), ("KRW", 0.25), ("TWD", 0.15), ("MYR", 0.15)]
RM_PER_UNIT = {"JPY": 0.030, "KRW": 0.0033, "TWD": 0.14, "MYR": 1.0}   # sample rates (assumed)
ANOMALIES = [("A1", 6), ("A2", 3), ("A3", 4), ("A4", 2), ("A5", 3)]
RULES = [("R1", 0.12, {"A1": 1.0, "A3": 1.0, "A2": 0.3, "A4": 0.3, "A5": 0.3}),
         ("R2", 0.10, {"A1": 0.5, "A2": 0.5, "A3": 0.5, "A4": 0.5, "A5": 0.9}),
         ("R3", None, None),
         ("R4", 0.08, {"A1": 0.2, "A2": 0.2, "A3": 0.2, "A4": 0.4, "A5": 0.2}),
         ("R5", None, None),
         ("R6", 0.04, {"A1": 0.3, "A2": 0.3, "A3": 0.3, "A4": 0.3, "A5": 0.3})]


def pick(rng, table):
    u, acc = rng.random(), 0.0
    for row in table:
        acc += row[-1]
        if u < acc:
            return row[0]
    return table[-1][0]


def make_day(seed):
    rng = random.Random(seed)
    rows = []
    for _ in range(ROWS):
        amt = round(math.exp(math.log(180.0) + 0.9 * rng.gauss(0.0, 1.0)), 2)
        rows.append({"op": OPS[rng.randrange(4)], "st": pick(rng, STATUSES), "ch": pick(rng, CHANNELS),
                     "cur": pick(rng, CURRENCIES), "amt": amt, "an": None})
    idx = rng.sample(range(ROWS), sum(n for _, n in ANOMALIES))
    k = 0
    for code, n in ANOMALIES:
        for _ in range(n):
            rows[idx[k]]["an"] = code
            k += 1
    for r in rows:
        r["fired"] = []
        for name, p0, p1 in RULES:
            if name == "R3":
                hit = r["amt"] >= 1000
            elif name == "R5":
                hit = r["ch"] == "e-wallet" and r["st"] in ("S1", "S2")
            else:
                hit = rng.random() < (p1[r["an"]] if r["an"] else p0)
            if hit:
                r["fired"].append(name)
    return rows


NEW = [("N1 difference beyond RM 1.00", 0.010, {"A1": 1.0, "A3": 0.9}),
       ("N2 duplicate pair", 0.003, {"A2": 1.0}),
       ("N3 credit without order", 0.002, {"A4": 1.0}),
       ("N4 paid, no credit after 2 days", 0.002, {"A5": 1.0})]


def catch_and_time(rows, red, ranked, folded, budget):
    checked = set(id(r) for r in red[:budget])
    for r in ranked[:max(0, budget - len(red))]:
        checked.add(id(r))
    caught = 0.0
    for r in rows:
        if r["an"]:
            if id(r) in checked:
                caught += P_CHECK
            elif id(r) not in folded:
                caught += P_GLANCE
    return caught, checked


rows = make_day(SEED)
n_an = sum(1 for r in rows if r["an"])
rng = random.Random(SEED + 81)
for r in rows:
    r["new"] = [nm for nm, p0, p1 in NEW if rng.random() < (p1.get(r["an"], 0.0) if r["an"] else p0)]
    r["tie"] = rng.random()
print("new red rule                    fires  anomalies  PPV")
for nm, _, _ in NEW:
    f = [r for r in rows if nm in r["new"]]
    tp = sum(1 for r in f if r["an"])
    print("{:31s} {:5d}  {:9d}  {:.3f}".format(nm, len(f), tp, tp / len(f) if f else 0.0))
red_new = [r for r in rows if r["new"]]
tp_new = sum(1 for r in red_new if r["an"])
rest = [r for r in rows if not r["new"]]
ranked = sorted([r for r in rest if r["fired"]],
                key=lambda r: (-("R6" in r["fired"]) - ("R2" in r["fired"]), -len(r["fired"]), r["tie"]))
folded = set(id(r) for r in rest if not r["fired"] and r["st"] in ("S7", "S9"))
print("new: red rows {} (share {:.3f}), anomalies in red {} of {}, PPV {:.3f}, red per screen {:.2f}".format(
    len(red_new), len(red_new) / ROWS, tp_new, n_an, tp_new / len(red_new), len(red_new) / (ROWS / SCREEN)))
print("icon rows {}, folded clean S7/S9 rows {}, anomalies inside folded rows {}".format(
    len(ranked), len(folded), sum(1 for r in rows if id(r) in folded and r["an"])))
red_old = [r for r in rows if r["fired"]]
old_caught = 0.0
p_chk = min(1.0, BUDGET / len(red_old))
for r in rows:
    if r["an"]:
        p = p_chk if r["fired"] else 0.0
        old_caught += p * P_CHECK + (1 - p) * P_GLANCE
new_caught, checked = catch_and_time(rows, red_new, ranked, folded, BUDGET)
print("expected anomalies caught a day: old {:.2f}, new {:.2f}; miss rate old {:.3f}, new {:.3f}".format(
    old_caught, new_caught, 1 - old_caught / n_an, 1 - new_caught / n_an))
spare = BUDGET - len(red_new)
print("proper checks left for icon rows after red: {}; anomalies reached through icons {}".format(
    spare, sum(1 for r in ranked[:spare] if r["an"])))
old_sec = ROWS * 4.0 + BUDGET * 120.0 + (len(red_old) - BUDGET) * 0.5 * 15.0
new_sec = (ROWS - len(folded)) * 4.0 + BUDGET * 120.0 + (len(ranked) - max(0, spare)) * 0.5 * 15.0
print("seconds per row: old {:.2f}, new {:.2f}; change {:+.2f}".format(
    old_sec / ROWS, new_sec / ROWS, (new_sec - old_sec) / ROWS))
for budget in (30, 45, 60, 90):
    c, _ = catch_and_time(rows, red_new, ranked, folded, budget)
    print("budget {:3d}: new view miss rate {:.3f}".format(budget, 1 - c / n_an))
print("sensitivity of the new red rules scaled by m (expected values, all red rows checked):")
for m in (1.0, 0.8, 0.6, 0.4):
    found = 0.0
    for code, n in ANOMALIES:
        p_red = 1.0
        for _, _, p1 in NEW:
            p_red *= 1.0 - m * p1.get(code, 0.0)
        p_red = 1.0 - p_red
        found += n * (p_red * P_CHECK + (1 - p_red) * P_GLANCE)
    print("  m {:.1f}: expected miss rate {:.3f}".format(m, 1 - found / n_an))

Single source and citation

Fan, Y. (2026). Cognitive Unloading of Dense Data Workbenches, Conditional-Format Salience and Zero-Training Error-Proof Form Engineering (Version 0.1.0), Chapter 8: Practice: Rebuilding the Harbourbridge Reconciliation Back Office from Diagnosis to Redesign. 5D Library. DOI pending

DOI pending · monograph page goes live after publication · 5DL-B-138 · v0.1.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026138,
  author    = {Fan, Yafu},
  title     = {Cognitive Unloading of Dense Data Workbenches, Conditional-Format Salience and Zero-Training Error-Proof Form Engineering},
  year      = {2026},
  publisher = {5D Library},
  edition   = {0.1.0},
  note      = {DOI pending}
}

Other chapter answers in this book