What does Joining Genealogies and the Perpetual Clan Estate conclude, in numbers, about “Keeping a Ledger of Likelihood Ratios: Two Thresholds and a Review Zone”?
Answer: Chapter 2 uses Fellegi-Sunter likelihood ratios with two thresholds to cut false merges from 36.33 to 0.612 at the cost of reviewing 15.4 pairs, and estimates the parameters by expectation-maximisation.
Scope of the numbers: All data are synthetic or illustrative, all calculations come from its programs, and it is general analysis, not legal, financial or business advice; it compares no lineages or ethnic groups. Marked boxes separate published evidence, runnable Python calculations (20 programs), classical sources and the author's hypotheses.
Data table
The numbers below are printed verbatim by the chapter's Python program and can be reproduced by running it. standard-library Python onlyprinted values that match the answer: 0.612, 15.4, 36.33
| A. field weights in bits (illustrative m, u) | |
| field m u agree disagree | |
| given name | 0.95 0.0152 +5.97 -4.30 |
| birth year +-2 | 0.90 0.2500 +1.85 -2.91 |
| father name | 0.85 0.0152 +5.81 -2.71 |
| village | 0.80 0.2000 +2.00 -2.00 |
| spouse surname | 0.75 0.1000 +2.91 -1.85 |
| B. 32 patterns | 9 link, 9 review, 14 non-link |
| upper threshold +8.26 bits, lower threshold +0.50 bits | |
| P(link | non-match) mu | 0.000256 |
| P(non-link | match) lambda | 0.0147 |
| P(review | match) | 0.0820; P(review | non-match) = 0.0061 |
| C. expected counts for the 60 x 40 books of p12 | |
| false links 0.612, missed links 0.147, review pairs 15.4 | |
| name-only rule: false links 36.33, missed links 0.50 |
Show the full program
# Chapter 2. p21: Fellegi-Sunter weights, two thresholds, error rates.
# Model (Fellegi and Sunter 1969): for each field k a pair agrees with
# probability m_k if it is a true match and u_k if it is not. Under
# conditional independence the log-likelihood ratio of a comparison
# pattern is the sum of field weights log2(m/u) on agreement and
# log2((1-m)/(1-u)) on disagreement. Sorting all 2^5 patterns by weight
# and cutting from the top until P(link | non-match) would exceed MU,
# and from the bottom until P(non-link | match) would exceed LAM, gives
# the upper and lower thresholds; patterns in between go to clerical
# review. m and u values are illustrative; u for names is the sum of
# p^2 from p11 (Zipf s = 0.8).
import math
from itertools import product
FIELDS = [("given name", 0.95, 0.0152), ("birth year +-2", 0.90, 0.25),
("father name", 0.85, 0.0152), ("village", 0.80, 0.20),
("spouse surname", 0.75, 0.10)]
MU, LAM = 0.0005, 0.02
NM, NU = 10, 2390 # the 60 x 40 books of p12
def patterns():
rows = []
for g in product((1, 0), repeat=len(FIELDS)):
pm, pu, wts = 1.0, 1.0, []
for a, (_, m, u) in zip(g, FIELDS):
pm *= m if a else 1.0 - m
pu *= u if a else 1.0 - u
wts.append(math.log2(m / u) if a else
math.log2((1.0 - m) / (1.0 - u)))
rows.append((math.fsum(wts), pm, pu, g))
rows.sort(key=lambda r: (-r[0], r[3]))
return rows
def thresholds(rows):
up, acc = 0, []
while up < len(rows) and math.fsum(acc + [rows[up][2]]) <= MU:
acc.append(rows[up][2])
up += 1
lo, acc = len(rows), []
while lo > up and math.fsum(acc + [rows[lo - 1][1]]) <= LAM:
acc.append(rows[lo - 1][1])
lo -= 1
return up, lo
if __name__ == "__main__":
print("A. field weights in bits (illustrative m, u)")
print(" field m u agree disagree")
for name, m, u in FIELDS:
print(" {:<15} {:.2f} {:.4f} {:>+6.2f} {:>+6.2f}".format(
name, m, u, math.log2(m / u),
math.log2((1.0 - m) / (1.0 - u))))
rows = patterns()
up, lo = thresholds(rows)
link, rev, non = rows[:up], rows[up:lo], rows[lo:]
def mass(rs, i):
return math.fsum(r[i] for r in rs)
print("B. 32 patterns: {} link, {} review, {} non-link".format(
len(link), len(rev), len(non)))
print(" upper threshold {:+.2f} bits, lower threshold {:+.2f} bits"
.format(link[-1][0], non[0][0]))
print(" P(link | non-match) mu = {:.6f}".format(mass(link, 2)))
print(" P(non-link | match) lambda = {:.4f}".format(mass(non, 1)))
print(" P(review | match) = {:.4f}; P(review | non-match) = {:.4f}"
.format(mass(rev, 1), mass(rev, 2)))
print("C. expected counts for the 60 x 40 books of p12")
print(" false links {:.3f}, missed links {:.3f}, review pairs {:.1f}"
.format(NU * mass(link, 2), NM * mass(non, 1),
NM * mass(rev, 1) + NU * mass(rev, 2)))
print(" name-only rule: false links {:.2f}, missed links {:.2f}"
.format(NU * FIELDS[0][2], NM * (1.0 - FIELDS[0][1])))Single source and citation
Fan, Y. (2026). Genealogical DAG Merge Algorithms, Generation-Poem Congruence Coding and the Institutional Economics of the Fan Clan Charitable Estate (Version 1.0.0), Chapter 2: Keeping a Ledger of Likelihood Ratios: Two Thresholds and a Review Zone. 5D Library. DOI pending
DOI pending · monograph page goes live after publication · 5DL-B-334 · v1.0.0 · CC-BY-NC-ND-4.0
BibTeX
@book{fan2026334,
author = {Fan, Yafu},
title = {Genealogical DAG Merge Algorithms, Generation-Poem Congruence Coding and the Institutional Economics of the Fan Clan Charitable Estate},
year = {2026},
publisher = {5D Library},
edition = {1.0.0},
note = {DOI pending}
}Other chapter answers in this book
- Chapter 1 · Same Name, Same Generation: Collisions Are Far Commoner Than They Look
- Chapter 3 · A Lineage Is a Directed Acyclic Graph: Topological Sorting and Cycle Detection
- Chapter 4 · The Generation Poem Is a Congruence Code: Dropped Generations, Extra Generations and Collisions Across Cycles
- Chapter 5 · Going to the Southern Seas Was Chain Migration: Branching Processes and the Crowding of Ports
- Chapter 6 · The Perpetual Charitable Estate: Extinction Probabilities, a Ration Ledger and the Risk of Governance
- Chapter 7 · The Classics Examined: Fan Zhongyan's Rules for the Charitable Estate and the Extinction Problem of Galton and Watson
- Chapter 8 · The Quarterly Merge Check-up: Procedure, Distilled Tool and Pre-registered Refutation