5D Library › Two-Scale Renormalization, Occupancy Entropy and Thrust-Wave Superposition in Discrete Cyclical Networks · 中文
Chapter 2 Shannon Entropy and Peak of an Occupancy Distribution: The Entropy–Peak Bound Theorem
Fourteen particles, twelve cells
Put identical particles into cells, with particles in cell and . This chapter is concerned only with occupation numbers, not with where the cells stand on the ring, so a state is described by the occupation vector . Entropy, vacancies, peak and variance depend only on the multiset of occupation numbers, that is, on an integer partition of 14 into at most 12 positive parts. There are 133 such partitions, and every theorem in this chapter can be checked on each of them.
The two typical states in the author’s framework are “Fixed-Mode”, with ten cells holding 1 and two cells holding 2 and no vacancy, and “Wave-Rider”, with the particles concentrated in a few cells and many vacancies. The author holds that the two show an “inverse relation” in variance, extremes and tail probabilities, and calls it the “inverse theorem of occupancy entropy and peak thrust”. This chapter states as theorems what can be proved and as statistical tendencies what cannot.
Calculation
Definitions (the author’s model variables). Write the shares , the Shannon entropy in bits (with ) (Shannon 1948), the number of vacancies , the peak (largest occupation) , and the occupation variance . Two further entropies: the Rényi entropy of order 2 (collision entropy) and the min-entropy (Rényi 1961).
The entropy–peak bound theorem
Calculation
Theorem 2.1 (entropy–peak bound theorem; called the “inverse theorem of occupancy entropy and peak thrust” in the author’s framework). For any occupation vector:
Vacancy bound: . Among distributions on only non-empty cells, entropy is largest for the uniform one (Cover and Thomas 2006). Equivalently : the higher the entropy, the fewer the vacancies.
Upper bound from the peak: if , then and is strictly decreasing for . Proof: with the largest share fixed at , the remaining is spread over the other cells, and entropy is largest when it is spread evenly, which gives ; differentiating, exactly when . The bound has the same form as Fano’s inequality in information theory.
Lower bound on the peak: , hence . Proof: is Jensen’s inequality (concavity of the logarithm); gives .
Variance identity: ; with (c), . Variance and collision entropy determine each other, and the lower the Shannon entropy, the higher the lower bound on the variance.
The most even integer pattern: among integer occupations with and , the pattern of largest entropy is unique: “ten 1s and two 2s”, with bits, below the continuous maximum . The proof is the majorisation argument of the next section.
Calculation
The program checks the six relations in (a) to (d) (labelled (a)–(f) in its output) on all 133 patterns, lists the three patterns of highest entropy, and compares, for each peak , the largest entropy actually attainable with the bound .
Calculation
# N = 14 particles in K = 12 cells. Entropy, vacancies, peak and variance depend only on
# the multiset of occupations, so the integer partitions of 14 into at most 12 parts
# cover every case. Check the entropy-peak bounds on all of them.
import math
N, K = 14, 12
def partitions(n, k, top=None):
if n == 0:
yield (); return
for f in range(min(n, n if top is None else top), 0, -1):
if k: yield from ((f,) + r for r in partitions(n - f, k - 1, f))
def stats(part):
p = [x / N for x in part]
h = -sum(q * math.log2(q) for q in p) # Shannon entropy, bits
h2 = -math.log2(sum(q * q for q in p)) # Renyi-2 (collision) entropy
var = sum(x * x for x in part) / K - (N / K) ** 2 # variance over all 12 cells
return h, h2, -math.log2(max(p)), K - len(part), max(part), var
def fano(q):
"""Largest entropy when the biggest share is q >= 1/K (rest spread evenly)."""
r = (1 - q) / (K - 1)
return -q * math.log2(q) - (0 if r == 0 else (1 - q) * math.log2(r))
if __name__ == "__main__":
rows = [(stats(p), p) for p in partitions(N, K)]; e = 1e-12
print("partitions of 14 into at most 12 parts:", len(rows))
checks = {"(a) H <= log2(K - V)": lambda s: s[0] <= math.log2(K - s[3]) + e,
"(b) H <= fano(m / N)": lambda s: s[0] <= fano(s[4] / N) + e,
"(c) H >= H2 >= Hmin": lambda s: s[0] >= s[1] - e and s[1] >= s[2] - e,
"(d) m >= N * 2^(-H)": lambda s: s[4] >= N * 2 ** -s[0] - e,
"(e) Var = (N^2/K)(2^-H2 - 1/K)":
lambda s: abs(s[5] - N * N / K * (2 ** -s[1] - 1 / K)) < 1e-9,
"(f) Var >= (N^2/K)(2^-H - 1/K)":
lambda s: s[5] >= N * N / K * (2 ** -s[0] - 1 / K) - e}
for name, ok in checks.items():
print(f"{name:31s}", all(ok(s) for s, _ in rows))
rows.sort(key=lambda r: -r[0][0])
print("highest-entropy configurations (H in bits; log2 12 = %.4f):" % math.log2(K))
for s, p in rows[:3]:
print(f" {str(p):40s} H {s[0]:.4f} V {s[3]} m {s[4]} Var {s[5]:.3f}")
print("max H for each peak m, against the bound fano(m / N):")
for m in range(2, 8):
best = max(s[0] for s, p in rows if s[4] == m)
print(f" m = {m}: max H {best:.4f} bound {fano(m / N):.4f}")
print("fano(m / N) for m = 2 ... 14 is decreasing:",
all(fano(m / N) > fano((m + 1) / N) for m in range(2, 14)))
# partitions of 14 into at most 12 parts: 133
# (a) H <= log2(K - V) True
# (b) H <= fano(m / N) True
# (c) H >= H2 >= Hmin True
# (d) m >= N * 2^(-H) True
# (e) Var = (N^2/K)(2^-H2 - 1/K) True
# (f) Var >= (N^2/K)(2^-H - 1/K) True
# highest-entropy configurations (H in bits; log2 12 = 3.5850):
# (2, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1) H 3.5216 V 0 m 2 Var 0.139
# (3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1) H 3.4677 V 0 m 3 Var 0.306
# (2, 2, 2, 1, 1, 1, 1, 1, 1, 1, 1) H 3.3788 V 1 m 2 Var 0.306
# max H for each peak m, against the bound fano(m / N):
# m = 2: max H 3.5216 bound 3.5569
# m = 3: max H 3.4677 bound 3.4677
# m = 4: max H 3.2359 bound 3.3341
# m = 5: max H 2.9781 bound 3.1642
# m = 6: max H 2.6995 bound 2.9620
# m = 7: max H 2.4037 bound 2.7297
# fano(m / N) for m = 2 ... 14 is decreasing: TrueAll six inequalities and identities hold on all 133 patterns. The highest entropy belongs to “ten 1s and two 2s” (3.5216 bits, variance 0.139). The second highest, “one 3 and eleven 1s”, also has no vacancy, with entropy 3.4677 and variance 0.306; the third, “three 2s and eight 1s”, has one vacancy, entropy 3.3788 and also variance 0.306. So the framework’s “even = zero vacancies” needs qualifying: more than one pattern has no vacancy, and neither entropy nor variance is fixed by the number of vacancies alone.
The upper bound falls monotonically with , and this is the strict part of “the higher the peak, the lower the entropy”: once one cell holds particles, entropy cannot exceed . But the bound is not always attained. For bound and actual maximum coincide (3.4677); for the actual maximum 3.5216 lies below the bound 3.5569; for the actual maximum 2.4037 lies well below the bound 2.7297, because the integer constraint keeps the remaining particles from being spread truly evenly.
Majorisation: when “low entropy, high peak” is a theorem
Theorem 2.1 gives bounds, not a one-to-one correspondence between entropy and peak. A monotone relation that holds strictly between the two needs a tool of comparison: majorisation.
Calculation
Definition and Theorem 2.2 (majorisation). Sort two occupation vectors with the same total in decreasing order. If every partial sum of the first terms of is at least the corresponding partial sum of (), then majorises , written . Intuitively, is more “concentrated”: can be reached from by a sequence of moves each taking one particle from a fuller cell to an emptier one (Marshall et al. 2011).
If , then , , and . The first three hold because entropy is Schur-concave while the maximum and the sum of squares are Schur-convex. Proof of the fourth: if has non-zero entries, the sum of the first entries of is at least that of , which is , so has at most non-zero entries.
Corollary: “ten 1s and two 2s” is majorised by all 133 patterns, so it has the largest entropy, the smallest peak, the smallest variance and the fewest vacancies, and it is unique (Theorem 2.1(e)).
The catch is that majorisation is only a partial order: many pairs of patterns cannot be compared.
Calculation
The program checks every pair of patterns: are there exceptions to Theorem 2.2 among comparable pairs, and among incomparable pairs, does the lower-entropy pattern also have the lower peak?
Calculation
# a MAJORIZES b if, both sorted in decreasing order, every partial sum of a is >= that of
# b. Then H(a) <= H(b) and max(a) >= max(b) are forced; for incomparable pairs nothing is.
# Count both kinds among the 133 occupancy partitions of 14 into 12 cells.
import math
from itertools import accumulate, combinations
N, K = 14, 12
def partitions(n, k, top=None):
if n == 0:
yield (); return
for f in range(min(n, n if top is None else top), 0, -1):
if k: yield from ((f,) + r for r in partitions(n - f, k - 1, f))
def majorizes(a, b):
pa, pb = list(a) + [0] * (K - len(a)), list(b) + [0] * (K - len(b))
return all(x >= y for x, y in zip(accumulate(pa), accumulate(pb)))
def H(p):
return -sum(x / N * math.log2(x / N) for x in p)
if __name__ == "__main__":
comp = inc = against = 0; viol = {"H": 0, "max": 0, "var": 0, "vacancies": 0}
for a, b in combinations(list(partitions(N, K)), 2):
if majorizes(b, a): a, b = b, a
if majorizes(a, b):
comp += 1
viol["H"] += H(a) > H(b) + 1e-12; viol["max"] += max(a) < max(b)
viol["var"] += sum(x * x for x in a) < sum(x * x for x in b)
viol["vacancies"] += len(a) > len(b)
else: # lower entropy but ALSO lower peak?
inc += 1; lo, hi = (a, b) if H(a) < H(b) else (b, a)
against += max(lo) < max(hi)
print(f"pairs: {comp + inc}; comparable {comp} ({comp / (comp + inc):.1%}),"
f" incomparable {inc}")
print("comparable pairs violating the theorem:", viol)
print(f"incomparable pairs where the lower-entropy one has the LOWER peak: {against}"
f" ({against / inc:.1%})")
a, b = (4, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1), (3, 3, 2, 2, 1, 1, 1, 1)
print(f"example: {str(a):34s} H {H(a):.3f}, max {max(a)}")
print(f" {str(b):34s} H {H(b):.3f}, max {max(b)}")
# pairs: 8778; comparable 7373 (84.0%), incomparable 1405
# comparable pairs violating the theorem: {'H': 0, 'max': 0, 'var': 0, 'vacancies': 0}
# incomparable pairs where the lower-entropy one has the LOWER peak: 926 (65.9%)
# example: (4, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1) H 3.236, max 4
# (3, 3, 2, 2, 1, 1, 1, 1) H 2.842, max 3The 133 patterns form 8,778 pairs. Of these, 7,373 (84.0 %) can be compared by majorisation, and among them Theorem 2.2 has no exception. The remaining 1,405 pairs are not comparable, and in 926 of them (65.9 %) the lower-entropy pattern also has the lower peak. The program’s example: “one 4 and ten 1s” has entropy 3.236 and peak 4, while “two 3s, two 2s and four 1s” has lower entropy (2.842) but peak only 3. The first piles the extra particles into one cell; the second spreads them over several while leaving four cells empty. So “the lower the entropy, the higher the peak” is false as a general proposition. It is a theorem between patterns comparable by majorisation, and guarantees nothing between patterns that are not.
The joint distribution under random allocation
If each of the 14 particles falls independently and with equal probability into one of the 12 cells, the probability of each pattern can be computed exactly: the number of ways of assigning a partition to cells, times the multinomial coefficient , divided by . This is the classical occupancy problem (Feller 1968).
Calculation
The program computes the probability of each pattern exactly with fractions, derives the distributions of vacancies, peak and entropy and their correlations, and checks them against 100,000 random throws.
Calculation
# Throw N = 14 particles independently and uniformly into K = 12 cells. Each occupancy
# partition has an exact probability: (labelled arrangements) x multinomial / 12^14.
import math, random
from fractions import Fraction
from collections import Counter
N, K = 14, 12
def partitions(n, k, top=None):
if n == 0:
yield (); return
for f in range(min(n, n if top is None else top), 0, -1):
if k: yield from ((f,) + r for r in partitions(n - f, k - 1, f))
def prob(part):
cells = math.factorial(K) // math.factorial(K - len(part))
for c in Counter(part).values(): cells //= math.factorial(c)
ways = math.factorial(N)
for x in part: ways //= math.factorial(x)
return Fraction(cells * ways, K ** N)
def entropy(part):
return -sum(x / N * math.log2(x / N) for x in part)
def corr(xs, ys, ws):
mx = sum(w * x for x, w in zip(xs, ws)); my = sum(w * y for y, w in zip(ys, ws))
cov = sum(w * (x - mx) * (y - my) for x, y, w in zip(xs, ys, ws))
vx = sum(w * (x - mx) ** 2 for x, w in zip(xs, ws))
return cov / math.sqrt(vx * sum(w * (y - my) ** 2 for y, w in zip(ys, ws)))
if __name__ == "__main__":
parts = list(partitions(N, K)); ps = [prob(p) for p in parts]
print("total probability:", sum(ps))
w = [float(p) for p in ps]
H = [entropy(p) for p in parts]; V = [K - len(p) for p in parts]
M = [max(p) for p in parts]; S = [sum(x * x for x in p) / K for p in parts]
ev = sum(wi * v for wi, v in zip(w, V))
print(f"E[vacancies] = {ev:.4f}; formula K(1 - 1/K)^N = {K * (1 - 1 / K) ** N:.4f}")
dist = Counter()
for wi, v in zip(w, V): dist[v] += wi
print("P(V = v): " + " ".join(f"{v}:{dist[v]:.4f}" for v in range(0, 8)))
uni = ps[parts.index((2, 2) + (1,) * 10)]
print(f"P(most even state, ten 1s and two 2s) = {float(uni):.6f}"
f" (1 in {1 / uni:,.0f})")
for m in (3, 4, 5, 6):
print(f"P(peak m >= {m}) = {sum(wi for wi, x in zip(w, M) if x >= m):.4f}")
print(f"E[H] = {sum(wi * h for wi, h in zip(w, H)):.4f} bits (maximum 3.5216)")
print("exact correlations under random allocation:")
print(f" H with V {corr(H, V, w):+.3f} | H with m {corr(H, M, w):+.3f}"
f" | H with Var {corr(H, S, w):+.3f} | V with m {corr(V, M, w):+.3f}")
rng = random.Random(14); hs, vs = [], [] # Monte Carlo check
for _ in range(100000):
part = tuple(Counter(rng.randrange(K) for _ in range(N)).values())
hs.append(entropy(part)); vs.append(K - len(part))
print(f"Monte Carlo (100,000 throws): E[V] {sum(vs) / len(vs):.4f},"
f" corr(H, V) {corr(hs, vs, [1 / len(hs)] * len(hs)):+.3f},"
f" P(V = 0) {vs.count(0) / len(vs):.4f}")
# total probability: 1
# E[vacancies] = 3.5493; formula K(1 - 1/K)^N = 3.5493
# P(V = v): 0:0.0013 1:0.0246 2:0.1404 3:0.3193 4:0.3251 5:0.1534 6:0.0329 7:0.0030
# P(most even state, ten 1s and two 2s) = 0.001120 (1 in 893)
# P(peak m >= 3) = 0.8493
# P(peak m >= 4) = 0.2798
# P(peak m >= 5) = 0.0507
# P(peak m >= 6) = 0.0067
# E[H] = 2.9135 bits (maximum 3.5216)
# exact correlations under random allocation:
# H with V -0.959 | H with m -0.708 | H with Var -0.956 | V with m +0.514
# Monte Carlo (100,000 throws): E[V] 3.5482, corr(H, V) -0.959, P(V = 0) 0.0011The exact calculation gives a mean of 3.5493 vacancies, in agreement with the formula . The probability of no vacancy at all is only 0.0013, and the most even pattern, “ten 1s and two 2s”, has probability 0.00112, about once in 893 throws. Three or four vacancies are most common (probabilities 0.3193 and 0.3251). The peak is at least 4 with probability 0.2798, at least 5 with 0.0507 and at least 6 with only 0.0067. The mean entropy is 2.9135 bits, about 0.6 bits below the maximum.
The exact correlations among the four quantities are: entropy with vacancies −0.959, entropy with variance −0.956, entropy with peak −0.708, and vacancies with peak +0.514. A Monte Carlo run of 100,000 throws gives almost the same numbers. Under random allocation, then, entropy, vacancies and variance are nearly different readings of one quantity, whereas the link between entropy and peak is much looser: the squared correlation is only 0.50. The “inverse relation” the author describes does exist under random allocation, but it is a statistical tendency whose strength depends on which quantity measures the “peak”; calling it “conservation” or an “inverse law” is inaccurate.
Note
Two kinds of “entropy”. The entropy here is Shannon entropy, the uncertainty about which cell a particle lies in, measured in bits. It has the same mathematical form as thermodynamic entropy (Boltzmann’s entropy is the logarithm of the number of microstates), but there is no temperature, energy or second law in this book: particles do not spontaneously “tend towards high entropy” unless we stipulate some random process of redistribution. When the book calls a pattern “high-entropy” it is only describing the shape of its distribution.
Hypothesis · not established
Author’s hypothesis: concentration and extremes (not yet established). The author conjectures that in real systems in which money, attention or people are allocated among a fixed number of categories, periods of lower allocation entropy are followed more often by extreme moves (for example, one category’s share changing by more than a threshold fixed in advance within one period). This goes beyond Theorem 2.1: the theorem bounds entropy and peak at the same moment and says nothing about low entropy predicting future extremes. A testable prediction: for a pre-registered data set and threshold, the frequency of next-period extreme moves in the lowest entropy quintile exceeds that in the highest. If the 95 % confidence interval for the difference contains no positive values, the hypothesis is refuted. A program in Chapter 5 shows that such a correlation can appear even in a null model with no mechanism at all, so the test must be made against a suitable null model.
Chapter summary
What can be proved are bounds: ; for a given peak , with decreasing for ; ; and an identity linking variance and collision entropy. With 14 particles in 12 cells, the integer pattern of largest entropy is unique, “ten 1s and two 2s” (3.5216 bits). “The lower the entropy, the higher the peak” is a theorem only between patterns comparable by majorisation; of 8,778 pairs, 1,405 are not comparable, and in 65.9 % of those the direction is reversed. Under random allocation the mean number of vacancies is 3.55, no vacancy occurs with probability only 0.0013, and the correlation between entropy and peak is −0.708: a statistical tendency, not a conservation law. The next chapter adds the author’s excitation rules on top of occupation and asks what “an empty cell borrows half from the one opposite” brings.
References
Fan, Y. (2026). Two-Scale Renormalization, Occupancy Entropy and Thrust-Wave Superposition in Discrete Cyclical Networks: Algebraic Topology and Monte Carlo Verification on a Twelve-Node Ring Lattice [离散周期网络中的双尺度重整化、占位数分布熵与推力波叠加定理] (Version 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.23796761 · Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0).