5D Library › Acoustic Persuasion, Prosodic Information Theory and Interpersonal Physiological Synchrony · 中文
Chapter 2 The Information Theory of Prosody: Uniform Information Density and the Power of the Pause
Surprisal
Shannon’s information theory gives us a way to measure how much new information a word brings: the more unexpected a word is in its context, the more information it carries. The quantity is called surprisal:
measured in bits. After “our cost per customer fell by —”, “ninety” is more unexpected than “ten” and so carries more information.
Uniform information density
Levy and Jaeger proposed the uniform information density hypothesis: speakers tend to spread information fairly evenly in time, avoiding places where it is so dense that it exceeds the listener’s capacity and places where it is so sparse that time is wasted (Levy and Jaeger 2007; Jaeger 2010). One way to do this is to slow down, pause or use a fuller syntactic form where information is high, and to speed up and reduce where it is low.
Evidence
Where pauses occur. As early as 1958, Goldman-Eisler found that pauses in natural speech tend to come before less predictable words (Goldman-Eisler 1958). Different languages, similar information rates. Coupé and colleagues compared read speech in 17 languages and found that the information carried per syllable differs greatly between languages, but that languages with dense syllables are spoken more slowly and languages with sparse syllables faster, so that information rates cluster around 39 bits per second (Coupé et al. 2019). Human speech may operate near a roughly constant information rate.
The information flow of one sentence of a pitch
Calculation
One sentence of a pitch: “In eighteen months our cost per customer fell by ninety percent.” The probability of each word in context is an illustrative listener model, not a trained language model. Three deliveries are compared: an even 4.5 syllables per second; a rushed 6.5 syllables per second; and the even rate with a 0.6-second pause before the key word “ninety” and the key word lengthened by 30 %.
# Surprisal and information rate along one sentence of a pitch. Surprisal of a word is
# -log2 P(word | preceding words). The probabilities below are an ILLUSTRATIVE listener model (how
# expected each word is at that point), not output of a trained language model.
# Uniform Information Density (Levy & Jaeger 2007; Jaeger 2010): speakers tend to spread information
# evenly; lengthening or pausing before improbable words is one phonetic way to do so (an extension).
import math
WORDS = [("in", 0.20, 1), ("eighteen", 0.02, 2), ("months", 0.60, 1), ("our", 0.30, 1), ("cost", 0.10, 1),
("per", 0.50, 1), ("customer", 0.40, 3), ("fell", 0.15, 1), ("by", 0.60, 1),
("ninety", 0.004, 2), ("percent", 0.80, 2)] # (word, probability in context, syllables)
KEY = "ninety"
def timeline(rate_syl_s, pause_before_key=0.0, key_stretch=1.0):
t, rows = 0.0, []
for w, p, syl in WORDS:
if w == KEY: t += pause_before_key
dur = syl / rate_syl_s * (key_stretch if w == KEY else 1.0)
rows.append((w, -math.log2(p), t, dur)); t += dur
return rows, t
total_bits = sum(-math.log2(p) for _, p, _ in WORDS)
print(f"total surprisal of the sentence: {total_bits:.1f} bits; key word '{KEY}': {-math.log2(0.004):.1f} bits")
for label, args in (("steady, 4.5 syllables/s", (4.5,)), ("rushed, 6.5 syllables/s", (6.5,)),
("paced: 4.5 syl/s, 0.6 s pause before key word, key word 30 % longer", (4.5, 0.6, 1.3))):
rows, T = timeline(*args)
peak = max(bits / dur for _, bits, _, dur in rows)
key = [bits / dur for w, bits, _, dur in rows if w == KEY][0]
print(f"{label}:\n {T:.2f} s, mean {total_bits / T:.1f} bits/s, peak {peak:.1f} bits/s (the key word: {key:.1f} bits/s)")
# total surprisal of the sentence: 27.8 bits; key word 'ninety': 8.0 bits
# steady, 4.5 syllables/s:
# 3.56 s, mean 7.8 bits/s, peak 17.9 bits/s (the key word: 17.9 bits/s)
# rushed, 6.5 syllables/s:
# 2.46 s, mean 11.3 bits/s, peak 25.9 bits/s (the key word: 25.9 bits/s)
# paced: 4.5 syl/s, 0.6 s pause before key word, key word 30 % longer:
# 4.29 s, mean 6.5 bits/s, peak 14.9 bits/s (the key word: 13.8 bits/s)
The whole sentence carries about 28 bits of surprisal, of which the key word “ninety” alone carries 8. Spoken evenly, the instantaneous information rate at the key word is about 18 bits per second; rushed, it rises to 26; with the pause and lengthening it falls to 14.
A listener of limited capacity
A high information rate is not a problem in itself; the problem is whether the listener can keep up. The model below treats the listener as a system with a processing limit: when information arrives faster than it can be processed, it piles up in a “queue”; if the queue is long when a word ends, that word is only partly encoded.
Calculation
The listener processes at most 10 bits per second, and once the backlog exceeds 6 bits almost nothing new is encoded (both numbers illustrative). Compare how fully the key word is encoded under four deliveries: even, rushed, nervous (rushed, plus two 0.3-second “uh”s, the second right before the key word), and pause plus lengthening.
# A limited-capacity listener (a model, not a measured law). Information arrives word by word (the
# surprisal of the previous program, spread evenly over each word's duration). The listener
# integrates it at most C bits per second; anything faster waits in a backlog. A word is encoded
# in proportion to how empty the backlog is when it ends: encoded = max(0, 1 - backlog / B).
# C = 10 bits/s and B = 6 bits are illustrative choices.
import math
# The sentence and delivery timings of the previous program, repeated so this one runs on its own.
WORDS = [("in", 0.20, 1), ("eighteen", 0.02, 2), ("months", 0.60, 1), ("our", 0.30, 1), ("cost", 0.10, 1),
("per", 0.50, 1), ("customer", 0.40, 3), ("fell", 0.15, 1), ("by", 0.60, 1),
("ninety", 0.004, 2), ("percent", 0.80, 2)] # (word, probability in context, syllables)
KEY = "ninety"
def timeline(rate_syl_s, pause_before_key=0.0, key_stretch=1.0):
t, rows = 0.0, []
for w, p, syl in WORDS:
if w == KEY: t += pause_before_key
dur = syl / rate_syl_s * (key_stretch if w == KEY else 1.0)
rows.append((w, -math.log2(p), t, dur)); t += dur
return rows, t
C, B, DT = 10.0, 6.0, 0.001
def listen(words_timed, fillers=()):
"""words_timed: (word, bits, start, duration). fillers: list of (start, duration), 0 bits."""
events = sorted([(s, d, b, w) for w, b, s, d in words_timed] + [(s, d, 0.0, "uh") for s, d in fillers])
backlog, t, out = 0.0, 0.0, {}
for start, dur, bits, w in events:
while t < start: # silence: the backlog drains
backlog = max(0.0, backlog - C * DT); t += DT
steps = max(1, int(dur / DT))
for _ in range(steps):
backlog = max(0.0, backlog + bits / steps - C * DT); t += DT
if w != "uh": out[w] = max(0.0, 1 - backlog / B)
return out
def shift(rows, after_word, by):
out, moved = [], False
for w, b, s, d in rows:
out.append((w, b, s + (by if moved else 0.0), d))
if w == after_word: moved = True
return out
steady, _ = timeline(4.5)
rushed, _ = timeline(6.5)
paced, _ = timeline(4.5, 0.6, 1.3)
# a nervous delivery: rushed, plus two 0.3 s fillers ('uh'), the second just before the key word
nervous = shift(shift(rushed, "our", 0.3), "by", 0.3)
fill = [(dict((w, s) for w, _, s, _ in nervous)["our"] - 0.3, 0.3), (dict((w, s) for w, _, s, _ in nervous)["ninety"] - 0.3, 0.3)]
print("delivery | key word encoded | mean over all words")
for label, rows, f in (("steady 4.5 syl/s", steady, ()), ("rushed 6.5 syl/s", rushed, ()),
("nervous: rushed + 2 fillers", nervous, fill), ("paced: pause 0.6 s + key word longer", paced, ())):
enc = listen(rows, f)
print(f"{label:37s} | {enc[KEY]:16.0%} | {sum(enc.values()) / len(enc):18.0%}")
# delivery | key word encoded | mean over all words
# steady 4.5 syl/s | 41% | 90%
# rushed 6.5 syl/s | 0% | 53%
# nervous: rushed + 2 fillers | 18% | 75%
# paced: pause 0.6 s + key word longer | 63% | 92%In this model:
- Spoken evenly, the key word is about four-tenths encoded, and the sentence as a whole about nine-tenths.
- Rushed, the key word is hardly encoded at all, and the sentence on average only about half: information arrives too fast and the backlog keeps growing.
- Pause plus lengthening raises the encoding of the key word to about six-tenths, the highest of the four. The pause clears the backlog before the key word arrives, and the lengthening spreads its information over more time.
- A noteworthy result: the nervous delivery (rushed plus two “uh”s) is better than rushing alone, raising the key word’s encoding from almost nothing to about two-tenths. The reason is simple: “uh” itself carries almost no information, and the 0.3 seconds it takes give the listener time to clear the backlog, much as a pause does.
The model supports the advice to pause and slow down before key information, but it also qualifies the popular claim that fillers cause “channel overload”.
Evidence
Fillers are not simply harmful. Corley and colleagues recorded listeners’ EEG and found that words following “er” elicited a reduced N400 effect (an EEG component linked to how expected a word is), and were better remembered in a later memory test (Corley et al. 2007). Fraundorf and Watson found that plot points in a story preceded by a filled pause (“uh”) were remembered better than those with no pause or interrupted by a cough (Fraundorf and Watson 2011). On the other hand, Brennan and Williams found that fillers and pauses before an answer led listeners to judge the speaker less sure of the answer (Brennan and Williams 1995).
So fillers work two ways: like a pause, they give the listener time, improving encoding and memory of the next word; they may also signal that the speaker is unsure, lowering listeners’ judgements of the speaker’s confidence. A silent pause might keep the first benefit and avoid the second cost, but this still needs a direct experimental comparison.
Note
What the models can show. The probabilities, processing limit and backlog threshold in these two programs are illustrative and unmeasured. They show how a mechanism could logically work: creating time before an information peak can help the listener take in the information at the peak. They cannot prove that this is how listeners actually process speech, nor give the best length for a pause. Coupé and colleagues’ 39 bits per second is a language-level average information rate at the syllable level, and cannot be compared directly with the word surprisals in this model.
Hypothesis · not established
The silent-pause-beats-filler hypothesis (not yet established). Inserting a silent pause or a filler “uh” of the same length before key information helps listeners remember it about equally, but the silent pause leads listeners to rate the speaker as surer and more credible. If experiments show that fillers help memory clearly more than silent pauses, or that the two do not differ in credibility ratings, the hypothesis is weakened.
Summary
Measured by surprisal, the information in a sentence is often concentrated in a few unexpected words. The uniform information density hypothesis holds that speakers tend to spread information out in time, and pauses in natural speech tend to come before less predictable words. A limited-capacity listener model shows that rushing can leave key information almost unencoded, and that pausing before and lengthening the key word improves this markedly; fillers act somewhat like pauses, but may also lower listeners’ judgement of the speaker’s confidence. The next chapter turns to another prosodic feature: where the pitch goes at the end of a sentence.
References
Fan, Y. (2026). Acoustic Persuasion, Prosodic Information Theory and Interpersonal Physiological Synchrony: Pitch Contours, Pause Entropy and Listener Autonomic Coupling in Public Speaking and Leadership [声音说服力、韵律信息论与人际生理同步] (Version 1.0.0). 5D Library. https://doi.org/10.5281/zenodo.23753267 · Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0).