What the book covers

1. Weber’s law and Fechner’s law: the just noticeable difference and internal noise, the correction to Weber’s law at low intensities, the step from Weber to Fechner and the dispute over it, and where three logarithmic units, the decibel, the stellar magnitude and the Richter magnitude, actually come from.
2. Stevens’s power law: magnitude estimation, the exponents of different senses and the conditions under which they were measured, cross-modality matching, and the criticisms of the power law and of magnitude estimation.
3. Optimal multisensory integration: maximum-likelihood combination with weights set by precision, visual–haptic judgements of height, the ventriloquist effect, and the cases in children and in some adult experiments where optimal integration fails.
4. Causal inference and the McGurk effect: a hierarchical Bayesian model of whether two signals have one source or two, the transition from fusion to segregation, and individual differences in the McGurk effect.
5. Multisensory neurons in the superior colliculus: superadditivity, inverse effectiveness and the normalisation model, and the statistical criticisms of inverse effectiveness.
6. Sensory substitution and cross-modal plasticity: tactile and auditory substitutes for vision, evidence that the visual cortex of blind people is recruited by other senses, and the part played by active movement in learning.

Chapter 7 reads these modern studies beside passages on the senses in the Liezi, the Zhuangzi and the Śūraṅgama Sūtra, and beside Qian Zhongshu’s study of “synaesthesia” (tonggan) in classical Chinese poetry, and marks the boundary between them. Chapter 8 offers a four-step multisensory precision checklist and a pre-registered randomised controlled trial of vibrotactile feedback for training vocal pitch and formants, with pitch-imitation error without feedback 24 hours later as its single primary outcome.

The book overlaps with several others in the series. Volume 26, Probabilistic Judgement, Brier-Score Calibration and the Statistics of Collective Intelligence, discusses signal detection theory in its Chapter 4; volume 10, The Biomechanics of Phonation and Attentional Shift, and volume 29, Vocal Biomechanics, Non-Linear Source-Filter Coupling and Auditory-Motor Feedback, discuss auditory feedback and the fusion of hearing and body sense under masking; volume 2, Turning the Hearing Inward, discusses the passage on perfect penetration through the ear faculty in fascicle 6 of the Śūraṅgama Sūtra. Where this book meets those topics it recalls them briefly, with a reference, and does not repeat their programs.

Claims corrected on checking

Several points in the author’s framework needed correction or qualification once they were checked. Each chapter says so where it arises, and the Editor’s Notes published with the book collect them:

• The “psychophysical origin” of logarithmic units. The three units have different histories and no common psychophysical origin. The magnitude classes of stars go back to ancient astronomers’ visual grading of brightness, and in 1856 Pogson fixed the brightness ratio between adjacent magnitudes at the fifth root of 100 (about 2.512), before Fechner’s book of 1860. The decibel is a telephone-engineering unit of power ratio, chosen because logarithms turn the multiplying losses of successive line sections into sums. The Richter magnitude (1935) is the logarithm of the amplitude recorded by a standard seismometer, taken to compress an enormous range, and has nothing to do with human sensation. Nor is the decibel a measure of loudness: loudness roughly doubles for every 10 dB.
• From Weber to Fechner is not a “direct integration”. Getting from $\Delta I/I = k$ to $S = k\ln(I/I_0)$ needs an extra assumption: that every just noticeable difference is subjectively the same size. Miller (1947) reported a JND for the intensity of white noise of about 0.41 dB above about 30 dB sensation level, and noted at the same time that equal numbers of JNDs do not correspond to equal loudness. Luce and Edwards (1958) criticised Fechner’s derivation; Dzhafarov and Colonius (2011) pointed out that Fechner had a second derivation that does not use JNDs: the logarithmic law follows if the subjective dissimilarity of two stimuli depends only on the ratio of their intensities, a principle logically independent of Weber’s law; the familiar derivation from Weber’s law plus equal JNDs holds only approximately, the better the smaller the Weber fraction. This book treats the step as an assumption to be tested, not as a conclusion of the derivation.
• The form of the low-intensity correction. $\Delta I/(I + I_a) = k$ is a common approximation. If the internal additive and multiplicative noises are independent (their variances add), the result is $\Delta I = k\sqrt{I^2 + I_a^2}$; the two forms differ only near $I \approx I_a$, by at most about 40 % (Chapter 1 program). The contrast between the dropping pin and the shout in the street is mainly masking by background noise, which is related to Weber’s law but is not the same experiment.
• The conditions behind Stevens’s exponents. Brightness 0.33 refers to a 5° target viewed in the dark (a point source or a brief flash gives about 0.5); visual length 1.0 to a projected line; electric shock 3.5 to current through the fingers; loudness 0.67 is relative to sound pressure, or about 0.33 relative to sound intensity. The exponents also change with the method (estimation or production) and the stimulus range.
• The size and replication of optimal-integration experiments. Ernst and Banks (2002) had four observers, and the visual–haptic conflicts never exceeded 11 %; Alais and Burr (2004) had six, and their average gain in precision (1.425 times) was close to the prediction (1.414), but only two individual observers reached statistical significance. A 2019 study of audiovisual spatial localisation concluded that integration does not strictly follow maximum-likelihood weights. The formulas in the framework are correct.
• The causal-inference model and the McGurk effect are two different paradigms. The 2007 model of Körding and colleagues was fitted to the spatial localisation of flashes and beeps, not to speech; causal inference was applied to the McGurk effect in later work (Magnotti and Beauchamp, 2017). McGurk and MacDonald (1976) reported that 98 % of adults heard the fused syllable; later studies found fusion rates ranging from 17 % to 58 % across stimuli, and from never to always across people.
• How strong the evidence for superadditivity and inverse effectiveness is. Superadditivity in single collicular neurons appears mainly when the single-sense stimuli are weak; additive or subadditive responses are more common. Part of inverse effectiveness may come from regression to the mean, floor and ceiling effects, and the choice of units. At the level of behaviour, seeing a speaker’s mouth helps most in understanding speech in noise at an intermediate signal-to-noise ratio (about −12 dB), not at the worst one.
• Sensory-substitution devices and the “visual cortex”. The 1969 device of Bach-y-Rita and colleagues was an array of 400 vibrators (20 × 20) in the back of a chair, stimulating the skin of the back; electrotactile arrays on the tongue came later (a 1998 technical note describes 49 electrodes). Amedi and colleagues (2007) reported activation of a shape-processing region of the lateral occipital cortex by sound-coded shapes, not of primary visual cortex; the evidence that primary visual cortex is activated when blind people read Braille comes from Sadato and colleagues (1996). Most studies are small, and claims that users “really see” must be qualified.
• Titles and wording of the classics. Kangcangzi’s “my body is at one with my mind, my mind with my qi, my qi with my spirit, my spirit with nothingness” is in the “Zhongni” chapter of the Liezi. “The eye became like the ear, the ear like the nose, the nose like the mouth; there was no difference … my mind concentrated, my body dissolved, bone and flesh all melted”, in the wording the framework quotes, is in the “Huangdi” chapter; the “Zhongni” chapter has an almost identical passage. Note especially that Kangcangzi himself says “Those who spread this are wrong. I can see and hear without using ears and eyes, but I cannot exchange the functions of ears and eyes”, denying that he “sees with his ears and hears with his eyes”. The passage on the fasting of the mind in the Zhuangzi’s “In the World of Men” is quoted from the received text (“hearing stops at the ear”) and translated as it stands. “Thus the six faculties function interchangeably” is in fascicle 4 of the Śūraṅgama Sūtra, not in the chapter on perfect penetration through the ear faculty in fascicle 6. The line “on the red apricot branch, spring is noisy” (Song Qi, “Yu lou chun”) and Li Yu’s objection to it, as cited in Qian Zhongshu’s “Synaesthesia”, check out.
• Which volume is “volume 2”. The author asked that the book leave out circadian rhythms because volume 2 had covered them. Checking the catalogue: volume 2 is Turning the Hearing Inward: A Bioacoustic Study of the Ergen Yuantong Listening Method; light and circadian rhythms are covered in volume 14, Dynamical Systems of Neurotransmitters and Endocrinology. This book does not touch circadian rhythms; volume 2 is relevant to the Śūraṅgama Sūtra in Chapter 7 and is cross-referenced there.
• An audiovisual synchrony window of “about ±100 ms”. The window is asymmetric: sound arriving before the picture is noticed more easily. The International Telecommunication Union’s broadcasting recommendation gives detectability thresholds of about 45 ms with sound early and 125 ms with sound late; fusion of speech is common from about 30 ms early to about 170 ms late; and the window varies with the stimulus and the person and can be narrowed by training.

How to read the boxes

As in other 5D Library books, different kinds of statement are marked separately.

About this edition

This is version 1.0.0, the first complete edition: a preface and eight chapters, in Chinese and English. The book contains 17 Python programs and 4 vector figures. The text, boxes and code were drafted with an AI writing tool from the author’s title, framework and notes; the author is responsible for reviewing and revising them and for checking every source. By the standards set out in volume 33, readers should know that AI-drafted text can get bibliographic details wrong, that the author has a direct commercial interest in the subject (see below), and that the trial in the book is so far only a protocol. Corrections and missing references are welcome through the 5D Library website.