关于「比特账与词汇空缺度」,《嵌词阈值与单语等效》给出的定量结论是什么?

结论:修正后的比特账把码长与剩余歧义加在一起。在示意参数下,六个空缺词里有两个嵌入后码长更长,可总账上五个更省,只有一个更贵;省下的比特从 +1.33 到 +7.22 不等。空缺度可以用最近邻词的特征失配率来量,但空缺深浅与嵌词收益并不同步,还要看最近邻词周围有多少相似概念。语言各自在简单与精确之间取舍,空缺因此而生;双语者借用另一种语言的分类边界,就能把意思切得更细。

数据口径:嵌词收益只在读者共享双语词库时成立:把单语表达、嵌入与嵌入加注释写成覆盖率的三条成本线,示意参数下 kiasu 在书面渠道的覆盖率阈值为 0.844;读者不发问时,失败会变成沉默的误读;各地区分页面值得嵌的词,合成混合页面后可能全部跌破阈值。 书中参数为示意,全部计算由书中程序生成,不对应任何真实客户、对话或成交;马来语、闽南语、粤语、英语、华语一律平等对待。

English version

对照数据表

下表数字由本章 Python 程序原样打印,可重新运行复现。 程序只用 Python 标准库与结论吻合的打印值: 1.33, 7.22

itemLmLeHmHeTmTeLe/LmTe/Tm
kiasu19.9914.972.60.422.5915.370.7490.680
lepak14.0814.552.10.316.1814.851.0330.918
paiseh14.9513.551.80.516.7514.050.9060.839
shiok10.4415.141.50.611.9415.741.4501.318
sajiao22.4117.292.40.724.8117.990.7720.725
yuanfen18.0816.292.90.920.9817.190.9010.819
bits saved for a listener who knows the word (Tm - Te):
kiasu +7.22lepak +1.33paiseh +2.70
shiok -3.80sajiao +6.82yuanfen +3.80
查看完整验算程序
# Chapter 3. p31: code length plus residual ambiguity, six lexical gaps.
# Lm: bits of a monolingual circumlocution under the listener's bigram
# model (product of the listed token probabilities). Le: bits of the
# inserted word, -log2 P(word | shared bilingual lexicon), plus 2 bits of
# switch cue (in and out). Hm, He: residual ambiguity, the expected bits
# still needed to pin the intended sense after each rendering. Every
# probability is an illustrative parameter, not a corpus estimate.
import math

ITEMS = [
    # name, circumlocution token probabilities, P(word), Hm, He
    ("kiasu", [0.05, 0.10, 0.04, 0.08, 0.06], 1 / 8000, 2.6, 0.4),
    ("lepak", [0.08, 0.12, 0.06, 0.10], 1 / 6000, 2.1, 0.3),
    ("paiseh", [0.10, 0.07, 0.09, 0.05], 1 / 3000, 1.8, 0.5),
    ("shiok", [0.12, 0.06, 0.10], 1 / 9000, 1.5, 0.6),
    ("sajiao", [0.03, 0.05, 0.04, 0.06, 0.05], 1 / 40000, 2.4, 0.7),
    ("yuanfen", [0.05, 0.04, 0.03, 0.06], 1 / 20000, 2.9, 0.9),
]
CUE = 2.0


def lengths(item):
    name, toks, pw, hm, he = item
    lm = math.fsum(-math.log2(p) for p in toks)
    le = -math.log2(pw) + CUE
    return lm, le, hm, he


if __name__ == "__main__":
    print("item      Lm     Le    Hm   He     Tm     Te   Le/Lm  Te/Tm")
    for it in ITEMS:
        lm, le, hm, he = lengths(it)
        tm, te = lm + hm, le + he
        print("{:<7} {:>6.2f} {:>6.2f} {:>4.1f} {:>4.1f} {:>6.2f} {:>6.2f}"
              " {:>6.3f} {:>6.3f}".format(it[0], lm, le, hm, he, tm, te,
                                          le / lm, te / tm))
    gains = []
    for it in ITEMS:
        lm, le, hm, he = lengths(it)
        gains.append((lm + hm) - (le + he))
    print("bits saved for a listener who knows the word (Tm - Te):")
    print("  " + "  ".join("{} {:+.2f}".format(it[0], g)
                           for it, g in zip(ITEMS[:3], gains[:3])))
    print("  " + "  ".join("{} {:+.2f}".format(it[0], g)
                           for it, g in zip(ITEMS[3:], gains[3:])))

唯一数据源与引用

范雅福(Fan, Y.). (2026). 《多语参杂语码转换信息论、词汇空缺补偿与单语种等效精准表达母典》第 3 章「比特账与词汇空缺度」(版本 1.0.0). 5D Library. DOI 待发布

DOI 待发布 · 专著页正式发布后上线(本站暂无专著页) · 5DL-B-381 · v1.0.0 · CC-BY-NC-ND-4.0

BibTeX
@book{fan2026381,
  author    = {Fan, Yafu},
  title     = {Information Theory of Code-Switching, Lexical-Gap Compensation and Monolingual Precision Equivalence in Multilingual Markets},
  year      = {2026},
  publisher = {5D Library},
  edition   = {1.0.0},
  note      = {DOI pending}
}

本书其他章节结论