本文从数学角度完整拆解模型量化（Quantization）：它如何把 FP32 浮点权重/激活映射成低比特整数，S 和 Z 怎么求，量化误差从何而来、如何严格证明它的界，信噪比怎么推，以及为什么 8-bit 甚至 4-bit 量化之后，模型精度损失通常并不严重。 内容框架与 MIT 6.5940 / TinyML（韩松 Song Han）课程的 Quantization 一讲一致。
收录于微信公众号：AngryCode 愤怒的代码
0. TL;DR
- 量化的本质是一条仿射变换：r = S(q − Z)。r 是 float，q 和 Z 都是整数，S 是 float——全公式只有一次浮点乘法。
- S = 数据范围 ÷ 整数范围。数据范围有三条求法：min-max、裁剪+MSE、裁剪+KL 散度。
- 误差来自四舍五入，可严格证明 |ε| ≤ S/2、零均值、方差 S²/12（均匀分布假设下积分可得）。
- 每加 1 bit，量化信噪比提升 6 dB（SQNR = 6.02n dB）；int8 的噪声只有满量程的 ~0.4%。
- 误差在求和/传播时正负抵消（标准差按 √N 增长），且激活函数 Lipschitz 连续（ReLU 的 L=1）使误差不被放大，因此整网几乎不掉点。
- 矩阵乘法可以拆成纯整数 GEMM + 一次尺度相乘，这是量化省算力、省带宽、省功耗的数学来源。
1. 为什么要量化
端侧/边缘设备（手机、MCU、NPU）受四个硬约束：
- 内存：FP32 一个权重占 4 字节，int8 只占 1 字节 → 模型体积缩小 4 倍。
- 带宽：内存搬运是端侧推理的主要能耗来源，位宽降低直接成比例省电。
- 算力：整数 MAC 单元面积、功耗、时延都远低于 FP32 MAC。
- 实时性：整数运算 + 更小的内存占用 → 更低延迟。
一句话：量化不是为了"更好"，而是为了"能跑"。 而它之所以可行，是因为量化误差在数学上可控——这就是本文要讲清楚的。
2. 出发点：FP32 到底存了什么
IEEE 754 单精度浮点数 = 1 符号位 + 8 指数位 + 23 尾数位：
x = (−1)^s × 2^(e−127) × (1 + f / 2^23)
- 数值范围：约 ±3.4 × 10³⁸
- 相对精度：约 2⁻²³ ≈ 1.2 × 10⁻⁷（约 7 位有效数字）
关键认知：神经网络权重是训练出来的统计量，不是物理常数。 它不需要 7 位有效数字的精度。这为量化提供了巨大的冗余空间。
3. 量化的数学定义：仿射映射
3.1 核心公式
量化的核心就是把实数 r 映射到整数 q：
r = S · (q − Z) （反量化：整数 → 实数）
q = clamp( round(r / S) + Z , q_min, q_max ) （量化：实数 → 整数）
clamp 是什么：限幅函数——把值夹进 [min, max] 区间，太小压到 min、太大压到 max、区间内原样返回：
clamp(x, min, max) = min 若 x < min
= x 若 min ≤ x ≤ max
= max 若 x > max
在量化里它的作用：round(r/S) + Z 可能超出整数合法范围——比如 r 是校准时没见过的 outlier，算出的整数超出 [q_min, q_max]（如 int8 算出 300 或 −254）。clamp 保证结果一定落得进 n 比特（int8 就是 [−128, 127]），不会溢出回绕。代价是被夹掉的值产生裁剪误差（无界，见 5.1 节）——这正是校准（KL/MSE 选阈值）要优化的那个误差的来源。clamp 在不同框架里也叫 clip（NumPy/TensorFlow）、饱和运算 saturate（硬件）。
3.2 四个量各自的类型（一个常见误解）
Z 不是 float，是整数（和 q 同一个整型）：
所以 r = S(q − Z) 的运算过程是：
(q − Z) → 整数减法
× S → 乘一个 float，得到 float
整个公式里只有一个 float 乘法，其余都是整数运算——这正是量化能提速的关键。
为什么 Z 一定是整数？ 看 Z 的求法：
Z = round( q_min − r_min / S )
q_min 是整数，r_min/S 是 float，相减后 round 成整数，所以 Z 天生是整数，且落在 [q_min, q_max] 内，能用和 q 一样的 n 比特存下来。如果 Z 是 float，硬件就得做浮点减法、多存一个 float，量化就失去意义了。
一个具体例子：数据范围 r ∈ [−0.5, 1.0]，uint8 量化（q ∈ [0, 255]）：
S = (1.0 − (−0.5)) / 255 = 1.5/255 ≈ 0.005882
Z = round(0 − (−0.5)/0.005882) = round(85) = 85
验证 r = 0：S(q − Z) = 0 ⟹ q = Z = 85。实数 0 对应的整数就是 85——Z 是整数。
3.3 为什么叫仿射变换
因为 r = S·q − S·Z 是一条直线：斜率 S，截距 −S·Z。它允许"缩放 + 平移"，所以既能表示对称分布（Z=0），也能表示非对称分布（Z≠0，如 ReLU 输出的非负激活）。本文的一切结论都从这条直线推出。
4. 怎么求 S 和 Z
S 的计算核心一句话：S = 数据范围 ÷ 整数范围。整数范围由位宽定死，真正要决定的是"数据范围"怎么取。
n 比特整数的表示范围：
4.1 方法一：min-max 非对称（全覆盖，无裁剪）
S = (r_max − r_min) / (q_max − q_min)
Z = round( q_min − r_min / S )
验证端点对齐：r_min = S·(q_min − Z)，r_max = S·(q_max − Z)。
例子：某层 ReLU 激活，范围 [0, 6.0]，用 uint8：
S = (6.0 − 0) / (255 − 0) = 6/255 ≈ 0.023529
- 优点：一个值都不裁剪，全盖住。
- 缺点：对 outlier 极其敏感。如果 99.9% 的值落在 [0, 0.5]、只有一个 6.0，S 就被这个 outlier 撑大，绝大多数值的量化误差被无谓放大。
4.2 方法二：min-max 对称（Z=0）——为什么必须加绝对值
对称量化强制 Z = 0，映射退化为 r = S·q：
α = max(|r_min|, |r_max|)
S = α / q_max （有符号时 q_max = 2^(n−1) − 1）
这里 abs 不是可选项，是定义本身。 原因：对称量化要求映射关于 0 对称——整数能取 −127 和 +127，对应的实数就是 −S·127 和 +S·127，所以实数区间必须是 [−α, α]，而 α 必须 ≥ 所有数据里"离 0 最远"的值，即最大绝对值。
反例：不加绝对值会怎样。设张量 r_min = −2.0，r_max = 1.0：
- ❌ 不加 abs：max(r_min, r_max) = 1.0，S = 1.0/127。那么 −2.0 对应整数 = −2.0/(1/127) = −254，超出 int8 范围 [−127, 127]，被硬夹到 −127，所有小于 −1.0 的负值全部失真。
- ✅ 加 abs：α = max(|−2|, |1|) = 2.0，S = 2.0/127。区间 [−2, 2] → [−127, 127]，完美覆盖。
数据范围（不对称）： [----r_min=-2.0 ····· 0 ····· r_max=1.0----]
对称量化强制摊开成： [----−α=-2.0 ······· 0 ······· +α=+2.0----]
↑
右边 [1.0, 2.0] 这段是"浪费"的
对称量化为了保持 Z=0 的简洁，代价就是两边必须对称摊开，数据偏一边时另一半范围就浪费了——这也是它和非对称量化的核心区别。
实现细节：有符号 int8 的 q_max 用 127 而不是 128，为了 ±q_max 对称（−128 被闲置）——这是"强制对称"付出的微小代价，换来的 GEMM 大幅简化（见第 7 节）。
4.3 方法三：裁剪 + 校准（针对 outlier）
min-max 的问题在 outlier。更聪明的做法：故意裁掉少量极端值，换取更小的步长。
设裁剪阈值 T（0 < T ≤ max|r|），只量化 [−T, T] 内的值，超过 T 的直接夹到端点：
S = T / q_max （对称）
T 的选取是核心，两条准则：
① MSE 准则：选 T 使总误差的均方最小：
T* = argmin_T E[ (Q(r) − r)² ]
= argmin_T [ 裁剪误差(超出T的部分) + 量化误差(S²/12) ]
T 太小 → 裁剪误差大；T 太大 → S 大、量化误差大。两者此消彼长，最优 T 平衡它们。
② KL 散度准则（NVIDIA TensorRT 的做法）：把 FP32 值和量化后的值各做成直方图，选 T 使两个分布的信息损失最小：
T* = argmin_T KL( P_fp32 ‖ P_quant(T) )
直观理解：选一个 T，让"量化前的数值分布"和"量化后的数值分布"长得最像。 对高斯/拉普拉斯分布的权重，最优 T 的经验值约在 ±(2.5 ~ 4)σ——不是 min-max 的全范围，而是收敛在几倍标准差处。
4.4 数据从哪来：权重 vs 激活（一个关键区别）
所以 PTQ 流程里常看到"用 100~1000 张校准图片跑前向、统计每层激活的分布"——那就是在给激活算 S（和 Z）。
4.5 方法小结
5. 量化误差的数学分析
量化后的值记为 Q(r) = S · round(r/S + Z) − S·Z，误差：
ε = Q(r) − r
5.1 性质 1：误差有界——|ε| ≤ S/2 的完整证明
这个上界只在"值落在量化范围内、未被裁剪"时成立。证明只需三步，核心是把误差拆成"步长 S × 四舍五入误差"。
第一步：定义中间变量，把 r 用 t 表示。
令 t = r/S + Z，则反解出 r = S·(t − Z)。t 的含义是"r 在整数刻度上的位置"（还没取整）。量化就是对这个位置四舍五入：
q = round(t) = round(r/S + Z)
第二步：把量化误差改写。
ε = Q(r) − r
= S·(round(t) − Z) − S·(t − Z) ← 代回 Q(r) 和 r
= S·( round(t) − t ) ← S·Z 和 −S·Z 抵消
令 δ = round(t) − t（以刻度为单位的四舍五入误差，无量纲），得到：
ε = S · δ
第三步：证明 |δ| ≤ 1/2。
任取实数 t，它一定落在某个整数区间 [n, n+1] 内（n = ⌊t⌋）。四舍五入到最近的整数，结果要么是 n、要么是 n+1，距离分别是 t−n 和 (n+1)−t。而这两段距离之和：
(t − n) + ((n+1) − t) = 1
两段加起来才 1，所以较短的那段一定 ≤ 1/2。四舍五入恰好选的是最近整数，因此：
|δ| = |round(t) − t| = min(t−n, (n+1)−t) ≤ 1/2
代入第二步：
|ε| = S·|δ| ≤ S/2 ∎
直观理解：四舍五入的本质是"往最近的刻度靠"，而刻度间距是 1（在 t 的坐标系里），所以最多走半个刻度。换算回实数坐标系，每个刻度代表 S，所以最多错 S/2。
数值例子：S = 0.1, Z = 0, r = 0.53：
t = 0.53/0.1 = 5.3 → round = 5 → δ = 5 − 5.3 = −0.3
ε = S·δ = 0.1 × (−0.3) = −0.03
|ε| = 0.03 ≤ 0.05 = S/2 ✓
⚠️ 一个必须记住的例外：这个上界只对"范围内的值"成立。若 r 超出 [r_min, r_max]，它会被 clamp 硬夹到端点，此时误差可以任意大——裁剪误差无界：
总误差 = 量化误差(≤ S/2) + 裁剪误差(无界)
这正是 4.3 节"KL 散度选阈值"要解决的问题：范围太窄 → 裁剪误差大；太宽 → S 大 → 量化误差大。
5.2 性质 2：误差零均值、方差 S²/12——完整推导
方差 S²/12 是在一个前提假设下算出来的：量化误差 ε 在 [−S/2, S/2] 上均匀分布。
前提：为什么 ε 是均匀的。 设 t = r/S + Z，分数部分 frac = t − ⌊t⌋ ∈ [0, 1)。四舍五入后：
frac ∈ [0, 0.5) → round(t) = ⌊t⌋ → δ = round(t)−t = −frac ∈ (−0.5, 0]
frac ∈ [0.5, 1) → round(t) = ⌊t⌋+1 → δ = 1 − frac ∈ (0, 0.5]
关键：如果 frac 在 [0,1) 上均匀分布（输入数据足够"乱"、步长足够小时这是很好的近似），那么 δ 就在 (−0.5, 0.5] 上均匀分布。再乘上常数 S，得到：
ε = S·δ 在 [−S/2, S/2] 上均匀分布
这就是"误差均匀模型"（Bennett 均匀量化模型）。
期望（零均值）：均匀分布的 PDF 是 f(ε) = 1/S（区间长度正好是 S，总概率 1）：
E[ε] = ∫ ε·f(ε) dε = (1/S) ∫_{−S/2}^{S/2} ε dε = 0
因为 ε 是奇函数，在关于 0 对称的区间上积分必为 0。
方差（逐步积分）：Var(ε) = E[ε²] − (E[ε])²，而 E[ε]=0，所以：
Var(ε) = E[ε²] = (1/S) ∫_{−S/2}^{S/2} ε² dε
= (1/S) · [ ε³/3 ]_{−S/2}^{S/2}
= (1/S) · [ (S/2)³/3 − (−S/2)³/3 ]
= (1/S) · [ S³/24 − (−S³/24) ]
= (1/S) · ( S³/12 )
= S²/12
标准差：
σ_ε = S / √12 ≈ 0.289·S
几何直觉：均匀分布在 [−S/2, S/2] 上的随机变量，RMS 值是 S/√12 ≈ 0.289S——比最大值 S/2 = 0.5S 小不少，因为大部分误差落在 0 附近，落在 ±S/2 极端处的是少数。
数字例子：int8、范围 [−1,1]：S = 1/127 ≈ 0.00787。
σ_ε = 0.00787/√12 ≈ 0.00227 ← 误差的"典型大小"
max = S/2 ≈ 0.00394 ← 误差的"最坏情况"
典型误差只有满量程的 0.23%，最坏也才 0.39%——这就是 int8 精度够用的直观来源。
⚠️ 假设什么时候会失效：
- 数据近乎常数（某层激活值都差不多）→ 误差不是随机的，是确定性偏置，方差公式失真。
- 步长太大（低比特，如 2-bit）→ 分数部分不再近似均匀。
- 数据被裁剪（clipping）→ 裁剪误差混进来，不再是均匀模型。
此时 S²/12 只能当粗略估计，实际误差要靠校准（KL 散度 / MSE）来度量。
5.3 性质 3：信噪比与 SQNR = 6.02n dB
信噪比（SNR）= 信号功率 ÷ 噪声功率：
SNR = P_signal / P_noise
"功率"是什么？ 对一串数字 x，功率就是均方值 P = E[x²]。关键性质：对零均值信号，功率 = 方差（Var = E[x²] − E[x]²，均值 0 时第二项为 0）。这正是上面 S²/12 能直接当"噪声功率"用的原因。
dB 是什么？ dB（分贝）是一个用对数表示"比值"的单位——它不是某个物理量，而是回答"大了多少倍"的压缩写法：
功率比： dB = 10 · log₁₀( P / P₀ )
幅度比： dB = 20 · log₁₀( A / A₀ )
- 是比值，不是绝对值：必须有两个数才能算 dB（工程里"−3 dBm"等写法是约定了参考值 P₀ = 1 毫瓦，本质仍是比值）。
- 为什么 10 或 20：dB 度量功率比，用 10·log；功率正比于幅度的平方，所以幅度比要用 20·log（10·log(A²/A₀²) = 20·log(A/A₀)）。
为什么要用对数？ 两个理由：
- 数值范围太大，对数压扁：信噪比可从 1 倍到 10 万倍——10× 是 10 dB、100× 是 20 dB、1000× 是 30 dB、63096× 才 48 dB。对数把跨数量级的比值压成好记的小数字。
- 乘法变加法：信号经 5 级放大、每级 10 倍，总增益 10⁵ 倍；用 dB 只需 10+10+10+10+10 = 50 dB。级联系统增益直接相加——这是 dB 在工程里无处不在的根本原因。
几个值得记住的锚点：
量化的信噪比专门叫 SQNR（量化信噪比），噪声就是量化误差 ε。推导分四步：
步骤 1：信号功率（信号在 [−α, α] 均匀分布，区间长 2α，方差 (b−a)²/12）：
P_signal = (2α)² / 12 = α² / 3
步骤 2：噪声功率（量化误差在 [−S/2, S/2] 均匀）：
P_noise = S² / 12
步骤 3：步长与位宽的关系（n 比特有 2ⁿ 个刻度，铺满总长 2α）：
S = 2α / 2ⁿ
步骤 4：相除：
SQNR = (α²/3) / (S²/12) = 4α² / S²
= 4α² / (4α²/2^(2n)) = 2^(2n)
SQNR(dB) = 10·log₁₀(2^(2n)) = 20n·log₁₀(2) = 6.02·n dB
"6 dB/bit" 的含义：6.02·n dB 是总量公式，6 dB/bit 是它的斜率——同一件事的两种说法，就像"车费 = 6.02×n 元"和"单价 6.02 元/公里"。6.02·n 这个式子本身就是"每 bit 6.02 dB × 共 n 个 bit"。
从推导链验证"为什么每 +1 bit 恰好 +6 dB"，看 n → n+1 时发生什么：
① 刻度数翻倍：2ⁿ → 2ⁿ⁺¹
② 步长减半： S = 2α/2ⁿ → S/2
③ 噪声功率降到 1/4：S²/12 → (S/2)²/12
④ 信噪比 ×4，换算成 dB：10·log₁₀(4) = 20·log₁₀(2) = 6.02 dB
每 +1 bit 都重复这条链条、都 +6.02 dB——注意第④步就是上面锚点表里的"+6 dB = 幅度 ×2"：加 1 bit 让整数范围恰好翻倍。n 次累加就是 6.02·n。用 n 从小到大看更直白：
（口语里常说"6 dB/bit"，6 是 6.02 的约整。）
48 dB 意味着什么：把 dB 反解回倍数——dB 的定义是"倍数取对数"，反过来就是"对数当指数"：
dB = 10·log₁₀(P/P₀) ← 定义
dB/10 = log₁₀(P/P₀) ← 两边 ÷10
P/P₀ = 10^(dB/10) ← 两边作为 10 的指数，对数消掉
即 10^(dB/10) 就是 10·log₁₀(·) 的逆运算。代入 48：
功率比 = 10^(48/10) ≈ 63096 ← 信号功率是噪声的 6 万多倍
幅度比 = 10^(48/20) ≈ 251 ← 信号幅度是噪声的 251 倍
（幅度版除以 20 是因为幅度平方才是功率；一致性检查：251² ≈ 63000 ✓。口算技巧：每 10 dB 添一个零——48 dB = 40 dB + 8 dB ≈ ×10000 × 6.3。）
即 int8 量化的噪声幅度只有满量程的 1/251 ≈ 0.4%。
一个常见变体：若你见过 6.02n + 1.76 dB，那是满幅正弦波的经典 ADC 公式——正弦波功率 = α²/2（峰值 α），比均匀信号 α²/3 大，多出 10·log₁₀(1.5) ≈ 1.76 dB。区别只在信号形状假设，本质是同一套推导；本文假设均匀信号，用 6.02n dB。
5.4 性质 4：误差相加按 √N 增长，而非 N
若 N 个误差彼此独立：
Var(Σ εᵢ) = N · S²/12 ⟹ std(Σ εᵢ) = √N · S / √12
均值恒为 0。正负误差互相抵消，总误差不会线性爆炸——这是"整张网络几乎不掉点"的统计根基。
6. 部署视角：S/Z 的保存与恢复
6.1 S 和 Z 必须单独保存
S、Z 是量化的"字典"——量化后的权重只是一堆整数，没有 S、Z 就翻译不回真实值，所以必须随模型一起保存/下发（S 一般存 FP32 或 FP16，Z 存成和 q 一样的整数类型）。
但保存开销极小，具体看粒度。以一层卷积权重 [512, 512, 3, 3] ≈ 236 万参数 为例：
对比：INT8 本身把权重从 9.4 MB 压到 2.36 MB。S/Z 元数据通常连 1% 都不到。粒度越细字典越大——这是粒度权衡的另一半。
6.2 「恢复」有两种玩法
玩法 A：反量化后算浮点（weights-only 量化）
存的是 int8 权重 → 推理时先 r = S(q−Z) 还原成 float → 做 float 矩阵乘
激活值保持 float。好处：改动小，LLM 推理（如 llama.cpp 的 GGUF）常用这招。缺点：计算本身还是 float，没吃到整数算力的便宜。
玩法 B：整数 GEMM（integer-only，端侧主流）
权重和激活都保持整数 → 直接做整数矩阵乘 Q_w·Q_x → 最后统一乘一次 S_w·S_x
这里不逐个数恢复——还原动作被"推迟"到整个矩阵乘算完之后，只乘一次标量。既省内存，又省算力。下一节推导它的数学。
7. 从标量到矩阵：整数 GEMM
7.1 矩阵形式的仿射映射：J 的作用
W ≈ S_w·(Q_w − Z_w·J) 就是标量公式 r = S(q−Z) 的矩阵版——对 W 的每个元素套一遍 S(q−Z)（per-tensor，共用一套 S、Z），用矩阵符号一次写出来。
问题在于"矩阵元素减标量"不规范，于是引入全 1 矩阵 J（和 W 同形状）：
Z_w · J = 每个元素都是 Z_w 的矩阵
这样"减标量 Z_w"就规范地写成了"减矩阵 Z_w·J"：
Q_w − Z_w · J = Q_w 的每个元素都减去 Z_w
注意：Z_w · J 里的 · 是标量乘矩阵，不是矩阵乘法。
数值例子（非对称、Z≠0）：激活 x 范围 [−0.5, 1.0]，uint8，S_x = 1.5/255 ≈ 0.005882，Z_x = 85：
x = [ −0.5 ] Q_x = [ 0 ]
[ 0.0 ] [ 85 ]
[ 1.0 ] [ 255 ]
Q_x − Z_x·J = [ 0 ] [ 85 ] [ −85 ]
[ 85 ] − [ 85 ] = [ 0 ]
[ 255] [ 85 ] [ 170 ]
S_x·(Q_x − Z_x·J) = 0.005882 · [ −85 ] = [ −0.5 ] ✓
[ 0 ] [ 0.0 ] ✓
[ 170 ] [ 1.0 ] ✓
Q_x 里存的 85 代表实数 0，所以要减掉 85（即 Z_x）"归零"，再乘步长。
为什么要写成带 J 的形式？ 纯粹是为了下面 GEMM 展开的代数方便——有了 J 才能用分配律把"整数部分"和"零点修正项"干净拆开。J 是代数工具，实际代码里就是"每个元素减 Z"，不会真的存一个全 1 矩阵。
7.2 GEMM 展开
权重矩阵 W 与输入 x 量化后：
W ≈ S_w · (Q_w − Z_w · J)
x ≈ S_x · (Q_x − Z_x · J)
矩阵乘法用分配律展开：
W·x ≈ S_w · S_x · [ Q_w·Q_x − Z_w·J·Q_x − Z_x·Q_w·J + Z_w·Z_x·J·J ]
- Q_w·Q_x 是纯整数矩阵乘法——唯一昂贵的一步，而整数 MAC 单元比 FP32 便宜约一个数量级。
- 后三项只是"按行/列求和 + 乘标量"，可预计算或廉价完成。
- 若两侧均对称（Z_w = Z_x = 0），后三项全部消失：
W·x ≈ S_w · S_x · (Q_w · Q_x)
只剩"一次整数 GEMM + 一次标量乘法"。这正是对称量化在硬件上最受欢迎的原因。
卷积同理：权重核在空间滑动时被共享，零点修正项可以提前算好，不增加每步的额外开销。
7.3 什么是"纯整数 GEMM"：一个点积例子
GEMM（GEneral Matrix Multiply，通用矩阵乘法）就是矩阵乘法，神经网络里全连接层和卷积的核心。每个输出元素是一个点积——"相乘再累加"（Multiply-Accumulate，MAC）：
C[i,j] = Σ_k A[i,k] × B[k,j]
"纯整数"指：操作数（Q_w、Q_x）都是整数，中间每一次乘、每一次加都是整数运算，浮点不混在循环里。
具体例子：权重 W = [1.0, 0.5, −0.5]，输入 x = [2.0, 1.0, −1.0]，真实点积：
W·x = 1.0×2.0 + 0.5×1.0 + (−0.5)×(−1.0) = 3.0
量化成 int8（对称，W 范围 [−1,1] → S_w=1/127，x 范围 [−2,2] → S_x=2/127）：
Q_w = [127, 64, −64]
Q_x = [127, 64, −64]
纯整数 GEMM 算的是：
Q_w·Q_x = 127×127 + 64×64 + (−64)×(−64) = 24321 ← 全是整数乘加
最后乘一次标量还原：
W·x ≈ S_w·S_x × (Q_w·Q_x) = (1/127 × 2/127) × 24321 ≈ 3.016 ≈ 3.0 ✓
贵的部分（3 次乘 + 3 次加）全在整数域，浮点只用了 1 次（最后乘 S_w·S_x）。
一个工程细节：整数点积累加会溢出 int8，所以累加器（accumulator）通常用 int32，最后再缩回。
7.4 为什么这很重要
- 硬件便宜：整数乘法器/加法器在芯片上的面积和功耗，比浮点单元小一个数量级。
- 数据搬得少：int8 比 float32 少 4 倍字节，内存带宽省 4 倍。
8. 为什么量化后精度损失不大
把前面的数学拼起来，答案有四层：
8.1 误差统计层：小、均匀、互相抵消
- 误差有界且零均值（性质 1、2）：单个误差 ≤ S/2，int8 时只有满量程千分之几。
- 误差互相抵消（性质 4）：求和时标准差按 √N 增长而非 N，网络各层的成千上万个误差不会叠加成大误差。
8.2 网络鲁棒层：Lipschitz 连续
Lipschitz 连续给"函数变化有多猛"设上限。函数 f 是 L-Lipschitz 连续的，若存在常数 L ≥ 0，对任意 x, y：
|f(x) − f(y)| ≤ L · |x − y|
L 就是 Lipschitz 常数。一句话直觉：输入变化多少，输出最多变化 L 倍，绝不超过。
对可导函数，Lipschitz 常数就是导数的最大绝对值 L = sup |f'(x)|——即"导数处处有界，函数不会比某条直线更陡"。
以 ReLU 为例感受 L=1 的含义：
输入 x 从 1.0 变成 1.1（扰动 0.1）
ReLU(1.1) − ReLU(1.0) = 0.1 → 变化正好等于扰动（不放大）
输入从 −1.0 变成 −0.9（扰动 0.1）
ReLU(−0.9) − ReLU(−1.0) = 0 − 0 = 0 → 变化为 0（比 L·扰动还小）
量化误差 ε 是加到每层输出上的小扰动。扰动逐层传播会不会爆炸，就看每层是否 Lipschitz：
输出扰动 ≤ L · 输入扰动
- 常用激活函数（ReLU/sigmoid/tanh）都是 Lipschitz 的，误差逐层最多线性累积，不会指数爆炸。
- ReLU（L=1）和 sigmoid（L=0.25）不仅不放大，还常常收缩——误差越传越弱。
- 反例：若激活像 x² 那样导数无界，小误差经过几层就可能被疯狂放大，量化就崩了。神经网络常用的激活函数恰好都避开了这一点——这就是量化误差"传不远"的数学保障。
延伸：权重矩阵本身也可以算 Lipschitz 常数（它的最大奇异值）。完整一层 y = φ(Wx+b) 的 Lipsc
本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
This is a mathematically complete dismantling of the model quantization (Quantification): how it maps the FP32 floating point weight/activation map to a low bit integer, how S and Z requested, where quantization error came from, how strictly it proved its boundaries, how it was pushed, and why, after 8-bit and even 4-bit Z83Z, the loss of the accuracy of the model is usually not significant. The content framework is consistent with the Quantification of the MIT 6.5940 / TinyML (Hanson Song Han) course.
In WeChatWeChat Official Account: AngryCode The Code of Anger
0. TL;DR
- The essence of quantization is a simulation transformation: r = S(q — Z). r is float, q and Z are all integers, S is float -- the whole formula has only one floating point multiplier.
-S = Data range ÷ integer range. The data range is based on three requirements: min-max, cropping +MSE, cropping + KL bulk.
- Errors from rounding can be strictly demonstrated by ≤ S/2, zero average, variance S2/12 (achievable points under even distribution assumptions).
6 dB (SQNR = 6.02n dB); int8 noise is only ~0.4% of the full range.
- A positive and negative offset (standard difference increased by √N) at sum/dissemination, and the activation function Lipschitz continues (L=1) so that the error is not magnified and therefore the whole network is almost intact.
- The matrix multiplication can be broken down to a pure integer GEMM + multiplication at a single scale, which is a mathematical source of quantization savings in computing, bandwidth and utility.
1. Why quantization
The end/margin equipment (cell phones, MMU, NPU) is subject to four hard restraints:
- Memory: FP32, with a weight of 4 byte, int8 only reduced four times the size of the 1 byte → model.
- Bandwidth: Memory handling is the main source of energy consumption for end-side reasoning, with a reduction in bit width being directly proportionally power-saving.
- Calculus: Integer MAC unit size, utility, latency are well below FP32 MAC.
- Real-time: integer operation + smaller memory occupancy, lower delay.
One thing: quantization is not "better," but "run." And it works because quantization errors are mathematically manageable -- that's what this is about.
2. From the point of departure: what exactly does FP32 have?
IEEE 754 = 1 symbol + 8 index + 23 end-digit:
x = (−1)^s × 2^(e−127) × (1 + f / 2^23)
- Numerical range: approximately ±3.4 x 1038
- Relative precision: approximately 2-23 ≈ 1.2 x 10-7 (approximately 7 significant figures)
Key cognition: Neuronet weight is the statistically trained, not the physical constant. It does not require the precision of a 7-bit effective number. This provides huge redundancy space for quantization.
3. Mathematical definition of quantization: mimic map
3.1 Core formula
The core of quantization is to map the actual r to the integer q:
r = S (q — Z) (reverse quantization: Integer)
q =clamp (round(r / S)+Z, q min, q max) (quantization: real integer)
What is the clamp: a limited range function - insert values into [min, max] compartments, press too little to min, too much to max and return as follows:
Clamp(x, min, max) = min if x < min
= x If am x max
= max if x > max
Its role in quantization: round(r/S) + Z may exceed the lawful range of integers - e.g., r is outsider not seen at calibration and calculates an integer exceeding [q min, q max] (e.g. int8 calculates 300 or −254). clamp guarantees that the result will go in n bits (int8 is [-128, 127] and will not spill back. The price is a cut-off error in the value that has been caught. This is exactly the source of the error in calibration (KL/MSE selection threshold). clamp is also called crip (NumPy/TensorFlow), saturation operation (hardware) in different frames.
3.2 Four volumes of each type (a common misunderstanding)
Z is not float, is integer (same integer with q):
So the operation of r =S(q)-Z is:
(q — Z) Integer reduction
x S-rated, get a float
There's only one float multiplication in the whole equation, and the rest is integer calculations -- that's the key to quantization.
Why must Z be an integer?
Z = round( q_min − r_min / S )
q min is integer, r min/S is float, minus round integer, so Z is born integer and falls within [q min, q max] and can be saved with n bits like q. If Z was float, the hardware would have to be float-down, an extra float, and quantization would have lost its meaning.
A specific example: data range r ∈ [-0.5, 1.0], uint8 quantization (q ∈ [0, 255]:
S = (1.0 − (−0.5)) / 255 = 1.5/255 ≈ 0.005882
Z = round(0 − (−0.5)/0.005882) = round(85) = 85
Validation r = 0:S(q — Z) = 0 ⟹ q = Z = 85. Real 0 corresponds to the integer 85-Z is the integer.
3.3 Why is it called a simulation?
Because r = S.q - S.Z. is a straight line: slope S, cut-S.Z. It allows "Zoom + Horizontal" so it can represent both symmetric distribution (Z=0) and asymmetric distribution (Z≠0, such as the non-negative activation of RLU output). All conclusions in this paper are drawn from this line.
4. How to request S and Z
S = data range integer range. The integer range is defined by bit width, and what really determines is the "data range".
n Integer range:
4.1 Method I: Min-max Asymmetric (full coverage, no cropping)
S = (r_max − r_min) / (q_max − q_min)
Z = round( q_min − r_min / S )
Validation peer alignment: r min = S (q min — Z), r max = S (q max — Z).
Example: Level ReLU activated, range [0, 6.0], with uit8:
S = (6.0 − 0) / (255 − 0) = 6/255 ≈ 0.023529
- Advantages: No value cut, all covered.
- Shortcomings: extremely sensitive to outliers. If 99.9 per cent of the value falls on [0,0.5] and only one 6.0, the S is boosted by this outlier, the majority of the value quantization error is unnecessarily magnified.
4.2 Method II: Min-max symmetry (Z=0) - why absolute values must be added
Symmetric Z quantization Force = 0, map degradation to r = S.q:
α = max(|r_min|, |r_max|)
S = alpha / q max (q max = 2^ (n−1) - 1)
Here abs is not optional, is the definition itself. Reason: The symmetric quantization requires a map of 0 symmetrics - the integers can take -127 and +127, the actual numbers are -S.127 and +S.127, so that the actual range must be [-alpha, alpha], and the alpha must be the value of the "most distance from -0" in all data, the maximum absolute value.
Example: What is not absolute? min = -2.0, r max = 1.0:
- Without abs: max(r min, r max) = 1.0, S = 1.0/127. Then -2.0 = −2.0/(1/127) = −254, which is beyond the range of int8 [—127, 127], is pinned to −127, and all negative values less than −1.0 are all distorted.
- Add abs: alpha = max (|2|,|1|) = 2.0, S = 2.0/127. Intersection [-2, 2] [-127, 127], fully covered.
Data range (asymmetrical): [--r min=-2.0 — 0 — — r max=1.0 —]
Symmetrical quantization forced amortization: [------------------------- -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
↑
Right.
In order to maintain the simplicity of the symmetric Z quantization, the cost is that the two sides must spread over the symmetry, and the other half of the range is wasted when the data are side by side — this is the core difference between it and the symmetric Z quantization.
Details of realization: q max with a symbol int8 uses 127 instead of 128, which is considerably simplified by GEMM for the small cost of "compulsory symmetry" (−128 idle).
4.3 Method III: Crop + calibration (for outliers)
Min-max's problem is outlier. Smarter: deliberately cutting down a small number of extreme values in return for a smaller step.
Sets the cut threshold T (0 < T≤ max|r|), only the value in quantization [-T, T], which exceeds the direct T-string to the end:
S = T / q max (symmetric)
The selection of T is central, with two guidelines:
1 MSE Code: Select T to minimize the average of total errors:
T* = argmin_T E[ (Q(r) − r)² ]
= argmin T[clip error (over and above T) + quantization error (S2/12)]
T's too small to cut big; T's too big S's big, quantization's big. The two are different, and T balances them best.
2 KL Bulk Guidelines (NVIDIA TensorRT practice): make histograms of FP32 and quantization values, selecting T to minimize the loss of information in both distributions:
T* = argmin_T KL( P_fp32 ‖ P_quant(T) )
Intuitive understanding: Select a T that looks like "value distribution before quantization" and "value distribution after quantization". The best T experience of the weight of the Gos/Lapras distribution is about ± (2.5 ~ 4) - not in the full range of Min-max, but in several times the standard difference.
4.4 Data from: weight vs activated (a key difference)
So PTQ processes often see "Run forward with 100-1000 calibrated pictures, measure the distribution of each layer active" -- that's for active S (and Z).
4.5 Methodological summary
5. Mathematical analysis of quantization errors
The value after quantization is marked as Q(r) = S round (r/S + Z) - S Z, error:
ε = Q(r) − r
5.1 Nature 1: Complete proof that the error has a boundary - |ε|S/2
This upper boundary was established only when "values fell within quantization and were not cut." Proves that only three steps are required, and the core is to break the error to "S X rounded the error."
Step 1: Define the intermediate variable, with r expressed as t.
t = r/ S + Z, unsolve r = S (t — Z). t means "r position on integer scales." quantization rounded this position:
q = round(t) = round(r/S + Z)
Step 2: Rewrite quantization error.
ε = Q(r) − r
= S.(round(t)-Z)-S.(t)-Z ← back to Q(r) and r
= S. (round(t) - t) ← S.Z. and -S.Z.
= round(t) - t (rounded error in tics, no sauna), received:
ε = S · δ
Step three: Prove ≤ 1/2.
t, which must fall within [n, n+1] of an integer range. Rounded to the nearest integer, the result is either n or n+1, and the distance is t-n and (n+1)-t, respectively. And the sum of the two:
(t − n) + ((n+1) − t) = 1
The two paragraphs add up to one, so the shorter part must be one-two. Rounded to the nearest integer, so:
|δ| = |round(t) − t| = min(t−n, (n+1)−t) ≤ 1/2
Step two:
|ε| = S·|δ| ≤ S/2 ∎
Intuitive understanding: The essence of rounding is "to the nearest scale" and the t-range is 1 (in the t-coordinate system), so it takes up to a half-scale. convert returns the real-number coordinates, each tic representing S, and is therefore at most wrong with S/2.
Examples of values: S = 0.1, Z = 0, r = 0.53:
t = 0.53/0.1 = 5.3 → round = 5 → δ = 5 − 5.3 = −0.3
ε = S·δ = 0.1 × (−0.3) = −0.03
|ε| = 0.03 ≤ 0.05 = S/2 ✓
⚠️ An exception that must be remembered: this upper boundary is set only for "value within range". If r exceeds [r min, r max], it will be clamp to the end, at which point the error can be any large - the cut error is wide:
Total error = quantization error (≤ S/2) + crop error (unlimited)
This is exactly what Section 4.3 "KL Diff Threshold" addresses: too narrow in scope, too wide in size, too wide in scope, too wide in scope, too wide in scope, and too wide in scope.
5.2 Nature 2: Zero average error, variance S2/12 - Full extrapolation
The difference S2/12 is calculated on the assumption that the error of quantization is evenly distributed on [-S/2, S/2].
Presumptence: Why the twilight is even. Set t = r/ S + Z, fraction frac = t - ⌊t⌋ [0, 1]. Rounded:
frac ∈ [0, 0.5) → round(t) = ⌊t⌋ → δ = round(t)−t = −frac ∈ (−0.5, 0]
frac ∈ [0.5, 1) → round(t) = ⌊t⌋+1 → δ = 1 − frac ∈ (0, 0.5]
Key: If frac is evenly distributed on [0,1] (which is a good approximation if the input data is enough "hot" and long enough) then zirconium is evenly distributed on (-0.5, 0.5). Multiply the constant S, get:
ε = S. δ evenly distributed on [-S/2, S/2]
This is the Bennett even model.
Expectations (zero average): evenly distributed PDF is f (ε) = 1/S (the length of the zone is S, total probability 1):
E[ε] = ∫ ε·f(ε) dε = (1/S) ∫_{−S/2}^{S/2} ε dε = 0
Because the thallium is an odd function, and the score on the symmetrical 0 must be 0.
Difference (step-by-step): Var(ε)=E[ε2]−(E[ε]2), while E[ε]=0, so:
Var(ε) = E[ε²] = (1/S) ∫_{−S/2}^{S/2} ε² dε
= (1/S) · [ ε³/3 ]_{−S/2}^{S/2}
= (1/S) · [ (S/2)³/3 − (−S/2)³/3 ]
= (1/S) · [ S³/24 − (−S³/24) ]
= (1/S) · ( S³/12 )
= S²/12
Standard deviation:
σ_ε = S / √12 ≈ 0.289·S
Geometric Intuitive: The random variable distributed evenly on [-S/2, S/2], the RMS value is S/√12 ≈ 0.289S - much smaller than the maximum S/2 = 0.5S, as most errors occur near 0, and a few at ±S/2 extremes.
Numerical example: int8, range [-1,1]: S = 1/127 ≈ 0.00787.
== sync, corrected by elderman ==
max = S/2 ≈ 0.00394 ← error "worst case"
The typical error is 0.23 per cent of the full range and 0.39 per cent of the worst -- this is the visual source of int8 precision.
Assuming that it will expire:
- The data are almost constant (all levels of activation are similar), the error is not random, it is definitive bias, and the variance formula is distorted.
- Too long (low bit, e.g., 2-bit) fractions are no longer nearly even.
- The data is clipping and cutting errors are mixed in, not even models.
At this point, the S2/12 can only be roughly estimated and the actual error is measured by calibration (KL diffusion / MSE).
5.3 Nature 3: Noise ratio with SQNR = 6.02n dB
SNR = signal power
SNR = P_signal / P_noise
What's power? For a series of numbers x, the power is the average P = E [x2]. Critical properties: = variance (Var = E[x2] - E[x]2, equal value 0 and second value 0). That's why the S2/12 above can be used directly as noise power.
What's db? dB is a unit with a logarithm of "comparison" -- it's not a physical amount, but a compression that answers "how many times greater":
Power ratio: dB = 10 y log10 (P / P0)
Ratio: dB = 20 & log10 (A / A0)
- is a margin, not an absolute value: two digits are necessary to calculate dB (the term "−3 dBm" in the project is the reference value P0 = 1 mW, which is still the value).
- Why do 10 or 20:dB measure power ratio, using 10 log; power ratio is just above square range, so range ratio is 20 log (10 log (A2/A02) = 20 log (A/A0).
Why use a logarithm?
- The range of values is too large and the logarithmic is flat: the cell noise ratio can be 1 to 100,000 times - 10 x 10 dB, 100 x 20 dB, 1,000 x 30 dB, 63096 x 48 dB. The logarithms crush the cross-numeric scale margin into a small number.
- Multiplication: The signal is magnified by 5 levels, 10 times each level, with a total gain of 105 times; using dB only 10+10+10+10+10 = 50 dB. Cascade system gain is directly added -- this is the root cause of dB being everywhere in the project.
Several anchors worth remembering:
quantization ' s noise ratio is specifically called SQNR (quantization ' s noise ratio), and the noise is quantization ' s error. Producing in four steps:
Step 1: Signal power (the signal is in [-alpha, alpha] evenly distributed, zone long 2al, variance (b-a)2/12):
P_signal = (2α)² / 12 = α² / 3
Step 2: Noise power (quantization error equals [-S/2, S/2]):
P_noise = S² / 12
Step 3: Step length in relation to bit width (n bits 2n scales, total length 2α):
S = 2α / 2ⁿ
Step 4: Separate:
SQNR = (α²/3) / (S²/12) = 4α² / S²
= 4α² / (4α²/2^(2n)) = 2^(2n)
SQNR(dB) = 10·log₁₀(2^(2n)) = 20n·log₁₀(2) = 6.02·n dB
Meaning of "6 dB/bit": 6.02 n dB is the total formula, and 6 dB/bit is its slope -- two terms of the same thing, like "car fare = 6.02 x n" and "unit price 6.02 $/km". 6.02.n The formula itself is "n bit per bit 6.02 dB x n".
Check from the chain "why every +1 bit happens +6 dB" and see what happens when n n+1:
1 Triple Scale: 2n → 2n + 1
2 Halve step: S = 2α/2n → S/2
3 Noise power down to 1/4: S/2/12 → (S/2/2/12)
4 Letter Noise Ratio x 4, convert to dB:10 log10(4) = 20 log10(2) = 6.02 dB
Each +1 bit repeats this chain, +6.02 dB -- Note that step 4 is "+6 dB = range x 2" in the above anchor table: plus 1 bit to double the integer range. n Plus is 6.02 n. Use n to look straighter:
(Speaks "6 dB/bit", 6 is about 6.02.
48 dB What does it mean: to invert the dB multiple - dB is defined as a "multiplier logarithmic" and vice versa:
dB = 10 log10 (P/P0) ← Definitions
dB/10 = log10 (P/P0)
P/P0 = 10^ (dB/10) ← Index to 10 on both sides, logarithms eliminated
10^ (dB/10) is the reverse of 10 log10(.). Substitute 48:
Power ratio = 10^ (48/10) ≈ 63096 ← signal power more than 60,000 times noise
= 10^ (48/20) ≈ 251 ← The signal range is 251 times the noise.
(Range version divided by 20 because range squares are power; consistency check: 2512 ≈ 63,000 zirconium. Taste techniques: Add zero per 10 dB - 48 dB = 40 dB + 8 dB ≈ × 10,000 x 6.3.
The noise range of int8 Z83Z is only 1/251 ≈ 0.4% of the full range.
A common variant: If you've seen 6.02n + 1.76 dB, it's the classic ADC formula with a swirl wave = alpha2/2 (peak alpha), which is larger than the flat signal alpha2/3, with 10 log10(1.5) ≈ 1.76 dB. The difference is only in the shape of the signal, which is essentially the same type of extrapolation; it assumes a flat signal using 6.02n dB.
5.4 Nature 4: Increase by √N plus error instead of N
If N is independent:
Var(Σ εᵢ) = N · S²/12 ⟹ std(Σ εᵢ) = √N · S / √12
Average constant is 0. Positive and negative errors offset each other, and total errors do not explode linearly — this is the statistical foundation of the entire network.
Deployment perspective: S/Z preservation and restoration
6.1 S and Z must be saved separately
S, Z is the "Dictionary" of quantization - the weight of quantization is just a bunch of integers, without which S, Z is not translated back to the true value, so they must be stored/released with the model (S General Save FP32 or FP16, Z is saved into the same integer type as q).
However, preservation costs are minimal, depending on the particle size. Example:
Comparison: INT8 itself presses the weight from 9.4 MB to 2.36 MB. S/Z metadata are usually not even 1%. The more the particle size, the bigger the dictionary -- this is the other half of the particle size.
6.2 There are two ways to "recover."
Play A: Post-quantization floats (weathers-only quantization)
Saves the int8 weight r = S(q-Z) back to float → for float matrix Multiplication
Activate values to keep float. Benefits: Small changes, LLM reasoning (e.g. llama.cpp GGF) is common. Disadvantages: The calculations themselves are float, without the integers.
Play method B: Integer-only
Weights and activations are kept integers.
There's no one-to-one recovery -- the reduction was "delayed" until after the entire matrix had been multiplied, only once. It saves memory and energy. Next section leads it to math.
7. From scale to matrix: integer GEMM
7.1 Simulation mapping in matrix form: The role of J
W ≈S w (Q w — Z w.J) is the matrix version of the mass formula r = S(q-Z) — each element of W is packaged over S(q-Z) (per-tensor, shared with a set of S, Z), written in a matrix symbol at a time.
The problem is that "matrix element reduction" is not regulated, so it introduces a full 1 matrix J (in the same shape as W):
Z w J = Each element is a Z w matrix
So it's a standard word for "deductive Z-w" as "reduced matrix Z-w-J":
Q w - Z w. J = Q w
Note: Z w J's is a scale multiplied matrix, not a matrix multiplied.
Numerical examples (asymmetric, Z≠0): Activate range x [−0.5, 1.0], uint8, S x=1.5255 ≈0.005882, Z x =85:
x = [ −0.5 ] Q_x = [ 0 ]
[ 0.0 ] [ 85 ]
[ 1.0 ] [ 255 ]
Q_x − Z_x·J = [ 0 ] [ 85 ] [ −85 ]
[ 85 ] − [ 85 ] = [ 0 ]
[ 255] [ 85 ] [ 170 ]
S_x·(Q_x − Z_x·J) = 0.005882 · [ −85 ] = [ −0.5 ] ✓
[ 0 ] [ 0.0 ] ✓
[ 170 ] [ 1.0 ] ✓
The 85 in Q x represents 0 real number, so the 85 (i.e. Z x) is "zero" and multiplication is long.
Why is it written in J's? It's simply for the convenience of the algebra that GEMM expands -- with J, it's possible to use the distribution law to clean up the whole number and zero point correction. J is an algebra tool. The actual code is "Elements minus Z". It does not really have a complete 1 matrix.
7.2 GEMM Expand
Weight matrix W and input x quantization:
W ≈ S_w · (Q_w − Z_w · J)
x ≈ S_x · (Q_x − Z_x · J)
The matrix multiplication is extended by the distribution law:
W·x ≈ S_w · S_x · [ Q_w·Q_x − Z_w·J·Q_x − Z_x·Q_w·J + Z_w·Z_x·J·J ]
- Q w.Q x is a pure integer matrix multiplication - the only expensive step, and the integer MAC unit is about one order of magnitude cheaper than FP32.
- The latter three are only "coding by row/list + multiplier" and can be expected or done cheaply.
- If both sides are symmetry (Z w = Z x = 0), the latter three are all gone:
W·x ≈ S_w · S_x · (Q_w · Q_x)
There's only one "integer GEMM+ one-scale multiplier". This is why the symmetric quantization is the most popular on hardware.
Volume homogeneity: Weight weights are shared when space slides, and zero-point amendments can be calculated well in advance without additional cost per step.
7.3 What's a "pure integer" GEMM: a point example?
GEMM (GEneral Master Multiplication, General Matrix) is the core of the matrix multiplication, full-connected layer and volume in the nervous network. Each output element is a point volume - "multiple-Accumulate", MAC:
C[i,j] = Σ_k A[i,k] × B[k,j]
"Preliminary integer" means that the number of operations (Q w, Q x) is an integer, each time a multiplication is added to the integer, and the floating point is not mixed in the cycle.
Example: W = [1.0, 0.5, −0.5], input x = [2.0, 1.0, −1.0], real point volume:
W·x = 1.0×2.0 + 0.5×1.0 + (−0.5)×(−1.0) = 3.0
quantization to int8 (symmetrical, W range [-1,1] → S w=1/127, x range [-2,2] → S x=2/127]:
Q_w = [127, 64, −64]
Q_x = [127, 64, −64]
GEMM calculates:
Q w.Q x = 127 x 127 + 64 x 64 + (−64) x (−64) = 24321 zirconium = all integers multiplied by
Last multiplied by standard reduction:
W·x ≈ S_w·S_x × (Q_w·Q_x) = (1/127 × 2/127) × 24321 ≈ 3.016 ≈ 3.0 ✓
All the expensive parts (3 times + 3 times) are in the integer domain, with only one float (last times S w.S x).
A project detail: the accumulation of integer points and the spilling of int8, so the loader (accumulator) usually uses int32, which is then retraced.
7.4 Why is this important?
- Hardware is cheap: the size and utility of the integer multiplication/merger on the chip is one order of magnitude smaller than the floating point unit.
- Few data moves: Int8 is 4 times less than float32 byte and memory bandwidth 4 times more.
Why is the post-quantization accuracy loss not significant?
Put the math together. The answer is four levels:
8.1 Error statistical layers: small, even, offsetting
- Serial and zero mean values (natures 1, 2): single error ≤ S/2, int8 with only a few parts per thousand of full range.
- Errors offset each other (characteristic 4): The sum-and-time-standard difference increases by N instead of N, and tens of thousands of errors across the network layers do not superimpose into large errors.
8.2 Network Stick Layer: Lipschitz Continuous
Lipschitz successively caps "how much change in function". Function f is a continuous L-Lipschitz, if there is a constant L ≥0, for any x, y:
|f(x) − f(y)| ≤ L · |x − y|
I'm the Lipschitz constant. One intuitive: input changes, output changes more than L times.
For conductive functions, the Lipschitz constant is the maximum absolute value of the yield L = sup |f'(x)| - i.e., "there is a boundary at the point of the guide and the function is not steeper than a line".
Feel the meaning of L=1 in ReLU:
Enter the x from 1.0 to 1.1 (disturbation 0.1)
RELU(1.1) - RELU(1.0) = 0.1 → Change is exactly the same as disturbance (no magnification)
Enter from -1.0 to -0.9 (disturbation 0.1)
RELU(-0.9) - RELU(-1.0) = 0 - 0 = 0 → change to 0 (less than L. Disturbing)
quantization error ε is added to the small disturbance of each layer of output. Disturbing layer by layer will explode.
Output Disturbing ≤ L · Enter Disturbing
- Common activated functions (ReLU/sigmoid/tanh) are all Lipschitz, with maximum linear accumulation of error layer by layer and no exponential explosion.
- ReLU(L=1) and sigmoid(L=0.25) are not only not amplified, but are often constricted - errors are weaker.
- In retrospect: if you activate a guide like x2, a small error can be dramatically magnified through several layers, and quantization collapses. The activation functions that are commonly used in the nervous network just avoid this -- that's the mathematical guarantee of the quantization error "not far."
Extension: The weight matrix itself can also count the Lipschitz constant (the largest odd value). Y = (Wx+b) full layer Lipsc
Auto-fetched and translated by AI Atlas. All rights belong to the original author.