机器学习-白板推导系列笔记（二十八）-BM

此文章主要是结合哔站shuhuai008大佬的白板推导视频：玻尔兹曼机_147min

全部笔记的汇总贴：机器学习-白板推导系列笔记

参考花书20.1

一、介绍

玻尔兹曼机连接的每个节点都是离散的二值分布，是全连接的，是为了解决局部最小值的问题而提出的玻尔兹曼机。

v = { 0 , 1 } D            h = { 0 , 1 } P L = [ L i j ] D ∗ D J = [ J i j ] P ∗ P W = [ W i j ] D ∗ P v=\{0,1\}^D\;\;\;\;\;h=\{0,1\}^P\\L=\Big[L_{ij}\Big]_{D*D}\\J=\Big[J_{ij}\Big]_{P*P}\\W=\Big[W_{ij}\Big]_{D*P} v={ 0,1}Dh={ 0,1}PL=[Lij​]D∗D​J=[Jij​]P∗P​W=[Wij​]D∗P​

{ p ( v , h ) = 1 Z exp ⁡ { − E ( v , h ) }                                      E ( v , h ) = − ( v T W h + 1 2 v T L v + 1 2 h T J h ) θ = { W , L , J } \left\{\begin{matrix} p(v,h)= \frac1Z\exp\{-E(v,h)\}\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\\E(v,h)=-(v^TWh+\frac12v^TLv+\frac12h^TJh)\end{matrix}\right.\\\theta=\{W,L,J\} { p(v,h)=Z1​exp{ −E(v,h)}E(v,h)=−(vTWh+21​vTLv+21​hTJh)​θ={ W,L,J}

二、Log似然的梯度

样本集合：$V,|V|=N$

$$\begin{gathered} P(v)=\sum _{h}p(v,h) \\ \frac{1}{N}\sum _{v\in V}\log P(v)\leftarrow log-likelihood \\ \frac{\partial }{\partial \theta }\frac{1}{N}\sum _{v\in V}\log P(v)=\frac{1}{N}\sum _{v\in V}\frac{\partial \log P(v)}{\partial \theta }\leftarrow gradientoflog-likelihood \end{gathered}$$

$$\frac{\partial \log P(v)}{\partial \theta }=\sum _v\sum _{h}p(v,h)\cdot \frac{\partial E(v,h)}{\partial \theta }-\sum _{h}p(h|v)\cdot \frac{\partial E(v,h)}{\partial \theta }$$
$$\begin{gathered} \frac{\partial \log P(v)}{\partial W}=\sum _v\sum _{h}p(v,h)\cdot (-v{h}^T)-\sum _{h}p(h|v)\cdot (-v{h}^T) \\ =\sum _{h}p(h|v)\cdot v{h}^T-\sum _v\sum _{h}p(v,h)\cdot v{h}^T \end{gathered}$$

所以，

$$\begin{gathered} \frac{1}{N}\sum _{v\in V}\frac{\partial \log P(v)}{\partial \theta }=\frac{1}{N}\sum _{v\in V}\sum _{h}p(h|v)\cdot v{h}^T-\frac{1}{N}\sum _{v\in V}\sum _v\sum _{h}p(v,h)\cdot v{h}^T \\ =\frac{1}{N}\sum _{v\in V}\sum _{h}p(h|v)\cdot v{h}^T-\sum _v\sum _{h}p(v,h)\cdot v{h}^T \\ =E_{P_{Data}}\left[v{h}^T\right]-E_{P_{model}}\left[v{h}^T\right] \end{gathered}$$
$$\begin{gathered} P_{Data}=P_{Data}(v)P_{model}(h|v) \\ {P}_{model}=P_{model}(h,v)=P_{model}(v)P_{model}(h|v) \end{gathered}$$

三、基于MCMC的随机梯度上升

由上述推导，同理可得：

$$\Delta W=\partial \left(E_{P_{Data}}\right[v{h}^T]-E_{P_{model}}[v{h}^T\left]\right)$$
$$\Delta L=\partial \left(E_{P_{Data}}\right[v{v}^T]-E_{P_{model}}[v{v}^T\left]\right)$$
$$\Delta J=\partial \left(E_{P_{Data}}\right[h{h}^T]-E_{P_{model}}[h{h}^T\left]\right)$$

$$\begin{gathered} P_{Data}=P_{Data}(v)P_{model}(h|v) \\ {P}_{model}=P_{model}(h,v)=P_{model}(v)P_{model}(h|v) \end{gathered}$$

$$W^{(t+1)}=W^{(t)}+\Delta W$$

$$\Delta w_{ij}=\partial (\underset{positivephase}{\underbrace{E_{P_{Data}}\left[v_i{h}_j\right]}}-\underset{negativephase}{\underbrace{E_{P_{model}}\left[v_i{h}_j\right]}})$$
但是无论是正向还是负向都是难以处理的，是intractable的。

$$\begin{gathered} p(v_i=1|h,v_{-i})=\sigma (\sum _{j=1}^P{w}_{ij}{h}_j+\sum _{k=1/i}^D{L}_{ik}{v}_k) \\ p(h_i=1|v,h_{-i})=\sigma (\sum _{j=1}^D{w}_{ij}{v}_j+\sum _{m=1/i}^P{J}_{im}{h}_m) \end{gathered}$$

RBM:（如下图）
$$\begin{gathered} p(h|v)=\prod _{j=1}^{3}p(h_j|v) \\ p(h_{j=1}|v)=p(h_{j=1}|v,h_{-j})=\sigma (\sum _{i=1}^P{W}_{ij}{v}_i) \end{gathered}$$

四、条件概率推导

$$\begin{gathered} p(v_i=1|h,v_{-i})=\sigma (\sum _{j=1}^P{w}_{ij}{h}_j+\sum _{k=1/i}^D{L}_{ik}{v}_k) \\ p(h_i=1|v,h_{-i})=\sigma (\sum _{j=1}^D{w}_{ij}{v}_j+\sum _{m=1/i}^P{J}_{im}{h}_m) \end{gathered}$$

p ( v i ∣ h , v − i ) = p ( v , h ) p ( h , v − i ) = 1 Z exp ⁡ { − E ( v , h ) } ∑ v i 1 Z exp ⁡ { − E ( v , h ) } = exp ⁡ { v T W h + 1 2 v T L v + 1 2 h T J h } ∑ v i exp ⁡ { v T W h + 1 2 v T L v + 1 2 h T J h } = exp ⁡ { v T W h + 1 2 v T L v } ∑ v i exp ⁡ { v T W h + 1 2 v T L v } = exp ⁡ { v T W h + 1 2 v T L v } exp ⁡ { v T W h + 1 2 v T L v } ∣ v i = 0 + exp ⁡ { v T W h + 1 2 v T L v } ∣ v i = 1 p(v_i|h,v_{-i})=\frac{p(v,h)}{p(h,v_{-i})}\\=\frac{\frac1Z\exp\{-E(v,h)\}}{\sum_{v_i}\frac1Z\exp\{-E(v,h)\}}\\=\frac{\exp\{v^TWh+\frac12v^TLv+\frac12h^TJh\}}{\sum_{v_i}\exp\{v^TWh+\frac12v^TLv+\frac12h^TJh\}}\\=\frac{\exp\{v^TWh+\frac12v^TLv\}}{\sum_{v_i}\exp\{v^TWh+\frac12v^TLv\}}\\=\frac{\exp\{v^TWh+\frac12v^TLv\}}{\exp\{v^TWh+\frac12v^TLv\}\Bigg|_{v_i=0}+\exp\{v^TWh+\frac12v^TLv\}\Bigg|_{v_i=1}} p(vi​∣h,v−i​)=p(h,v−i​)p(v,h)​=∑vi​​Z1​exp{ −E(v,h)}Z1​exp{ −E(v,h)}​=∑vi​​exp{ vTWh+21​vTLv+21​hTJh}exp{ vTWh+21​vTLv+21​hTJh}​=∑vi​​exp{ vTWh+21​vTLv}exp{ vTWh+21​vTLv}​=exp{ vTWh+21​vTLv}∣∣∣∣∣​vi​=0​+exp{ vTWh+21​vTLv}∣∣∣∣∣​vi​=1​exp{ vTWh+21​vTLv}​

所以，

p ( v i = 1 ∣ h , v − i ) = exp ⁡ { v T W h + 1 2 v T L v } ∣ v i = 1 exp ⁡ { v T W h + 1 2 v T L v } ∣ v i = 0 + exp ⁡ { v T W h + 1 2 v T L v } ∣ v i = 1 p(v_i=1|h,v_{-i})=\frac{\exp\{v^TWh+\frac12v^TLv\}\Bigg|_{v_i=1}}{\exp\{v^TWh+\frac12v^TLv\}\Bigg|_{v_i=0}+\exp\{v^TWh+\frac12v^TLv\}\Bigg|_{v_i=1}} p(vi​=1∣h,v−i​)=exp{ vTWh+21​vTLv}∣∣∣∣∣​vi​=0​+exp{ vTWh+21​vTLv}∣∣∣∣∣​vi​=1​exp{ vTWh+21​vTLv}∣∣∣∣∣​vi​=1​​
令 Δ = exp ⁡ { v T W h + 1 2 v T L v } \Delta=\exp\{v^TWh+\frac12v^TLv\} Δ=exp{ vTWh+21​vTLv}
所以，$$p(v_i=1|h,v_{-i})=\frac{{\Delta }_{v_i=1}}{{\Delta }_{v_i=0}+{\Delta }_{v_i=1}}$$

Δ v i = exp ⁡ { v T W h + 1 2 v T L v } = exp ⁡ { ∑ i ^ = 1 D ∑ j = 1 P v i ^ w i ^ j h j + 1 2 ∑ i ^ = 1 D ∑ k = 1 D v i ^ l i ^ k v k } = exp ⁡ { ∑ i ^ = 1 / i D ∑ j = 1 P v i ^ w i ^ j h j + ∑ j = 1 P v i w i j h j + 1 2 ( ∑ i ^ = 1 / i D ∑ k = 1 / i D v i ^ l i ^ k v k + ∑ i ^ = 1 / i D v i ^ l i ^ i v i + ∑ k = 1 / i D v i l i k v k ) } = exp ⁡ { ∑ i ^ = 1 / i D ∑ j = 1 P v i ^ w i ^ j h j + ∑ j = 1 P v i w i j h j + 1 2 ( ∑ i ^ = 1 / i D ∑ k = 1 / i D v i ^ l i ^ k v k + 2 ∑ k = 1 / i D v i l i k v k ) } = exp ⁡ { v i ( ∑ j = 1 P w i j h j + ∑ k = 1 / i D l i k v k ) + ∑ i ^ = 1 / i D ∑ j = 1 P v i ^ w i ^ j h j + 1 2 ∑ i ^ = 1 / i D ∑ k = 1 / i D v i ^ l i ^ k v k } \Delta_{v_i}=\exp\{v^TWh+\frac12v^TLv\}\\=\exp\{\sum_{\hat i=1}^D\sum_{j=1}^Pv_{\hat i}w_{\hat ij}h_j+\frac12\sum_{\hat i=1}^D\sum_{k=1}^Dv_{\hat i}l_{\hat ik}v_k\}\\=\exp\{\sum_{\hat i=1/i}^D\sum_{j=1}^Pv_{\hat i}w_{\hat ij}h_j+\sum_{j=1}^Pv_{i}w_{ij}h_j+\frac12\Big(\sum_{\hat i=1/i}^D\sum_{k=1/i}^Dv_{\hat i}l_{\hat ik}v_k+\sum_{\hat i=1/i}^Dv_{\hat i}l_{\hat ii}v_i+\sum_{k=1/i}^Dv_{i}l_{ik}v_k\Big)\}\\=\exp\{\sum_{\hat i=1/i}^D\sum_{j=1}^Pv_{\hat i}w_{\hat ij}h_j+\sum_{j=1}^Pv_{i}w_{ij}h_j+\frac12\Big(\sum_{\hat i=1/i}^D\sum_{k=1/i}^Dv_{\hat i}l_{\hat ik}v_k+2\sum_{k=1/i}^Dv_{i}l_{ik}v_k\Big)\}\\=\exp\{v_{i}\Big(\sum_{j=1}^Pw_{ij}h_j+\sum_{k=1/i}^Dl_{ik}v_k\Big)+\sum_{\hat i=1/i}^D\sum_{j=1}^Pv_{\hat i}w_{\hat ij}h_j+\frac12\sum_{\hat i=1/i}^D\sum_{k=1/i}^Dv_{\hat i}l_{\hat ik}v_k\} Δvi​​=exp{ vTWh+21​vTLv}=exp{ i^=1∑D​j=1∑P​vi^​wi^j​hj​+21​i^=1∑D​k=1∑D​vi^​li^k​vk​}=exp{ i^=1/i∑D​j=1∑P​vi^​wi^j​hj​+j=1∑P​vi​wij​hj​+21​(i^=1/i∑D​k=1/i∑D​vi^​li^k​vk​+i^=1/i∑D​vi^​li^i​vi​+k=1/i∑D​vi​lik​vk​)}=exp{ i^=1/i∑D​j=1∑P​vi^​wi^j​hj​+j=1∑P​vi​wij​hj​+21​(i^=1/i∑D​k=1/i∑D​vi^​li^k​vk​+2k=1/i∑D​vi​lik​vk​)}=exp{ vi​(j=1∑P​wij​hj​+k=1/i∑D​lik​vk​)+i^=1/i∑D​j=1∑P​vi^​wi^j​hj​+21​i^=1/i∑D​k=1/i∑D​vi^​li^k​vk​}
不难看出只有第一项与$v_i$有关，所以，
Δ v i = 0 = exp ⁡ { ∑ i ^ = 1 / i D ∑ j = 1 P v i ^ w i ^ j h j + 1 2 ∑ i ^ = 1 / i D ∑ k = 1 / i D v i ^ l i ^ k v k } \Delta_{v_i=0}=\exp\{\sum_{\hat i=1/i}^D\sum_{j=1}^Pv_{\hat i}w_{\hat ij}h_j+\frac12\sum_{\hat i=1/i}^D\sum_{k=1/i}^Dv_{\hat i}l_{\hat ik}v_k\} Δvi​=0​=exp{ i^=1/i∑D​j=1∑P​vi^​wi^j​hj​+21​i^=1/i∑D​k=1/i∑D​vi^​li^k​vk​}
Δ v i = 1 = exp ⁡ { ∑ j = 1 P w i j h j + ∑ k = 1 / i D l i k v k + ∑ i ^ = 1 / i D ∑ j = 1 P v i ^ w i ^ j h j + 1 2 ∑ i ^ = 1 / i D ∑ k = 1 / i D v i ^ l i ^ k v k } \Delta_{v_i=1}=\exp\{\sum_{j=1}^Pw_{ij}h_j+\sum_{k=1/i}^Dl_{ik}v_k+\sum_{\hat i=1/i}^D\sum_{j=1}^Pv_{\hat i}w_{\hat ij}h_j+\frac12\sum_{\hat i=1/i}^D\sum_{k=1/i}^Dv_{\hat i}l_{\hat ik}v_k\} Δvi​=1​=exp{ j=1∑P​wij​hj​+k=1/i∑D​lik​vk​+i^=1/i∑D​j=1∑P​vi^​wi^j​hj​+21​i^=1/i∑D​k=1/i∑D​vi^​li^k​vk​}
所以，
p ( v i = 1 ∣ h , v − i ) = Δ v i = 1 Δ v i = 0 + Δ v i = 1 = exp ⁡ { ∑ j = 1 P w i j h j + ∑ k = 1 / i D l i k v k + ∑ i ^ = 1 / i D ∑ j = 1 P v i ^ w i ^ j h j + 1 2 ∑ i ^ = 1 / i D ∑ k = 1 / i D v i ^ l i ^ k v k } exp ⁡ { ∑ i ^ = 1 / i D ∑ j = 1 P v i ^ w i ^ j h j + 1 2 ∑ i ^ = 1 / i D ∑ k = 1 / i D v i ^ l i ^ k v k } + exp ⁡ { ∑ j = 1 P w i j h j + ∑ k = 1 / i D l i k v k + ∑ i ^ = 1 / i D ∑ j = 1 P v i ^ w i ^ j h j + 1 2 ∑ i ^ = 1 / i D ∑ k = 1 / i D v i ^ l i ^ k v k } = exp ⁡ { ∑ j = 1 P w i j h j + ∑ k = 1 / i D l i k v k } 1 + exp ⁡ { ∑ j = 1 P w i j h j + ∑ k = 1 / i D l i k v k } = 1 1 + exp ⁡ { ∑ j = 1 P w i j h j + ∑ k = 1 / i D l i k v k } − 1 = σ ( ∑ j = 1 P w i j h j + ∑ k = 1 / i D L i k v k ) p(v_i=1|h,v_{-i})=\frac{\Delta_{v_i=1}}{\Delta_{v_i=0}+\Delta_{v_i=1}}\\=\frac{\exp\{\sum_{j=1}^Pw_{ij}h_j+\sum_{k=1/i}^Dl_{ik}v_k+\sum_{\hat i=1/i}^D\sum_{j=1}^Pv_{\hat i}w_{\hat ij}h_j+\frac12\sum_{\hat i=1/i}^D\sum_{k=1/i}^Dv_{\hat i}l_{\hat ik}v_k\}}{\exp\{\sum_{\hat i=1/i}^D\sum_{j=1}^Pv_{\hat i}w_{\hat ij}h_j+\frac12\sum_{\hat i=1/i}^D\sum_{k=1/i}^Dv_{\hat i}l_{\hat ik}v_k\}+\exp\{\sum_{j=1}^Pw_{ij}h_j+\sum_{k=1/i}^Dl_{ik}v_k+\sum_{\hat i=1/i}^D\sum_{j=1}^Pv_{\hat i}w_{\hat ij}h_j+\frac12\sum_{\hat i=1/i}^D\sum_{k=1/i}^Dv_{\hat i}l_{\hat ik}v_k\}}\\=\frac{\exp\{\sum_{j=1}^Pw_{ij}h_j+\sum_{k=1/i}^Dl_{ik}v_k\}}{1+\exp\{\sum_{j=1}^Pw_{ij}h_j+\sum_{k=1/i}^Dl_{ik}v_k\}}\\=\frac1{1+\exp\{\sum_{j=1}^Pw_{ij}h_j+\sum_{k=1/i}^Dl_{ik}v_k\}^{-1}}\\=\sigma(\sum_{j=1}^Pw_{ij}h_j+\sum_{k=1/i}^DL_{ik}v_k) p(vi​=1∣h,v−i​)=Δvi​=0​+Δvi​=1​Δvi​=1​​=exp{ ∑i^=1/iD​∑j=1P​vi^​wi^j​hj​+21​∑i^=1/iD​∑k=1/iD​vi^​li^k​vk​}+exp{ ∑j=1P​wij​hj​+∑k=1/iD​lik​vk​+∑i^=1/iD​∑j=1P​vi^​wi^j​hj​+21​∑i^=1/iD​∑k=1/iD​vi^​li^k​vk​}exp{ ∑j=1P​wij​hj​+∑k=1/iD​lik​vk​+∑i^=1/iD​∑j=1P​vi^​wi^j​hj​+21​∑i^=1/iD​∑k=1/iD​vi^​li^k​vk​}​=1+exp{ ∑j=1P​wij​hj​+∑k=1/iD​lik​vk​}exp{ ∑j=1P​wij​hj​+∑k=1/iD​lik​vk​}​=1+exp{ ∑j=1P​wij​hj​+∑k=1/iD​lik​vk​}−11​=σ(j=1∑P​wij​hj​+k=1/i∑D​Lik​vk​)

同理可得，

$$p(h_i=1|v,h_{-i})=\sigma (\sum _{j=1}^D{w}_{ij}{v}_j+\sum _{m=1/i}^P{J}_{im}{h}_m)$$

五、基于平均场理论的变分推断

$$L=ElBO=\log p_{\theta }(v)-KL(q_{\varphi }||p_{\theta })=\sum _h{q}_{\varphi }(h|v)\log p_{\theta }(v,h)+H[q]$$
q ϕ ( h ∣ v ) = ∏ j = 1 P q ϕ ( h j ∣ v )                      q ϕ ( h j = 1 ∣ v ) = ϕ j ,                ϕ = { ϕ j } j = 1 P q_\phi(h|v)=\prod_{j=1}^Pq_\phi(h_j|v)\;\;\;\;\;\;\;\;\;\;q_\phi(h_j=1|v)=\phi_j,\;\;\;\;\;\;\;\phi=\{\phi_j\}_{j=1}^P qϕ​(h∣v)=j=1∏P​qϕ​(hj​∣v)qϕ​(hj​=1∣v)=ϕj​,ϕ={ ϕj​}j=1P​
$$\begin{gathered} {\hat{\varphi }}_j=\underset{{\varphi }_j}{\mathrm{argmax}}L \\ =\underset{{\varphi }_j}{\mathrm{argmax}}\sum _h{q}_{\varphi }(h|v)[-\log Z+v^{T}Wh+\frac{1}{2}{v}^{T}Lv+\frac{1}{2}{h}^{T}Jh]+H[q] \\ =\underset{{\varphi }_j}{\mathrm{argmax}}\sum _h{q}_{\varphi }(h|v)[-\log Z+\frac{1}{2}{v}^{T}Lv]+\sum _h{q}_{\varphi }(h|v)[v^{T}Wh+\frac{1}{2}{h}^{T}Jh]+H[q] \\ =\underset{{\varphi }_j}{\mathrm{argmax}}\sum _h{q}_{\varphi }(h|v)[v^{T}Wh+\frac{1}{2}{h}^{T}Jh]+H[q] \\ =\underset{{\varphi }_j}{\mathrm{argmax}}\underset{①}{\underbrace{\sum _h{q}_{\varphi }(h|v)\cdot v^{T}Wh}}+\underset{②}{\underbrace{\frac{1}{2}\sum _h{q}_{\varphi }(h|v)\cdot h^{T}Jh}}+\underset{③}{\underbrace{H[q]}} \end{gathered}$$

$$\begin{gathered} ①=\sum _h{q}_{\varphi }(h|v)\cdot \sum _{i=1}^D\sum _{j=1}^P{v}_i{w}_{ij}{h}_j \\ =\sum _h\prod _{\hat{j}=1}^P{q}_{\varphi }(h_{\hat{j}}|v)\cdot \sum _{i=1}^D\sum _{j=1}^P{v}_i{w}_{ij}{h}_j \end{gathered}$$
因为，$$\begin{gathered} \sum _h\prod _{\hat{j}=1}^P{q}_{\varphi }(h_{\hat{j}}|v)\cdot v_1{w}_{12}{h}_2=\sum _{h_2}{q}_{\varphi }(h_2|v)\cdot v_1{w}_{12}{h}_2\cdot \sum _{h/h_2}\prod _{\hat{j}=1/2}^P{q}_{\varphi }(h_{\hat{j}}|v) \\ =\sum _{h_2}{q}_{\varphi }(h_2|v)\cdot v_1{w}_{12}{h}_2 \\ =q_{\varphi }(h_2=1|v)\cdot v_1{w}_{12} \\ ={\varphi }_2{v}_1{w}_{12} \end{gathered}$$
所以，$$①=\sum _{i=1}^D\sum _{\hat{j}=1}^P{\varphi }_{\hat{j}}{v}_i{w}_{i\hat{j}}$$
同理，$$②=\sum _{\hat{j}=1}^P\sum _{m=1/j}^P{\varphi }_{\hat{j}}{\varphi }_m{J}_{\hat{j}m}$$
$$③=-\sum _{j=1}^P[{\varphi }_j\log {\varphi }_j+(1-{\varphi }_j)\log (1-{\varphi }_j)]$$

分别对①、②、③求偏导，

$$\frac{\partial ①}{\partial {\varphi }_j}=\sum _{i=1}^P{v}_i{w}_{ij}$$
$$\frac{\partial ②}{\partial {\varphi }_j}=\sum _{m=1/j}^P{\varphi }_m{J}_{jm}$$
$$\frac{\partial ③}{\partial {\varphi }_j}=-\log \frac{{\varphi }_j}{1-{\varphi }_j}$$

所以，

令，$$\frac{\partial [①+②+③]}{\partial {\varphi }_j}=0$$
得，$${\varphi }_j=\sigma (\sum _{i=1}^D{v}_i{w}_{ij}+\sum _{m=1/j}^P{\varphi }_m{J}_{jm})$$
不动点方程，解法（坐标上升）
ϕ ^ = { ϕ ^ j } j = 1 P \hat\phi=\{\hat\phi_j\}^P_{j=1} ϕ^​={ ϕ^​j​}j=1P​

RBM：白板推导系列笔记（二十一）-受限玻尔兹曼机

下一章传送门：白板推导系列笔记（二十九）-深度玻尔兹曼机