文章目录
Matrix Factorization
SVD
默认超参：
n_factors=100, n_epochs=20, biased=True, init_mean=0,
init_std_dev=.1, lr_all=.005,
reg_all=.02,
参数： b u , b i , p u , q i b_u, b_i, p_u, q_i bu,bi,pu,qi
# user biases
cdef np.ndarray[np.double_t] bu
# item biases
cdef np.ndarray[np.double_t] bi
# user factors
cdef np.ndarray[np.double_t, ndim=2] pu
# item factors
cdef np.ndarray[np.double_t, ndim=2] qi
参数初始化
bu = np.zeros(trainset.n_users, np.double)
bi = np.zeros(trainset.n_items, np.double)
pu = rng.normal(self.init_mean, self.init_std_dev,
(trainset.n_users, self.n_factors))
qi = rng.normal(self.init_mean, self.init_std_dev,
(trainset.n_items, self.n_factors))
trainset对象可以计算全局平均值
cdef double global_mean = self.trainset.global_mean
error计算
# compute current error
dot = 0 # <q_i, p_u>
for f in range(self.n_factors):
dot += qi[i, f] * pu[u, f]
err = r - (global_mean + bu[u] + bi[i] + dot
参数更新
# update biases
if self.biased:
bu[u] += lr_bu * (err - reg_bu * bu[u])
bi[i] += lr_bi * (err - reg_bi * bi[i])
# update factors
for f in range(self.n_factors):
puf = pu[u, f]
qif = qi[i, f]
pu[u, f] += lr_pu * (err * qif - reg_pu * puf)
qi[i, f] += lr_qi * (err * puf - reg_qi * qif)
rating的预测值
r ^ u i = μ + b u + b i + q j T p u \hat{r}_{u i}=\mu+b_{u}+b_{i}+q_{j}^{T} p_{u} r^ui=μ+bu+bi+qjTpu
带正则化的损失函数
∑ r u i ∈ R train ( r u i − r ^ u i ) 2 + λ ( b i 2 + b u 2 + ∥ q i ∥ 2 + ∥ p u ∥ 2 ) \sum_{r_{u i} \in R_{\text {train }}}\left(r_{u i}-\hat{r}_{u i}\right)^{2}+\lambda\left(b_{i}^{2}+b_{u}^{2}+\left\|q_{i}\right\|^{2}+\left\|p_{u}\right\|^{2}\right) ∑rui∈Rtrain (rui−r^ui)2+λ(bi2+bu2+∥qi∥2+∥pu∥2)
梯度下降
梯度下降的公式很好想明白，因为减去了 r ^ \hat{r} r^，再加上函数求导规则，MSE损失 e 2 e^2 e2的导数就是 2 e ⋅ d e d w 2e\cdot \frac{de}{dw} 2e⋅dwde 。 − b i -b_{i} −bi求导就剩下一个负号了，留下来的就是 − 2 e -2e −2e， p u ⋅ q i p_u\cdot q_i pu⋅qi求导还能留一个（对方）。
e u i = r u i − r ^ u i e_{u i}=r_{u i}-\hat{r}_{u i} eui=rui−r^ui
b u ← b u + γ ( e u i − λ b u ) b i ← b i + γ ( e u i − λ b i ) p u ← p u + γ ( e u i ⋅ q i − λ p u ) q i ← q i + γ ( e u i ⋅ p u − λ q i ) \begin{aligned} b_{u} & \leftarrow b_{u}+\gamma\left(e_{u i}-\lambda b_{u}\right) \\ b_{i} & \leftarrow b_{i}+\gamma\left(e_{u i}-\lambda b_{i}\right) \\ p_{u} & \leftarrow p_{u}+\gamma\left(e_{u i} \cdot q_{i}-\lambda p_{u}\right) \\ q_{i} & \leftarrow q_{i}+\gamma\left(e_{u i} \cdot p_{u}-\lambda q_{i}\right) \end{aligned} bubipuqi←bu+γ(eui−λbu)←bi+γ(eui−λbi)←pu+γ(eui⋅qi−λpu)←qi+γ(eui⋅pu−λqi)
SVD++
I ( u ) I(u) I(u)为该用户所评价过的所有电影的集合， y j y_j yj为隐藏的“评价了电影 j j j”反映出的个人喜好偏置。
收缩因子取集合大小的根号是一个经验公式，并没有理论依据。
r ^ u i = μ + b u + b i + q i T ( p u + ∣ I u ∣ − 1 2 ∑ j ∈ I u y j ) \hat{r}_{u i}=\mu+b_{u}+b_{i}+q_{i}^{T}\left(p_{u}+\left|I_{u}\right|^{-\frac{1}{2}} \sum_{j \in I_{u}} y_{j}\right) r^ui=μ+bu+bi+qiT(pu+∣Iu∣−21∑j∈Iuyj)
NMF
The optimization procedure is a (regularized) stochastic gradient descent with a specific choice of step size that ensures non-negativity of factors, provided that their initial values are also positive.
p u f ← p u f ⋅ ∑ i ∈ I u q i f ⋅ r u i ∑ i ∈ I u q i f ⋅ r ^ u i + λ u ∣ I u ∣ p u f p_{u f} \leftarrow p_{u f} \cdot \frac{\sum_{i \in I_{u}} q_{i f} \cdot r_{u i}}{\sum_{i \in I_{u}} q_{i f} \cdot \hat{r}_{u i}+\lambda_{u}\left|I_{u}\right| p_{u f}} puf←puf⋅∑i∈Iuqif⋅r^ui+λu∣Iu∣puf∑i∈Iuqif⋅rui
q i f ← q i f ⋅ ∑ u ∈ U i p u f ⋅ r u i ∑ u ∈ U i p u f ⋅ r ^ u i + λ i ∣ U i ∣ q i f q_{i f} \leftarrow q_{i f} \cdot \frac{\sum_{u \in U_{i}} p_{u f} \cdot r_{u i}}{\sum_{u \in U_{i}} p_{u f} \cdot \hat{r}_{u i}+\lambda_{i}\left|U_{i}\right| q_{i f}} qif←qif⋅∑u∈Uipuf⋅r^ui+λi∣Ui∣qif∑u∈Uipuf⋅rui
r ^ u i = μ + b u + b i + q j T p u \hat{r}_{u i}=\mu+b_{u}+b_{i}+q_{j}^{T} p_{u} r^ui=μ+bu+bi+qjTpu