Preface
The FM model is actually an upgraded version of the LR model, and on the basis of it, a binary feature cross is added to solve the problem of LR's assumption of feature independence.
The basic expression of the FM model is:
xi is the unified feature representation of all sample data after onehot encoding. Among them, w0, wi, and wij are all model parameters
From the perspective of the formula, the first half of the model is an ordinary LR linear combination, and the cross term in the second half is the combination of features. The number of combined features is n*(n-1)/2. Due to the sparsity feature of onehot encoding, when n reaches 1000w level, the corresponding model parameter amount is tens of billions; at the same time, due to sparsity, there are too few non-zero samples of xi and xj corresponding to each wij, which may easily lead to insufficient training.
Training Problems with Quadratic Parameters
Then the expression of the FM model is changed to:
The calculation is further simplified as follows (picture source recommendation algorithm (1) - FM factorization (principle + code) - Zhihu (zhihu.com) ):
After simplification, the complexity of FM is reduced from O(kn*n) to O(kn)
Comparison of advantages and disadvantages:
Model code reference: rec_sys: Summarizes recommended related knowledge codes in daily work (gitee.com)