Author: Zen and the Art of Computer Programming
1 Introduction
LightFM
LightFM is an open source recommendation system framework developed by Yelp, which can easily implement large-scale matrix factorization. The project is based on TensorFlow and Keras framework, which can handle large matrices quickly and efficiently. It has the following characteristics:
- Provides an easy way to train matrix factorization models by defining the interaction matrix between items and user and item feature vectors to learn factorization and apply it to recommender system tasks.
- Representing interaction data using sparse matrices allows efficient processing of large datasets and reduces memory requirements and computation time.
- The loss function is optimized by the optimizer, and the model effect can be adjusted by different cross-entropy loss functions or proportional imbalance weight loss functions.
In addition, LightFM provides many options to control the parameters of the recommendation model, including learning rate, regularization parameter, hidden component size, etc. These options can be used to control the performance of the model and improve recommendation accuracy and robustness.
Dataset introduction
In this paper, we adopt the Movielens-1M dataset for research. The dataset contains 1,000,209 user rating records for 3,706 movies. The format of the dataset is user_id item_id rating, representing user ID, movie ID, and user's rating for the movie, respectively.
data division
Divide the data set into training set (10%), validation set (10%) and test set (80%), where the training set is used for model training, the validation set is used for parameter selection, and the test set is used for final model evaluation .