Development of enterprise-level personalized recommendation system based on Spark2.x

Develop enterprise-level personalized recommendation system based on Spark2.x, focusing on algorithm principles
In the big data industry, in addition to understanding platform development, what kind of hard skills are needed? The answer is the algorithm principle. Knowing the principles of algorithms is more competitive than only developing platforms. At present, talents familiar with recommendation algorithms are scarce. Talents who understand the principles of recommendation algorithms and how to implement them using big data technology will be very popular. On the other hand, personalized recommendation has become the standard of Internet products. In the big data industry, the recommendation system is mainly used in e-commerce, information content (video or text) platforms (such as today's headline), this course is personalized The recommended algorithm case class focuses on improving the hard power of big data engineers from the perspective of algorithm principles. Combining with the latest version of Spark 2.x, we will teach you how to land the algorithm and take you from 0 to 1 to build a complete personalized recommendation system.

 

Chapter 1 Course Introduction and Study Guide

This section mainly introduces the course, learning route and guide, how to study this course better? Why study this course, what can you gain by studying this course?

Chapter 2 Understanding the ecology of the recommendation system
This chapter takes you to understand the ecology of the recommendation system, allowing you to reshape your perception of the recommendation system from your thinking. Understand which key elements the recommendation system is supported by, the classification of recommendation algorithms and what constitutes a good recommendation system

Chapter 3 Lay the foundation for the learning algorithm
This chapter reviews and combs the mathematical and statistical knowledge necessary for the learning algorithm, helps you consolidate the foundation, smooth the transition, and pave the way for the later learning and recommendation algorithm.

Chapter 4
explains the principle of collaborative filtering recommendation algorithm in detail. This chapter introduces the most commonly used and most popular collaborative filtering recommendation algorithm among recommendation algorithms. First of all, it is necessary to consolidate the unique mathematical foundation of collaborative filtering, and then start from three types of recommendation algorithms: user-based, item-based, and model-based, and perform code demonstration on them.

Chapter 5 Collaborative Filtering Principles Based on Spark
This chapter explains the recommended algorithm built into Spark: ALS. The ALS algorithm is fully explained from the three aspects of algorithm principle, implementation on Spark, and reading of source code.

Chapter 6 Recommendation System Construction-Requirements analysis and environment construction
began to practice the recommendation system! Are you ready? In this chapter, we conduct a needs analysis of the entire recommendation system. And lead the environment building hand in hand.

Chapter 7 recommends system construction-UI interface modules
start with simple content. General big data development engineers are mainly responsible for data collection and analysis. Here, for demonstration purposes, we have made simple front-end pages, using VUE, Element-UI EChatrs

Chapter 8 recommends system construction-the data layer is
well prepared, and finally it is on the right track. Is everyone unable to bear it? This chapter will lead you to develop the data layer part of the project, respectively to achieve data collection, cleaning, analysis and other functions.

Chapter 9 Recommendation System Construction-Recommendation Engine
This chapter will introduce the major and difficult points of this project, the construction of the recommendation engine module. Mainly explain the core of the recommendation module: recall, filtering, feature calculation and sorting. Gradually complete the construction of the real-time recommendation architecture.

Chapter 10 Recommendation System Construction-Storage of Recommendation Results
This chapter demonstrates the construction of an evaluation module for a personalized recommendation system. It mainly introduces the mainstream test module A / B test, and gradually develops and builds a complete A / B test background

Chapter 11 Recommendation System Construction—Recommendation Evaluation Module
This chapter demonstrates the end of the personalized recommendation system and the construction of the evaluation module. Mainly introduce the mainstream test module A / BTest, and gradually build a complete A / B test background

Chapter 12 Knowledge Development-Recommendation Algorithm Based on Association Rules
This chapter explains the two main association rule recommendation algorithms, Apriori and FP-Growth, and demonstrates the implementation of these two algorithms through Spark.

Chapter 13 Knowledge Development-Recommendation Algorithm Based on Machine Learning
This chapter mainly explains the mainstream recommendation algorithm based on machine learning. First introduce the principle of RBM random network, then show the recommendation algorithm based on RBN, CNN and RNN respectively, and demonstrate how to implement it.

Chapter 14 Knowledge Development-Content-based Recommendation Algorithm
This chapter mainly introduces the mainstream content-based recommendation algorithm, and introduces the TF-IDF algorithm, text vectorization, user behavior vectorization, and long-term model. Finally, a summary and outlook for all algorithm knowledge and course items.

Guess you like

Origin www.cnblogs.com/maomaozag/p/12693857.html