Big data actual combat project development of tens of millions novel website (storage, complex search, recommendation, analysis)
With the development of the times, novels have become the purpose of many people for decompression and relaxation. A novel can be seen by people and needs to be edited by many codes and programs, from the storage, popularity, ranking, From classification to search, recommendations based on the novels that users have read in the past, and real-time updates after the author updates chapters, the background of the novel updates the weight and preference of each book. These are actually behind the novel. issues that need resolving.
This course is a project to search, recommend, and analyze the recommendation, popularity, and content of millions of novel data. Let everyone truly master the content of big data storage, search and recommendation in the actual battle of real data.
1. The structure of the novel website is explained 1. The background and objectives of the
project 2. The architectural design of the
project 3. The structure selection of the project
4. The installation of components
Index and storage of tens of millions of novel data
1. Detailed explanation of website data
2. MySQL data migration to es
3. Logstash configuration writing
4. Logstash data migration
5. Novel storage and rowkey design
6. Novel importing javaBean class writing
7. Novel data migration MR writing
8. Data import rowkey code writing
9. MR driver part writing
10. Data migration to hbase
3. Realization of complex search functions of the
website 1. Introduction to the original code of the
website 2. Realization of the website search function
3. Adding the search function to the website
4. Viewing the results of the search page
5.
Explanation of the novel search word segmentation 6. The adding word segmentation function of the mapping word segmenter
7. Code addition IK word segmentation
8. Novel website highlight search realization
9. Pinyin search function realization
10. Pinyin search code modification
11. Obtain novel details page from es
12. Collection self-increment function realizes
13. es optimistic lock
14. Novel chapter data migration
15. Novel chapter query function realization
16. Chapter details page search
17. Website recently updated novel function realization
18. Website works classification aggregation display function
19. Novel monthly and Sunday popularity list
20. Serialization and end of top list
21. Top 10 writer list
22. New works function realization
4. Novel website recommendation engine
1. Overview of recommendation system
2. Introduction to recommendation algorithm
3. Content-based recommendation
4. Recommendation based on collaborative filtering
5. Recommendation based on association rules
5. Website analysis related
1. Novel website user data analysis
2. es cluster planning explanation
3. es cluster monitoring
4. Hbase monitoring
5. Project summary