1. Project introduction
Python commodity review data collection and analysis visualization system
Flask framework, MySQL database, requests crawler, can grab specified product reviews, Echarts visualization, multi-dimensional analysis of reviews, NLP sentiment analysis, LDA topic analysis, Bayes review classification
1. Explanation about the data: The comment data crawled from the flagship store of Xiaomi mobile phone JD.com
2. Bayesian classification algorithm: accuracy rate = 93.49%
2. Screenshot of the project
3. Supplementary instructions
1. NLP sentiment analysis
A value of 1 indicates a positive comment, and a value of 0 indicates a negative comment
2. Sentiment Analysis
The computational study of people's opinions, emotions, sentiments, evaluations, and attitudes toward products, services, organizations, individuals, issues, events, topics, and their attributes. Text sentiment analysis (Sentiment Analysis) is a common application in natural language processing (NLP) methods, and it is also an interesting basic task, especially the classification for the purpose of extracting the emotional content of text. It is the process of analyzing, processing, summarizing and inferring emotionally subjective texts.
This article will introduce sentiment polarity (inclination) analysis in sentiment analysis. The so-called sentiment polarity analysis refers to the positive, derogatory, and neutral judgments on the text. In most application scenarios, there are only two categories. For example, the words "love" and "dislike" belong to different emotional tendencies.
Most commonly used for short text analysis in the context of Weibo, WeChat, user forums, etc.
A large number of text preprocessing technical requirements [web page analysis, text extraction, regular expressions, etc.]
· The specificity of sentiment analysis
・The text length is relatively short
· The context is unique
The amount of information that needs to be extracted is relatively special (negative words and ambiguous words may lead to obvious misjudgments; new words appear quickly; word segmentation is difficult to achieve perfection)
# 1. Analysis based on bag of words model
Data overview:
There are approximately 10,000 positive and negative reviews captured from shopping websites. Covering digital, books, food and many other fields.
Sentiment Analysis Based on Dictionary Sentiment Dictionary
· OK 1 * good 2 · not good - 2 too bad - 3
Negative word processing: forward search for several entries for flipping
· Calculation of Sentiment Score
· Load sentiment dictionary as Dict
Traverse the entries in the sentence and add the corresponding scores
The sum of the scores is used as the sentiment score of the sentence
The average of the sentiment scores of all sentences is used as the sentiment score of the text
Sentiment Analysis Based on Bag of Words Model
Use the gold standard to get the training samples that have been accurately classified
Based on the bag-of-words model, treat sentiment analysis as a simple instance of text classification for processing
Prediction can be made according to classification or sentiment score. It is necessary to balance the accuracy and speed of the model
The effect is better than the dictionary model, but still ignores the relevant information of the context
You can consider using bigram or trigram to extract entries to consider the influence of negative words and degree adverbs at the same time
Can consider using keywords for model fitting
4. Other project learning
| 2023 recruitment data + multiple crawlers + real-time analysis + forecast recommendation | Travel recommendation management system | Flask+resident consumption data visualization |
| 2023 Recruitment Data Analysis Visualization System + Crawler 7 Salary Prediction Models | Python homestay recommendation system | django+Weibo Employment Public Opinion Analysis System |
| 2023 tourist attraction recommendation + hotel recommendation + attraction crawler/hotel crawler (where to go) | Equipment rental and maintenance management system | django+questionnaire system |
| 2023 National Rental Data Analysis + Crawler + Visualization (by province) | Python commodity supply and demand platform | django+drug information+QR code+visualization |
| 2023 movie data crawler + visualization + collaborative filtering recommendation system | Youzhixue Live Teaching Platform | Django+knowledge map computer composition principle course resource management system |
| 2023 news data analysis crawler visualization sentiment analysis | Movie Box Office Data Analysis | Django recruitment information data visualization |
| 2023Flask recruitment data visualization big screen | Django second-hand housing transaction forecast and display system | flask+b station food video+visual analysis |
| 2023 rental data + crawler + recommendation + visualization system | Visualization of epidemic monitoring based on Flask + mysql | Flask+c language knowledge knowledge map question and answer |
| 2023 Simple Edition_Recruitment Data Crawler + Cleaning + Visualization | Douban movie data analysis and visualization | Flask+Python housing price prediction system in Harbin |
| 2023 commodity data crawler + recommendation system (Apriori algorithm) | University Data Visualization_Echart | Scenic spot management system of flask+ big data analysis in Anhui |
| 2023 commodity data crawler + recommendation system + large visual screen | Python crawling recruitment data and visual analysis | Flask + Douban movie visual analysis |
| 2023 Sina Weibo + crawler + topic public opinion analysis system | Flask and crawler realize recruitment position data analysis visualization system | Flask + expressway management information analysis system + forgery |
| 2023 e-commerce data collection + forecasting + visualization system | Django book management system | Flask+Python-based epidemic data analysis |
| 2023 Recruitment Recommendation System + Visualization (Collaborative Filtering Algorithm) | Django employee attendance system | django+ Weibo Xinjiang public opinion information |
| 2023 Commodity Review Data Collection and Analysis Visualization System 1 | personal blog system | Django + vaccination + visual management system |
| 2023 Commodity Review Data Analysis and Visualization System 2 | Steam game data analysis visualization | django+recruitment visualization+resume |
| Flask+LSTM music double recommendation algorithm + crawler + visualization system (deep learning) | National economic and social development data visualization | django+ Chinese medicine recruitment + visualization |
| Flask+LSTM movie double recommendation algorithm + crawler + visualization system (deep learning) | Reptile Epidemic Data Visualization Data Management Epidemic News | flask+boss direct employment+visual analysis |
| Flask+LSTM recruitment double recommendation algorithm + crawler + visualization system (deep learning) | Django-based educational information management system source code | flask+b station data visualization system |
| Web crawler and collaborative filtering algorithm movie recommendation system (vue front end) | deepsort road vehicle tracking + speed real-time analysis system | flask+hadoop+recruitment data visualization |
| Movie recommendation algorithm + crawler + visualization system | Employee face attendance system | Douban movie data visualization analysis of flask+Python crawler |
| Flask+Vue tourist attraction data crawler+large screen real-time monitoring system | Animal recognition system based on Django | flask+ movie box office analysis visualization + prediction |
| Tourism data visualization large screen system based on Flask | License plate recognition system 1 | flask+sights+collaborative filtering |
| Douban movie + crawler + visual analysis | License plate recognition system 2 | flask+logic circuit knowledge map |
| Campus Microblog Hot Topic Discovery System Based on K-means Algorithm | UAV target recognition + target tracking (OpenCV+YOLO) | Design and Implementation of Python Visitor Reservation Registration System |
| News Recommendation System Based on Collaborative Filtering Algorithm | Face recognition OpenCV face detection pyqt5 framework, face management | selenium + customs statistics crawler |
| Fund data analysis crawler fund comparison Django framework data visualization | YOLO目标检测算法 实时目标检测 pytorch | 爬虫+科研项目自动采集 |
| 股票数据可视化+推荐系统（协同过滤推荐算法） | pyqt5+yoloV5目标检测平台 | 苏州旅游网 |
| boss直聘数据分析可视化+爬虫 | OCR车牌识别 YOLOv5 | flask+泰安市天气可视化+预测 |
| 网易云音乐数据爬虫+数据分析可视化系统 | 手势控制鼠标 | flask+物价价格指数 |
| 新闻混合推荐算法+爬虫+可视化（新浪新闻） | 手势控制音量 | flask+携程旅游可视化 |
| 拉钩招聘数据分析+7种预测模型+推荐 | 手势拖放物体 | flask+学习行为分析 |
| 链家租房数据爬虫+可视化 | 基于opencv的人脸识别智能考勤管理系统 | flask+疫情可视化 |
| 全国气象数据爬虫+可视化 | Python人脸识别签到考勤系统（opencv+pyqt5） | flask+影视评论+情感分析 |
| 商品销售数据分析+爬虫+可视化 | 基于深度学习的老照片修复系统 | flask+知识图谱河北省人工智能行业问答 |
| 微博舆情情感分析+爬虫+可视化（新浪） | 人物关系可视化及问答系统 知识图谱 人物关系 | pyqt+填写志愿系统 |
| Flask框架多元爬虫房价舆情系统系统 | 基于卷积神经的车牌识别系统 | Python的招聘信息+django可视化分析系统 |
| Flask舆情监控、舆情管理情感分析系统 | Python+Django实现基于人脸识别的门禁管理系统 | 航空爬虫系统 |
| 全国疫情数据爬虫+可视化大屏 | Python人脸识别签到 | 链家成都二手房推荐可视化_flask |
| 51job招聘爬虫+数据分析（前程无忧） | django+jd手机商品分析系统 | 爬虫二次开发 |
| 微博实时热点数据可视化大屏 | django+Prophet的空气质量分析的设计与实现 | 西藏舆情系统 |
| 微博热搜舆情分析 情感分析可视化系统 | django+python的商品评论情感分析系统 | flask+贴吧+高校可视化 |
| 全国5A级景区景点数据分析与可视化平台 | django+人脸识别考勤系统 | flask+鲜花销售可视化 |
| 实验室药品管理数据分析可视化 | django+Scrapy+瓜子二手车数据采集可视化系统 | flask+学生管理系统 |
| 微博用户情感分析系统 | django+保定市空气质量分析 | flask+义乌小商品可视化 |
| 电商数据爬虫+比价系统（京东）京东爬虫 可视化 | django+电影+知识图谱+推荐 | flask+音乐知识图谱和问答系统 |
| 淘宝数据爬虫与可视化 | django+短视频+标签+推荐 | flask+游戏角色分析系统 |
| 豆瓣电影数据爬虫与可视化 | django+个性化资讯推荐系统 | flask+知识图谱问答系统修改版 |
| 车牌识别+车辆管理+计费系统 | django+海风分类预测 | python+django+智能推荐销售商城 |
| 智能停车场计费管理系统 | django+基于爬虫的公众人物及其粉丝群体的系统 | 大数据分析的网络舆情信息监测平台构建 |
| 商品房数据可视化与房价预测推荐系统 | django+美食推荐系统+协同过滤 | Python爬虫+反腐系统 |
| 购物商城网站、电商网站 （51） | django+情感分析+豆瓣基于网络爬虫的影视系统 | 基于Flask学生成绩管理系统 |
| 基于Python的音乐播放管理系统（大学生） | django+网络爬虫的疫情数据可视化分析 | django社团管理系统_577ux（管理系统） |
| 电影数据分析可视化 | django+mysql+51招聘可视化修改 | django-新生入学管理系统 |
| 汽车评论舆情分析系统 | django+python的jd商品评价采集和协同过滤.. | django公务员考试信息管理系统_o88h3 |
| 景区评论 情感分析 舆情分析系统 | django+python招聘数据分析系统+mangodb | django基于Django的手办交易平台_qu1d0 |
| 疫情数据爬虫分析可视化系统 | django+renlian+kaoqing+keshihua | django基于python的企业it资产管理系统 |
| 淘宝评论情感分析舆情分析系统 | django+scrapy+猫咪网可视化 | django基于python的图书馆管理系统_582a0 |
| Django垃圾邮件分类检测系统 | django+大数据专业基础知识图谱 | django基于大数据的应届生求职系统 |
| 外卖订餐数据的分析与应用系统 | django+豆瓣电影协同过滤推荐+影评分词分类 | django宾馆管理系统 |
| Flask外卖订餐系统 | django+二手车网站数据分析与可视化 | django平南盛世名城小区疫情防控系统 |
| python农业信息数据可视化系统 | django+供热负荷动态系统+预测 | django摄影交流平台 |
| Flask商家订单管理 | django+基于Python的电影推荐系统的设计与系统 | django新生报到服务管理系统 |
| 电商用户行为日志数据分析系统 | django+基于线性回归的小说上架订阅预测 | django旅游信息管理系统 |
| IT类招聘信息数据的分析与应用系统 | django+逻辑电路知识图谱 | django旅游网站设计 |
| 基于协同过滤的淘宝商铺推荐系统 | django+商城+评论 | django智能旅游推荐系统 |
| 豆瓣电影爬虫+清洗+可视化系统 | django+网络小说17K网站数据可视化分析 | django校园用车管理系统 |
| Python招聘岗位信息聚合系统（拥有爬虫爬取、数据分析、可视化、互动等功能） | django+nba+可视化分析 | django水果销售系统 |
| 基于python爬虫的微博疫情舆论情感分析可视化系统 | django+python的jd商品手机分析系统+对比 | django物业信息管理系统_435y5 |
| 基于python爬虫的计算机专业毕业生就业分析可视化分析系统 | django+request多线程+知音漫客分析 | django用Python设计学生成绩管理系统_gpkm2 |
| 基于python爬虫的网易云歌单可视化分析系统 | django+Scrapy+游侠网游戏数据采集可视化系统 | django电影个性化推荐系统 |
| 基于python爬虫的旅游大数据分析可视化系统 | django+电商推荐系统 | django疫情数据可视化分析系统_l748x |
| 基于python的电商用户行为分析管理系统 | django+豆瓣网+书籍推荐 | django疫情防控下医院人员调动系统_1r117 |
| 基于Python的Flask框架开发的在线电影网站系统 | django+肺部疾病数据分析 | django药物管理系统 |
| 基于Python用户画像以及协同过滤的音乐推荐系统 | django+股票+智能投顾系统 | django邯郸地标美食导游平台 |
| 基于Python的音乐推荐系统 | django+基于python的股票筛选的设计与实现 | django食堂外卖系统_oq09k |
| 基于Python的电影推荐系统 | django+家庭用电智能管控系统 | django高校教室管理系统 |
| 基于Python的动漫推荐系统 | django+美食数据+分析系统 | django校园社团管理系统 |
| 基于Python的新闻推荐系统 | django+贫困大学生识别 | django电子病历管理系统 |
| 基于Python的图书推荐系统 | django+商家推荐系统 | 2023招聘推荐+爬虫+可视化（协同过滤） |
| 基于Python的旅游推荐系统 | django+网络小说可视化+预测 | 2023游戏数据采集可视化分析系统 |
| 基于Python的美食推荐系统 | flask+苏州旅游信息的爬取与可视化分析 | 2023股票数据采集分析可视化系统 |
| 基于Python的景点推荐系统 | flask+携程旅游景点热度+可视化展示 | 2023_B站数据采集分析+可视化+算法推荐系统 |
| 基于Python的商品推荐系统 | flask+学生老师信息管理 | 2023智慧交通大数据监控平台 |
| 基于Python的租房推荐系统 | flask+疫情防控数据可视化系统 | 2023双推荐算法+商品推荐系统+爬虫 |
| 基于Python的就业推荐系统 | flask+影评+nlb情感分析 | |
| 基于Python的 XX 推荐系统 （可定制） | flask+知识图谱+银行风控系统 | |
| 基于Python的高考志愿专业推荐高校推系统 | flask+中国电影知识图谱 | |
| 基于Python的图书推荐+可视化系统 | django+微博冬奥会舆情分析系统 | |
| 基于Python的旅游推荐+可视化系统 | django+微博疫情舆论+情感分析+可视化 | |
| 基于Python的新闻推荐+可视化系统 | django+招聘信息+可视化 | |
| 基于Python的汽车推荐+可视化系统 | django-python的购物推荐系统 | |
| 基于Python的电影推荐+可视化系统 | Python的影视数据可视化 票房预测 | |
| 大学生心理评价和分析管理系统 | flask+boss直聘-招聘信息获取与分析 | |
| 电影管理网站 | flask+Python+大屏的中国影业数据分析系统 | |
| 民宿管理网站 | flask+spark的前程无忧互联网+可视化 | |
| 商品库存管理系统 | flask+差旅+可视化分析 | |
| 特色美食管理系统 | flask+电影票房可视化+推荐 | |
| 疫情信息管理系统 | flask+福州房源分析系统 | |
| 音乐播放推荐系统 | flask+杭州租房可视化分析+预测 |