How Python gets a lot of movie reviews and makes visual presentations

foreword

"Keeping You Safe" is released today, has anyone watched it? How is it?

This is a movie I want to watch recently, but I don’t know how the reviews are. There should be a lot of people watching the screenings last weekend. You can collect and collect comments from friends who have seen them, and analyze them.

This week just happens to be on weekends, so I can go and have a look

insert image description here

okok, without further ado, let's start

development environment

Python 3.8
Pycharm

Code

The basic idea

Data source analysis:

Clear requirements:

What is the collected website?
What data is collected?
Review relevant data

Data sources related to packet capture analysis
Use the browser's built-in developer tools to perform packet capture analysis<key>

Open developer tools: F12 or right-click to inspect and select network
Refresh the page: Let the data content of this page be reloaded again
Keyword search: By keyword <required data>, search and query the corresponding data package

Use the acquired data for visual analysis

[Get the business card at the end of the complete source code]

send request

# 请求链接
690643772 ### 源码领取
url = f'https://****/subject/35457272/comments?start=20&limit=20&status=P&sort=new_score'
# 伪装模拟
headers = {
    
    
    # User-Agent 用户代理, 表示浏览器基本身份标识
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.0.0 Safari/537.36'
}
# 发送请求
response = requests.get(url=url, headers=headers)
# <Response [200]>
print(response)

Analytical data

# 把获取下来html字符串数据 <response.text>, 转成可解析对象 <Selector xpath=None data='<html lang="zh-CN" class="ua-windows ...'>
selector = parsel.Selector(response.text)  # ---> 你现金是美元, 没办法在中国使用 <先去银行兑换RMB>
# 第一次提取, 所有div标签
divs = selector.css('div.comment-item')
# for循环遍历, 把列表里面元素一个一个提取出来
for div in divs:
    name = div.css('.comment-info a::text').get()  # 昵称
    rating = div.css('.rating::attr(title)').get()  # 推荐
    date = div.css('.comment-time::attr(title)').get()  # 时间
    area = div.css('.comment-location::text').get()  # 地区
    votes = div.css('.votes::text').get()  # 有用
    short = div.css('.short::text').get().replace('\n', '')  # 评论
    # 数据存字典里面
    dit = {
    
    
        '昵称': name,
        '推荐': rating,
        '时间': date,
        '地区': area,
        '有用': votes,
        '评论': short,
        690643772 ### 源码领取
    }

data input

f = open('保你平安.csv', mode='a', encoding='utf-8-sig', newline='')
csv_writer = csv.DictWriter(f, fieldnames=[
    '昵称',
    '推荐',
    '时间',
    '地区',
    '有用',
    '评论',
])
csv_writer.writeheader()

Please add a picture description

Visualization

read related data

df = pd.read_csv('保你平安.csv')
df.head()

Please add a picture description

Recommended distribution

import pyecharts.options as opts
from pyecharts.charts import Pie

data_pair = [list(z) for z in zip(evaluate_type, evaluate_num)]
data_pair.sort(key=lambda x: x[1])

c = (
    Pie(init_opts=opts.InitOpts(bg_color="#2c343c"))
    .add(
        series_name="豆瓣影评",
        data_pair=data_pair,
        rosetype="radius",
        radius="55%",
        center=["50%", "50%"],
        label_opts=opts.LabelOpts(is_show=False, position="center"),
    )
    .set_global_opts(
        title_opts=opts.TitleOpts(
            title="推荐分布",
            pos_left="center",
            pos_top="20",
            title_textstyle_opts=opts.TextStyleOpts(color="#fff"),
        ),
        legend_opts=opts.LegendOpts(is_show=False),
    )
    .set_series_opts(
        tooltip_opts=opts.TooltipOpts(
            trigger="item", formatter="{a} <br/>{b}: {c} ({d}%)"
        ),
        label_opts=opts.LabelOpts(color="rgba(255, 255, 255, 0.3)"),
    )690643772 ### 源码领取
)
c.render_notebook()

Please add a picture description

Local distribution

import pyecharts.options as opts
from pyecharts.charts import Pie

data_pair = [list(z) for z in zip(area_type, area_num)]
data_pair.sort(key=lambda x: x[1])

d = (
    Pie(init_opts=opts.InitOpts(bg_color="#2c343c"))
    .add(
        series_name="豆瓣影评",
        data_pair=data_pair,
        rosetype="radius",
        radius="55%",
        center=["50%", "50%"],
        label_opts=opts.LabelOpts(is_show=False, position="center"),
    )690643772 ### 源码领取
    .set_global_opts(
        title_opts=opts.TitleOpts(
            title="地区分布",
            pos_left="center",
            pos_top="20",
            title_textstyle_opts=opts.TextStyleOpts(color="#fff"),
        ),
        legend_opts=opts.LegendOpts(is_show=False),
    )
    .set_series_opts(
        tooltip_opts=opts.TooltipOpts(
            trigger="item", formatter="{a} <br/>{b}: {c} ({d}%)"
        ),
        label_opts=opts.LabelOpts(color="rgba(255, 255, 255, 0.3)"),
    )
)
d.render_notebook()

Please add a picture description

After analyzing it like this, it seems to be pretty good. It should be worth a look.

You can have a blast this weekend

at last

This is the end of today’s article sharing, I have to plan what to eat when I go out and watch a movie tomorrow haha

I wish you a happy weekend~

If you have any questions about python, you can click on the business card at the end of the article to learn and communicate.