Automatic Checklist Detailed Tutorial 〖Python Edition〗

foreword

Hello everyone, today we will share with you a very practical thing. Recently, a fan messaged me privately, asking if there is a program that can automatically check the list of the big study. As a fan, as the secretary of the class, she feels a special headache every time she checks the list of the big study. Then I will write a small program that can automatically check the large learning list.

environment use

• python 3.9
• pycharm

module use

• requests

module introduction

• requests

        requests is a very practical Python HTTP client library. It is often used when crawlers and test servers respond to data. requests is a third-party library in Python language, which is specially used to send HTTP requests. It is much simpler to use than urllib.

• parcel

        parsel is a python third-party library, which is equivalent to css selector + xpath + re.

Parsel is developed by the scrapy team. It extracts the parsel in scrapy independently. It can easily parse html and xml content and obtain the required data.

Compared with BeautifulSoup, xpath and parser are more efficient and easier to use.

• re

        The re module is python's unique module for matching strings. Many functions provided in this module are implemented based on regular expressions, and regular expressions perform fuzzy matching on strings and extract the string parts you need. All languages ​​are common.

• os

        os is the abbreviation of "operating system". As the name suggests, the os module provides interfaces for various Python programs to interact with the operating system. By using the os module, on the one hand, it can easily interact with the operating system, and on the other hand, it can greatly enhance the portability of the code.

• csv

        It is a file format, commonly known as a comma-separated value file, that can be opened with Excel software or a text document. The data fields are separated by half-width commas (other characters can also be used), and when opened with Excel, commas will be converted into separators. The csv file stores tabular data in plain text and is compatible with various operating systems.

Module installation issues:

• If installing python third-party modules:

win + R, enter cmd and click OK, enter the installation command pip install module name (pip install requests) and press Enter

Click Terminal (terminal) in pycharm to enter the installation command

• Reason for installation failure:

• Fail one: pip is not an internal command

                Solution: set environment variable

• Failure 2: There are a lot of red reports (read time out)

                Solution: Because the network link times out, you need to switch the mirror source

   

    清华：https://pypi.tuna.tsinghua.edu.cn/simple
    阿里云：https://mirrors.aliyun.com/pypi/simple/
    中国科技大学 https://pypi.mirrors.ustc.edu.cn/simple/
    华中理工大学：https://pypi.hustunique.com/
    山东理工大学：https://pypi.sdutlinux.org/
    豆瓣：https://pypi.douban.com/simple/
    例如：pip3 install -i https://pypi.doubanio.com/simple/ 模块名

• Failure three: cmd shows that it has been installed, or the installation is successful, but it still cannot be imported in pycharm

                Solution: There may be multiple python versions installed (anaconda or python can only install one), just uninstall one, or the python interpreter in your pycharm is not set properly.

thoughts and ideas

Programming:

        We can write a program to help us check the qn university study list. This program can automatically check whether everyone in the list has completed the study task according to the study list we input. We can use a programming language such as Python to write this program.

        However, we found that this method is too troublesome. What is the difference between this and mine one by one. I thought it would be good to just build the list directly. save time and energy.

function realization

First of all, we crawled the list of class studies, and then compared it with the list of all students in the class. The program automatically outputs the list of unfinished youth studies. How to operate, see my step-by-step operation.

Get the maximum number of study periods

We get the number of sessions of the big study here, so we don't need to modify the value of table_name. The default here is the latest big study.

url = 'http://dxx.ahyouth.org.cn/api/peopleRankList?level1=%E7%9B%B4%E5%B1%9E%E9%AB%98%E6%A0%A1'
url_res = requests.get(url)
name = url_res.json()['list'][0]['name']
table_name = url_res.json()['list'][0]['table_name']
print("你正在查询",name,"青年大学习\n")

Run our program, today is the 10th period of big learning, explain, our program is no problem, we continue to go down.

 You are inquiring about 2023 - Term 10 Big Study

get list

Next, is the most important step, constructing the address below. Find the address of the list of big learning in your class. If you can use the developer tools, you can directly find the address below. It doesn’t matter if you don’t know how to use it. We can directly modify the value behind the level below.

url = 'http://dxx.ahyouth.org.cn/api/peopleRankStage'
data = {
    'table_name': table_name,
    'level1': '直属高校',
    'level2': '某某大学',
    'level3': '数理学院',
    'level4': '数学类2101',
}
res = requests.get(url, params=data)

level1 is a directly affiliated university, we don't need to modify it here.

level2 is the name of your school, just fill in the name of your own school.

​

level3 is the name of your college, just fill in the name of your own college. Here we take Anqing Normal University as an example. Everyone needs to pay attention to that the name should be exactly the same as the name we see. Especially the class name. Later, I won't go into too much detail.

​

level4 is the name of your class, just fill in your own class name.

​

Let's take "Mathematics 2101" as an example to explain the following code. Here we call  the method requests in the library  get() , pass in the URL and  data dictionary, and get the result returned by the server. The result is a dictionary containing the results of the query, which is the learned list.

We found that returning res.text will be garbled, we can transcode it, or solve it directly with json.

html_lists = res.json()['list']['list']

This code will  res.json() return a list containing multiple dictionaries, each dictionary representing a user's information. ['list']['list'] Represents the list corresponding to the first key in the dictionary, that is, the user's name and learning time. 

Let's see the effect:

​List Comparison

Next, and our most important part, is comparing our lists.

Let's take a look at the following program:

for html_list in html_lists:
    username = html_list['username']
    yixue_list.append(username)

We next loop through  html_lists each dictionary in the list, get  username the key in it, and add it to  yixue_list the list. This way, after the loop is over, yixue_list the list contains all the names of the class that have learned the big learning.

weixue_list = list(set(list_all) - set(yixue_list))
if not weixue_list:
    print("全部完成青年大学习")
else:
    print("未完成的名单如下\n", weixue_list)

Our code uses Python's set and list operations.

First, it  set removes duplicate names by converting all users' names to a collection. Then, it  converts the names in set the set  yixue_list to sets, and then  set converts the difference of the two sets into a new set.

Next, it checks to see if the new collection is empty, if it is empty, all users have completed the big study, and the program outputs a message. Otherwise, it outputs the list of classmates who did not complete the junior college study, and prints it out.

Here we put the names of all the students in our class into a list_alllist. Here, our function is realized.

list_all = ['张三', '李四', '王二麻子']

Effect

At this point, our program has been realized. Looking back, I will encapsulate the program, and everyone can use it directly.

​

 Let's package the py program into an exe file, let's see the running effect.

 full code

Below I put the complete code below. If you have any questions, you can leave a message in the comment area.

import requests

yixue_list = []
list_all = ['张三', '李四', '王二麻子']# 班级同学名单

url = 'http://dxx.ahyouth.org.cn/api/peopleRankList?level1=%E7%9B%B4%E5%B1%9E%E9%AB%98%E6%A0%A1'
url_res = requests.get(url)
name = url_res.json()['list'][0]['name']
table_name = url_res.json()['list'][0]['table_name']
print("你正在查询", name, "青年大学习\n")

url = 'http://dxx.ahyouth.org.cn/api/peopleRankStage'
data = {
    'table_name': table_name,
    'level1': '直属高校',
    'level2': '学校名字',
    'level3': '学院名字',
    'level4': '班级名字',
}

res = requests.get(url, params=data)

html_lists = res.json()['list']['list']

print(html_lists)
for html_list in html_lists:
    username = html_list['username']
    yixue_list.append(username)
weixue_list = list(set(list_all) - set(yixue_list))
if not weixue_list:
    print("全部完成青年大学习")
else:
    print(len(weixue_list))
    print("未完成的名单如下\n", weixue_list)