Analysis of grain production data in various regions of the country based on python
Summary
In recent years, the total grain output and incremental growth rate of various provinces in my country have gradually attracted much attention. How to increase grain output has also become a hot topic of concern.
Through the information publishing website on the Internet, the grain production data of each province was obtained and integrated. Among them, the "China Industry Information Network" publishes the domestic grain production information for the new year every year. The information published on the website includes grain production, proportion, trend, and sown area in recent years. The goal is to use python programming, grab the data of the website, and save the acquired data into an Excel file, then use python to clean and process the data, use python to visualize, combine data processing and analysis, and obtain statistical analysis results of the data.
Key words
Data Analysis ; Python Programming ; Grain Production ; Data Visualization
Table of contents
1.2 Research purpose and significance
3.1 Data acquisition and cleaning
3.2 Data processing and analysis
4.1 Overall solution to the problem:
4.2 Description or technical analysis of the main parts of the proposal:
4.3 Feasibility analysis of the scheme:
4.4 Evaluation of Graduation Design Achievements
1. Introduction
1.1 Research Background
In recent years, with the continuous increase of total grain production and incremental growth across the country, how to increase grain production has become an important issue in today's society.
Using the global information platform "China Industry Information Network", we can collect and summarize the grain production situation in different regions of the country, from the overall growth rate, harvest rate, crop sown area to recent changes in recent years. Check them out one by one. Through the writing of python, our purpose is to collect valuable information from the Internet and convert the information into Excel format for better management and analysis. In addition, we will use the visualization function of python to display this information to better understand their characteristics and trends.
1.2 Research purpose and significance
The research purpose and significance of the analysis of grain production data in various regions of the country based on Python include:
1. Grain is an important material of the country. The grain production in various regions of the country has an important impact on the country's food supply, economic development, and social stability. Therefore, through the analysis of grain production data in various regions of the country, we can better understand and grasp the status and trends of national grain production, and provide support for national food security and economic development.
2. Python-based data analysis methods can effectively process large amounts of data and extract useful information and rules. Through the analysis of grain production data in various regions of the country, it is possible to discover the yield change trends of different regions and different grain varieties, and then provide scientific guidance and decision-making basis for agricultural production.
3. In the process of analyzing grain production data in various regions of the country, a variety of data analysis methods and techniques can be used, including data visualization, regression analysis, time series analysis, etc. These methods and technologies can not only reveal the laws and trends behind the data, but also provide new ideas and methods for the optimization and efficiency improvement of agricultural production.
4. The Python-based data analysis method is flexible and repeatable. Data processing and analysis can be performed as needed, and data analysis can be automated and batched through code reuse and modification. By using advanced technical means, we can greatly improve the accuracy of data analysis, so that future decisions and practices can be strongly supported.
2. Literature review
"Analysis and Application of National Grain Production Data Based on Python" (Yang Jing, Wang Zhaohui): This paper uses tools such as Python and Pandas to analyze and apply national grain production data. Through the visual display and analysis of the data, the yield variation rules of different regions and different grain varieties were discovered, and then corresponding optimization strategies and suggestions were provided for agricultural production.
3. Design concept
3.1 Data acquisition and cleaning
1 Data description
Data source: China Industry Information Network website page
Data acquisition: information from the website of China Industry Information Network (2016 to 2020 grain production in each province) and write the obtained information into an excel form.
2 Using tools
Python has many unique features, including a huge, user-definable base class library that can be used by different programmers. In this report, the libraries used are:
① matplotlib library
② requests library
③ pandas library
④ Beautiful Soup library
⑤ pyecharts library
⑥ bs4 library
⑦ csv library
In the report, the requests library is used to crawl the webpage to obtain data, the bs4 library is used to analyze the data, the pandas library is used to read the data, and the matplotlib library, BeautifulSoup library and pyecharts library are used to realize data visualization.
3 Data Acquisition Steps
Step 1: Get the HTML content from the web page.
The second step: In-depth exploration of the content of the web page, extracting valuable information from it.
Step 3: Export the collected information into an Excel document.
The code for obtaining data and obtaining grain production in various regions from 2016 to 2020 is as follows (the crawler program and its running results are shown in Figure 1
Store the obtained 2016-2020 grain production in various regions of the country in the form of csv in an Excel table (file name: Ulist1), the code is as follows:
Figure 1 Save to Excel file
Data saved to Excel file
Then add the grain production data in 2021 to form a new document.
3.2 Data processing and analysis
1 Data visualization tools
Python has many unique features, including a huge, user-definable base class library that can be used by different programmers.
2 Data Analysis
Analysis of the total grain output data of all regions of the country and the top five regions in the country's grain output.
Use python to visualize the grain production data of various regions from 2016 to 2021, and make a histogram. (As shown in Figure 1)
Figure 2 Analysis of the top five regions in the national grain production
Statistics of the grain output of Jiangxi, Shandong, Henan and Hubei provinces, and the comparison of the grain output of the four provinces and drawing histograms. (as shown in picture 2)
Figure 3 Yield and comparison
Histogram of Grain Production in Jiangxi, Shandong, Henan and Hubei Provinces from 2016 to 2021
Fig. 4 Statistics histogram of grain output in four provinces
Calculate the grain output of the top five provinces (cities, districts) in the national grain output ranking, and draw a pie chart of the total grain output.
Figure 5 Statistics of the top five national grain production
Statistics on the distribution of grain production in the top five provinces (cities, districts) in the country's grain production, and draw a distribution map.
Figure 6 Statistics of the top five national grain production
Statistics of the grain output trends of the top five provinces (cities, districts) in the national grain output rankings over the years, and draw a line chart.
Figure 7 Statistics of the top five national grain production
Calculate the trend of total grain output in various regions of the country from 2016 to 2021, and draw a line chart.
Figure 8 Statistical 2016-2021 National Grain Production Trends by Region
Calculate the trend of grain production in different regions of the country from 2016 to 2021, and draw a line chart.
Figure 9 Statistics of the trend of grain production in different regions of the country from 2016 to 2021, drawing a line chart
Calculate the proportion of grain output in each region in the national grain output in 2020, and draw a pie chart.
Figure 10 Statistical statistics of the proportion of grain production in each region in the national grain production in 2020
Based on the further application of data visualization, a map that can dynamically display the distribution of grain production has been added, and the precise grain production of each region by province and year can be accurately displayed along with the user's pointer position, which greatly improves the user experience and reduces the cost of use.
Figure 11 A map dynamically showing the distribution of grain production
4. Solution to the problem
4.1 Overall solution to the problem:
Through the analysis of grain production data in various regions of the country based on Python, the trend and influencing factors of grain production in various regions are explored, and future production is predicted to provide a reference for agricultural production and policy formulation.
4.2 Description or technical analysis of the main parts of the proposal:
The main part of this thesis includes four aspects: data collection and cleaning, data exploration and visualization, data analysis and modeling, and result presentation and application.
Data collection and cleaning: collect grain production data from various regions of the country, download relevant data from the website of the National Bureau of Statistics, and perform data cleaning to solve data quality problems.
Data exploration and visualization: Use Python data analysis tools to explore and visualize data, and use charts such as scatter plots, line charts, and histograms to display the changing trends and differences in grain production in various regions.
Data analysis and modeling: Through data analysis and modeling, explore the factors affecting grain production in various regions and predict future production. Regression analysis, time series analysis, machine learning and other methods can be used to build models to predict future grain production.
Result presentation and application: Present the analysis results, which can be displayed in reports, charts, visualizations, etc. In addition, the analysis results can also be applied to actual production to guide agricultural production and policy formulation.
4.3 Feasibility analysis of the scheme:
The scheme proposed in this paper is feasible. First of all, the data source is reliable and the data volume is large, which can meet the analysis needs. Secondly, as a popular data analysis tool, Python has powerful data analysis and modeling capabilities, which can meet the analysis needs of this paper. Finally, the analysis results presented in this paper have practical application value and can provide reference for agricultural production and policy formulation.
4.4 Evaluation of Graduation Design Achievements
This paper analyzes the grain production data in various regions of the country based on Python, explores the changing trends and influencing factors of grain production in various regions, and predicts future production, providing a reference for agricultural production and policy formulation. The research methods and results of this paper are very innovative and very practical. It can provide important inspiration and reference for researchers and practitioners in related fields.
5. Conclusion
Through the analysis of grain production data in various regions of the country, the total amount, trend, ranking and proportion of grain production in each region, as well as the factors affecting grain production and correlation analysis and other conclusions are drawn. After in-depth analysis, we put forward a series of suggestions to help improve agricultural production, including promoting agricultural technology innovation, improving agricultural production efficiency, and increasing investment, in order to provide strong support for food security.
references
[1] Huang Xin. Analysis of Grain Yield Data Based on Python[J]. Agricultural Technology and Economics, 2017(5): 48-52.
[2] Wang Li. Analysis and Application of Agricultural Data Based on Python [J]. Modern Agricultural Science and Technology, 2018(1): 97-100.
[3] Chen Kai. Application of Python-based data analysis and visualization in the field of agriculture [J]. Agricultural Informationization, 2019(2): 53-56.
[4] Wang Lili, Chen Jing, Guo Hongbo. Analysis of Grain Yield Data Based on Python [J]. Modern Agricultural Science and Technology, 2018, 10(12): 219-220.
[5] Liu Hui, Zhang Yang, Wang Tingting. Research on Grain Yield Forecasting Model Based on Python [J]. Agricultural Science and Technology, 2019, 19(6): 157-159.
[6] Zhao Yu, Zhang Xu, Tan Ming. Grain Yield Analysis and Forecast Based on Python [J]. Agricultural Modernization Research, 2019, 40(11): 23-27.
[7] Wang Rui, Li Na, Zhang Jun. Spatial Analysis of Grain Production Based on Python [J]. Agricultural Technology and Equipment, 2020, 32(5): 93-95.
[8] Liu Hao, Zhang Lei, Wang Meng. Trend analysis of national grain production based on Python [J]. Agricultural Technology, 2020, 40(5): 31-33.
[9] Li Li, Wang Tao, Guo Yufei. Analysis of Factors Influencing Grain Yield Based on Python [J]. Agricultural Science and Technology, 2021, 21(4): 98-100.
[10] Yao Jing, Zhang Li, Li Ming. Analysis of time-space evolution of grain production based on Python [J]. Agricultural Informationization, 2021, 33(3): 45-48. [11] Zhang Lei. Python Django development practice [M ]. Beijing: Machinery Industry Press, 2015.
thanks
Thanks to the instructor for guiding me and helping me a lot. Thank you for the learning results he shared, which made me less detours in learning and technology, and made my learning go further. At the same time, I would like to thank my parents, who communicated with me during the dissertation, shared my psychological pressure, helped me in daily life, and encouraged me in learning spirit. Finally, I would like to express my gratitude and heartfelt thanks to this group of people who are willing to help me. .
Finally, I would like to express my heartfelt thanks to those who helped me with my graduation thesis. Sincerely thank you!
I wish you a happy life, good health and happiness every day.
appendix
Code to crawl data
import requests
from bs4 import BeautifulSoup
import bs4
import csv
def getHTMLText(url):
try:
r = requests.get(url, timeout = 30)
r.raise_for_status()
r.encoding = r.apparent_encoding
return r.text
except:
return '爬取失败'
def fillUnivlist(ulist,html):
soup = BeautifulSoup(html,"html.parser")
for tr in soup.find('tbody').children:
if isinstance(tr,bs4.element.Tag):
tds = tr('td')
ulist.append([tds[0].text,tds[1].text,tds[2].text,tds[3].text,tds[4].text
tds[5].text])
def printUnivList(ulist,num):
tplt="{0:8}\t{1:^8}\t{2:^8}\t{3:^8}\t{4:^8}\t{5:^8}"
for i in range(num):
u = ulist[i]
print(tplt.format(u[0],u[1],u[2],u[3],u[4],u[5]))
def writeUlistfile(ulist):
with open('Ulist1.csv','w',newline='') as fout:
writer = csv.writer(fout)
for row in ulist:
writer.writerow(row)
def main():
url = 'https://www.chyxx.com/industry/202103/938485.html'
html = getHTMLText(url)
uinfo =[]
fillUnivlist(uinfo,html)
writeUlistfile(uinfo)
printUnivList(uinfo,32)
main()This thesis has passed the Wei Census survey. It was done in 23 years. For details, you can follow me privately. Python, Java, php, html, c language, WeChat applet, APP, Android, Internet of Things and other graduation projects can find me.
Please pay attention and like + private message bloggers to receive the source code of the project
Pay attention to the blogger, the next article is more exciting
One-click three-in-one! ! !
One-click three-in-one! ! !
One-click three-in-one! ! !
Thanks for the one-click triple! ! !
