Python weather data analysis and processing, using python data to analyze weather

Hello everyone, I would like to share with you the research on weather prediction system based on python. Many people still don’t know this. Let’s explain it in detail below. Now let’s take a look!

Analysis of Chongqing meteorological data based on big data

Summary

In the information society, there is a need for targeted ways to obtain information, but the expansion of the ways is basically the direction that people strive for. Due to deviations in the perspective, people can often obtain different types of information, which is also the most difficult problem for technology to overcome. subject. For problems such as meteorological data, we conduct research and analysis on meteorological information, and then develop and design a meteorological data analysis system to solve the problem. Draw Pikachu with 100 lines of python code .

The main functional modules of the Chongqing meteorological data analysis system include system homepage, carousel chart, announcement message, resource management (weather information, information classification) system user (administrator, ordinary user) module management (weather forecast, weather data), adopting a comprehensive The software development and hardware setup of the object development model can well meet the needs of actual use, and the corresponding software setup and program coding work have been improved. Mysql is used as the main storage unit of background data, and Python technology is used. , Ajax technology is used for coding and development of business systems, and all functions of this system are realized. This report first analyzes the background, role, and significance of the research, laying the foundation for the rationality of the research work. Analyzing various requirements and technical issues of the meteorological data analysis system, proving the necessity and technical feasibility of the system, and then giving a basic introduction to the technical software and design ideas that need to be used to design the system, and finally implementing meteorological data analysis System and deployment operations use it.

Keywords: meteorological data; Python technology; Mysql database

Analysis of Chongqing meteorological data based on big data

Abstract

In the information society, there is a need for targeted access to information, but the expansion of the access is basically the direction of people's efforts. Because of the deviation in the perspective, people can often obtain different types of information, which is also the most difficult subject for technology to overcome. Aiming at meteorological data and other problems, the meteorological information is studied and analyzed, and then the meteorological data analysis system is developed and designed to solve the problem.

The main functional modules of Chongqing Meteorological Data Analysis System include the system home page, rotation chart, announcement message, resource management (weather information, information classification) system user (administrator, ordinary user) module management (weather forecast, weather data). The object-oriented development mode is adopted for software development and hardware installation, which can well meet the actual use needs, It has improved the corresponding software installation and program coding work, adopted MySQL as the main storage unit of background data, and adopted Python technology and Ajax technology to code and develop the business system, realizing all functions of the system. This report first analyzes the background, role and significance of the research, laying a foundation for the rationality of the research work. This paper analyzes the various requirements and technical problems of the meteorological data analysis system, proves the necessity and technical feasibility of the system, and then makes a basic introduction to the technical software and design ideas needed to design the system, and finally realizes the meteorological data analysis system and deploys and runs it.

Key words: meteorological data; Python technology; MySQL database

Table of contents

Chapter 1 Introduction

1.1 Research background

1.2 Development status

1.3 Research methods

Chapter 2 Introduction to Related Technologies

2.1 Development technical description:

2.2 Introduction to Python

2.3 Main functions of echars

2.4 Mysql database

2.5 Working principle of B/S system:

Chapter 3 System Analysis

3.1 Feasibility analysis

3.2 Functional requirements analysis

3.2.1 Front-end user function

3.2.2 Backstage administrator function

3.3 Non-functional requirements analysis

3.4 Security requirements analysis

3.4.1 System security

3.4.2 Data security

3.5 Data process analysis

Chapter 4 System Design

4.1 System architecture design

4.2 Overall system design

4.3 System function design

4.4 Database design

4.4.1 Data requirements analysis

4.4.2 Database conceptual design

4.4.3 Database table design

Chapter 5 System Implementation

5.1 Implementation of database access layer

5.2 Implementation of registration module

5.3 Implementation of login module

5.4 Implementation of user data modification module

5.5 Implementation of meteorological information display module

5.6 Implementation of weather forecast module

Chapter 6 System Testing

6.1 Test purpose

6.2 Functional testing

6.3 Performance testing

Chapter 7 Summary and Outlook

references

Acknowledgments

  1. introduction
    1. Research Background

In today's society, competition is fierce, and companies and even countries compete for resources. More important than resources is information. Information plays an increasingly important role in modern economic life and has become an important means of market competition. For enterprises, the importance of information is self-evident. Lack of information Information, even if you have funds, factories, materials and energy, it is very difficult to run a business because the business has no vitality. Therefore, information is the most important resource. Whoever has more information and has accurate information will have authority. Have a chance to win.

However, as the channels for people to obtain information become more and more extensive in modern society, in addition to traditional channels such as newspapers, radio, and television, the addition of new channels such as the Internet, mobile phones, and large outdoor screens everywhere allows people to obtain all kinds of information. The channels are constantly increasing, especially people's understanding of the importance of information is becoming more and more popular and in-depth, the information monopoly is broken, and a large amount of information is shared by people.

In ancient times, there were flying pigeons delivering messages, which could convey the urgent needs of the war; today, there are meteorological data, which can predict the weather tomorrow.

    1. Development status

Meteorological undertakings are technological and basic social welfare undertakings. Weather, climate and climate change are closely related to politics, economy, national defense and people's lives. They are related to all aspects of the national economy and social development, to the production, life and vital interests of the people, and to the implementation of major decisions of the party and the government.

In the context of global warming, extreme weather and climate events are increasing worldwide, abnormal weather phenomena are becoming more frequent, and meteorological disasters are becoming more and more destructive, which has attracted great attention from the international community15'. Competition in meteorological science and technology is becoming increasingly fierce, and its basic status in addressing climate change and supporting international negotiations is increasingly increasing.

According to the famous "Delphi Meteorological Law": the enterprise meteorological input-output ratio is 198, that is, for every 1 yuan invested in meteorological information, you can get an economic return of 98 yuan6.

In Germany, when the temperature exceeds 22 degrees Celsius, beer begins to sell well; for every 1 degree Celsius increase in temperature, an additional 2.3 million bottles of large-bottle beer are sold every day. The German Meteorological Company thus developed the "Beer Index" "7'. In addition, there are similar ride index, ice cream index, swimwear index, food mold index, etc., based on which merchants can formulate production and marketing plans in advance. Smart merchants Based on the weather data, inventory and product varieties can be determined in advance.

Although the production vitality of my country's meteorological services has not yet fully emerged, and there are still certain problems in the development of the meteorological service market, it is certain that with the development of the economy, the improvement of the quality of people's lives and the further development of meteorological science and technology, meteorological services With the further expansion of the field and the improvement of the meteorological service system that adapts to the development of market economy, meteorological services will surely enter all aspects of people's lives, and the meteorological service market will have a broader space for development! 8.

In the near future, Amap will tell you the weather conditions within one kilometer in 15 minutes, as well as the strong northwest wind blowing on a certain street 24 hours ago. When you are driving in heavy rain, Amap will remind you: "The road ahead is seriously flooded. Your vehicle may be flooded when entering. It is recommended that you take a detour."

All in all, the meteorological industry is in a rising stage, and various meteorological data software based on it are emerging one after another, showing a prosperous scene.

    1. Research methods

First, through engine search or relevant literature, we learned about the background of the system development and the significance of designing the system, and collected user demand information. Secondly, in terms of development tools, it was finally decided to use the Python platform to design and develop this system, and Mysql as the tool for designing the database. That is, the Python language is used to implement the user interface and is connected with the database to achieve complete communication functions. After that, the general functional modules of the system are designed. It is mainly analyzed from the perspective of convenience for system users and system administrators to clarify the functions that the system should have. The final step is to test the system, discover existing problems and find solutions through use case testing. Utilize the existing development platform, combine the knowledge you have learned, and complete the design with the guidance of the teacher to ensure the usability and practicality of the system.

  1. Introduction to related technologies
    1. Development technical description:

The front-end part of this system is developed based on the MVVM model, using the B/S mode, and the back-end part is developed based on the python Django framework.

Front-end part: The front-end framework uses the popular progressive JavaScript framework Vue.js. Use Vue-Router and Vuex to realize dynamic routing and global state management, Ajax to realize front-end and back-end communication, Element UI component library to quickly prototype the page, and the project front-end to achieve responsiveness through grid layout, which can be adapted to PC, tablet, mobile phone, etc. Perfect layout display for different screen sizes.

Back-end part: Use Django as the development framework and integrate Redis and other related technologies.

    1.  Introduction to Python

Python is a high-level scripting language that combines interpreted, compiled, interactive, and object-oriented programming. Python's design, compared to other languages ​​that often use English keywords and some punctuation marks from other languages, has a more distinctive grammatical structure than other languages ​​and is highly readable.

Interpreted languages: Similar to Python and Perl languages, this means that there is no compilation link in the development process.

Interactive languages: code can be executed directly after a Python prompt >>>.

Object-oriented language: Python supports object-oriented style or programming technique in which code is encapsulated in objects.

    1.  main functions of echars

ECharts is an open source visualization library implemented using JavaScript. It can run smoothly on PCs and mobile devices. It is compatible with most current browsers (IE8/9/10/11, Chrome, Firefox, Safari, etc.). The underlying layer relies on vector graphics. The library ZRender provides intuitive, interactive, and highly customizable data visualization charts.。 

    1. Mysql database

After many updates, Mysql has become very rich and complete in terms of functionality. It has undergone relatively large updates from version 4 to version 5, and has achieved good practical application results in actual commercial use. The latest version of Mysql supports information compression and encryption, which can better meet the needs for information security. At the same time, after multiple updates of the system, the mirroring function of the database itself has also been greatly enhanced, the smoothness of operation and ease of use have been greatly improved, and the use and creation of drivers have become more efficient and faster. The biggest change is the optimization of the display of spatial information, which makes it easier to label and calculate coordinates on application maps. The powerful backup function also ensures that users will feel more at ease during use. At the same time, the supported Office features also support users’ self-installation and use. The display form of information has also been greatly updated. Two very commonly used display areas have been added. One is the information area, where tables and texts have been classified, and the interface display is more refreshing and specific. The second is the information control of the instrument, which can display information in the instrument information area and compare multiple pieces of information at the same time, which brings great convenience to users' actual use [8][9].

In the actual implementation process of the meteorological data analysis system designed in this article, the main reason for finally choosing the Mysql database is that there will be a large number of frequent database operations in the application and development process of the enterprise's application system, and the security of the data The requirements are also very high. Based on these factors, Mysql, which has a relatively high security factor, was finally selected to store the background data of the meteorological data analysis system.

The overall structure diagram of the database management system is shown in the figure below.

Figure 2-1 Database structure

    1.  Working principle of B/S system:

The B/S architecture adopts the working mode of browser request and server response.

Users can access text, data, pictures, animations, video on demand, sounds and other information generated by Web servers on the Internet through a browser;

Each Web server can be connected to the database server in various ways, and a large amount of data is actually stored in the database server;

Download the program from the Web server to the local for execution. If you encounter instructions related to the database during the download process, the Web server will hand it over to the database server for interpretation and execution, and return it to the Web server, and the Web server will return it to the user. In this structure, many networks are connected together to form a huge network, that is, the global network. Each enterprise can build its own Internet based on this structure.

In the B/S mode, the user requests access through the browser to many servers distributed on the network. The browser's request is processed by the server, and the processing results and corresponding information are returned to the browser. Other data All processing and requests are completed by the Web Server. Through this framework structure and the browser embedded inside the operating system, this structure has become the mainstream structure pattern of today's software applications.

  1. system analysis
    1. Feasibility Analysis

In terms of technology, the current mainstream Python technology is used to build the main framework of the system. The front desk uses jquery and ajax to realize the design and aesthetic adjustment of the front desk page. The above technologies are all systematically studied by me and have been practiced in course design. , which can make development more convenient and systematic. From a technical perspective, this system is completely achievable.

In terms of practicality, the main task of this design is to analyze meteorological data, weather forecast, regional management, annual precipitation, air index, etc. in the meteorological data analysis system, which is in line with the current trend of development. From the user's perspective, we also consider system operating costs and human resources, and use convenient methods on the Internet to implement online business, making the business process more systematic, more user-friendly, and more practical.

In terms of economy, the main purpose of the meteorological data analysis system designed in this project is to provide more convenient and faster information query management and retrieval services, that is, information software that can be directly put into use. The main cost of the system is mainly concentrated on the subsequent maintenance and management update of usage data. However, once the system is put into actual operation and use, it can greatly improve the efficiency of information query and retrieval. At the same time, it is also necessary to effectively ensure the security of the queryer's information. At the same time, the practical application brought by this meteorological data analysis system The value of this aspect far exceeds the cost of developing and maintaining the actual system. Therefore, it is economically feasible to develop this software.

    1. Functional requirements analysis

The functions of the meteorological data analysis system are mainly divided into front-end users registering and logging in according to their own needs, browsing meteorological data and searching for selected weather. The background system administrator has different responsibilities. The administrator mainly handles meteorological data, weather forecast, regional management, annual precipitation, and air index.

The system use case diagram is shown below.

Figure 3-1 System use case diagram

      1. Front-end user function

Front-end users can be divided into unregistered user needs and registered user needs.

The functions for unregistered users are as follows:

Register an account: The user fills in personal information and verifies the mobile phone number.

Browse information: Users can browse weather information based on meteorological data and select a city to view details.

Browse weather data: Select a city to view its weather data.

The functions for registered users are as follows:

Login: Log in based on your account and password.

Maintaining personal information: Users can modify their registration information at any time due to changes in personal information.

Browse weather data: Select a city to view its weather data.

      1. Backend administrator function

Backend administrators can be divided into ordinary administrators and super administrators.

Administrator functions are as follows:

Change password: Administrators can change their login password to enter the system at any time to ensure system security.

Manage ordinary administrators: Manage ordinary administrators. Ordinary administrators can be added or removed when transferring relevant staff.

Weather data management: Maintain weather data information, add, delete, and modify information.

Meteorological data management: Maintain meteorological data information, add, delete and modify information.

User management: You can view the information of registered users and manage them.

    1. Non-functional requirements analysis

First of all, the main consideration is whether the system functional software can better meet the basic functional needs of various users in the specific design process. If it cannot better meet the user needs, then the existence of this system is of no value. The non-functional analysis of software systems is carried out from seven aspects: one is performance analysis, which is aimed at the system; one is security analysis, which is aimed at the system; one is integrity analysis, which is aimed at the system; one is maintainability analysis, which is aimed at the system; One is scalability analysis, targeting the system, and the other is performance analysis to adapt to the business. After a comprehensive comparison and analysis of the seven aspects of performance, security, expansion, and integrity of the meteorological data analysis system, it was found that corresponding non-functional requirements analysis is needed.

    1. Security requirements analysis
      1. System security

Security is very important for every system. A system with good security can protect corporate information and user information from being stolen. Improving system security is not only responsible for users, but also for enterprises. Especially for meteorological data analysis systems, good security must be provided to protect the entire system.

The system has permission control for users, and restricts users' permissions based on different roles to ensure the security of the system.

      1. Data security

The data in the database is input from the outside world. When the data is input, due to various reasons, the input data will be invalid or dirty data. Therefore, how to ensure that the input data complies with regulations has become the primary concern of database systems, especially multi-user relational database systems.

Therefore, when writing to the database, data integrity, correctness and consistency must be ensured.

    1. Data flow analysis

After analyzing the data flow of the system, the users of the system are divided into two categories: general users and administrators. The system mainly handles interface information transmission, login information verification, registration information reception, and responses to various user operations.

The top-level data flow diagram of the system is shown in the figure below.

Figure 3-2 Top-level data flow diagram

To determine the identity of the user, it is judged based on the login data and then jumps to the corresponding functional interface. Users within the system can operate the data, and the database center can receive the effective data stream transmitted by the system to perform corresponding operations on the data SQL statements.

The underlying data flow diagram of the system is shown in the figure below.

 Figure 3-3 Underlying data flow diagram

The system can be divided into two parts: the frontend and the backend. After each operation, the system returns the operation results. The data connection between the front-end and the back-end is mainly through the database, which means different operations are performed on the database respectively.

  1. system design
    1. System architecture design

The architectural design of this meteorological data analysis system is mainly divided into three layers, mainly including Web layer, business layer and Model layer. The web layer also includes the View layer and the Controller layer, and the Model layer includes the metadata extension layer and the data access layer.

The system architecture is shown in the figure below.

Figure 4-1 System architecture

    1. Overall system design

The meteorological data analysis system is generally divided into a front-end user module and a back-end administrator module.

The two modules appear to exist independently, but the database they access is the same. The functions of each module are compiled and produced based on previously completed demand analysis and review of relevant information.

To sum up, the system functional structure diagram is shown in the figure below.

Figure 4-2 System functional structure diagram

    1. System functional design

Login module: The login module is the entrance to the system. All users must log in before they can access the system. You need to enter your username and password to log in. If you try to log in multiple times, you need to enter a verification code. When logging in, you need to select the user's role, whether it is a general user or administrator login, etc. After successful login, the user's permissions will be obtained through the database and jumped to the user's home page.

Administrator user management module: Administrator management includes: administrator's addition, modification and deletion operations. When adding an administrator, first determine whether the administrator added by the user is an admin (super administrator). If not, the addition is successful. When modifying, if you are a super administrator, you can modify the information of all administrators. If you are an ordinary administrator, you can only modify your own information. Super administrators can delete all other administrators except themselves. Ordinary administrators cannot delete administrators.

Meteorological data module: Meteorological data facilitates users to view weather data on a certain date.

Weather data module: It can be divided into three modules: weather data browsing, weather data retrieval, and weather data maintenance. The administrator has the authority to maintain weather data, publish new weather data, update existing weather data, etc.

    1. Database Design
      1. Data requirements analysis

From the previous analysis, we can see that the most important things in the database are meteorological data, user information, and administrator information. The analysis can obtain the following data description:

Platform users: used to record various information of users, including user names, passwords, basic information, reputation, number of published information and other data items.

Administrator: Record the administrator’s login information. Including username, password, permissions and other data items.

Weather data: Storage of weather data content within the platform. Including title, weather data content, time and other data items.

      1. Database concept design

According to the previous data flow chart and combined with the functional module design of the system, each information entity that conforms to the system is designed.

The system ER diagram is shown below.

Figure 4-3 System ER diagram

      1. Database table design

The data tables owned by the meteorological data analysis system are as follows: user information table, meteorological data table, and comment table.

Since there are many data tables, only the main data tables of the system are shown, as shown in the table below.

Table access_token (login access duration)

serial number

name

type of data

length

Decimal places

Allow null values

primary key

default value

illustrate

1

token_id

int

10

0

N

Y

Temporary access badge ID

2

token

varchar

64

0

Y

N

temporary access badge

3

info

text

65535

0

Y

N

4

maxage

int

10

0

N

N

2

Maximum lifespan: Default 2 hours

5

create_time

timestamp

19

0

N

N

CURRENT_TIMESTAMP

Creation time:

6

update_time

timestamp

19

0

N

N

CURRENT_TIMESTAMP

Update time:

7

user_id

int

10

0

N

N

0

user ID:

Table air_index (air index)

serial number

name

type of data

length

Decimal places

Allow null values

primary key

default value

illustrate

1

air_index_id

int

10

0

N

Y

Air index ID

2

date

date

10

0

Y

N

date

3

air_quality

varchar

64

0

Y

N

air quality

4

region

varchar

64

0

Y

N

area

5

pollutant

text

65535

0

Y

N

pollutants

6

exit_guide

longtext

2147483647

0

Y

N

Travel Guide

7

recommend

int

10

0

N

N

0

Intelligent Recommendation

8

create_time

datetime

19

0

N

N

CURRENT_TIMESTAMP

creation time

9

update_time

timestamp

19

0

N

N

CURRENT_TIMESTAMP

Update time

Table annual_precipitation (annual precipitation)

serial number

name

type of data

length

Decimal places

Allow null values

primary key

default value

illustrate

1

annual_precipitation_id

int

10

0

N

Y

annual precipitation ID

2

particular_year

varchar

64

0

Y

N

years

3

date

date

10

0

Y

N

date

4

region

varchar

64

0

Y

N

area

5

weather_type

varchar

64

0

Y

N

weather type

6

rainfall

int

10

0

Y

N

0

rainfall

7

remarks

text

65535

0

Y

N

Remark

8

recommend

int

10

0

N

N

0

Intelligent Recommendation

9

create_time

datetime

19

0

N

N

CURRENT_TIMESTAMP

creation time

10

update_time

timestamp

19

0

N

N

CURRENT_TIMESTAMP

Update time

Table article (article: article for content management system)

serial number

name

type of data

length

Decimal places

Allow null values

primary key

default value

illustrate

1

article_id

mediumint

8

0

N

Y

Article id: [0,8388607]

2

title

varchar

125

0

N

Y

Title: [0,125] is used in the title tag of articles and html

3

type

varchar

64

0

N

N

0

Article classification: [0,1000] is used to search for articles of the specified type

4

hits

int

10

0

N

N

0

Number of clicks: [0,1000000000] Number of people who visited this article

5

praise_len

int

10

0

N

N

0

Number of likes

6

create_time

timestamp

19

0

N

N

CURRENT_TIMESTAMP

Creation time:

7

update_time

timestamp

19

0

N

N

CURRENT_TIMESTAMP

Update time:

8

source

varchar

255

0

Y

N

Source: [0,255] Source of article

9

url

varchar

255

0

Y

N

Source address: [0,255] used to jump to the website where the article was published

10

tag

varchar

255

0

Y

N

Tag: [0,255] is used to mark the relevant content of the article. Multiple tags are separated by spaces.

11

content

longtext

2147483647

0

Y

N

Text: the main content of the article

12

img

varchar

255

0

Y

N

cover picture

13

deion

text

65535

0

Y

N

Article description

Table article_type (article classification)

serial number

name

type of data

length

Decimal places

Allow null values

primary key

default value

illustrate

1

type_id

smallint

5

0

N

Y

Category ID: [0,10000]

2

display

smallint

5

0

N

N

100

Display order: [0,1000] determines the order in which categories are displayed.

3

name

varchar

16

0

N

N

Category name: [2,16]

4

father_id

smallint

5

0

N

N

0

Parent category ID: [0,32767]

5

deion

varchar

255

0

Y

N

Description: [0,255] describes the role of this classification

6

icon

text

65535

0

Y

N

Category icon:

7

url

varchar

255

0

Y

N

External link address: [0,255] If this category jumps to other websites, set it on the URL

8

create_time

timestamp

19

0

N

N

CURRENT_TIMESTAMP

Creation time:

9

update_time

timestamp

19

0

N

N

CURRENT_TIMESTAMP

Update time:

Table auth (user rights management)

serial number

name

type of data

length

Decimal places

Allow null values

primary key

default value

illustrate

1

auth_id

int

10

0

N

Y

Authorization ID:

2

user_group

varchar

64

0

Y

N

user group:

3

mod_name

varchar

64

0

Y

N

Module name:

4

table_name

varchar

64

0

Y

N

Table Name:

5

page_title

varchar

255

0

Y

N

page title:

6

path

varchar

255

0

Y

N

Routing path:

7

position

varchar

32

0

Y

N

Location:

8

mode

varchar

32

0

N

N

_blank

Jump method:

9

add

tinyint

3

0

N

N

1

Is it possible to add:

10

of the

tinyint

3

0

N

N

1

Can it be deleted:

11

set

tinyint

3

0

N

N

1

Whether it can be modified:

12

get

tinyint

3

0

N

N

1

Is it possible to view:

13

field_add

text

65535

0

Y

N

Add fields:

14

field_set

text

65535

0

Y

N

Modify fields:

15

field_get

text

65535

0

Y

N

Query fields:

16

table_nav_name

varchar

500

0

Y

N

Cross-table navigation name:

17

table_nav

varchar

500

0

Y

N

Cross-table navigation:

18

option

text

65535

0

Y

N

Configuration:

19

create_time

timestamp

19

0

N

N

CURRENT_TIMESTAMP

Creation time:

20

update_time

timestamp

19

0

N

N

CURRENT_TIMESTAMP

Update time:

Table comment (comment)

serial number

name

type of data

length

Decimal places

Allow null values

primary key

default value

illustrate

1

comment_id

int

10

0

N

Y

Comment ID:

2

user_id

int

10

0

N

N

0

Commenter ID:

3

reply_to_id

int

10

0

N

N

0

Reply comment ID: empty is 0

4

content

longtext

2147483647

0

Y

N

content:

5

nickname

varchar

255

0

Y

N

Nick name:

6

avatar

varchar

255

0

Y

N

Avatar address: [0,255]

7

create_time

timestamp

19

0

N

N

CURRENT_TIMESTAMP

Creation time:

8

update_time

timestamp

19

0

N

N

CURRENT_TIMESTAMP

Update time:

9

source_table

varchar

255

0

Y

N

Source table:

10

source_field

varchar

255

0

Y

N

Source field:

11

source_id

int

10

0

N

N

0

Source ID:

Table hits (user clicks)

serial number

name

type of data

length

Decimal places

Allow null values

primary key

default value

illustrate

1

hits_id

int

10

0

N

Y

Like ID:

2

user_id

int

10

0

N

N

0

Liked by:

3

create_time

timestamp

19

0

N

N

CURRENT_TIMESTAMP

Creation time:

4

update_time

timestamp

19

0

N

N

CURRENT_TIMESTAMP

Update time:

5

source_table

varchar

255

0

Y

N

Source table:

6

source_field

varchar

255

0

Y

N

Source field:

7

source_id

int

10

0

N

N

0

Source ID:

Table meteorological_data (meteorological data)

serial number

name

type of data

length

Decimal places

Allow null values

primary key

default value

illustrate

1

meteorological_data_id

int

10

0

N

Y

Meteorological data ID

2

weather

varchar

64

0

Y

N

weather

3

date

varchar

64

0

Y

N

date

4

lowest_temperature

varchar

64

0

Y

N

lowest temperature

5

highest_temperature

varchar

64

0

Y

N

highest temperature

6

air_quality_index

varchar

64

0

Y

N

Air quality index

7

wind_direction

varchar

64

0

Y

N

wind direction

8

recommend

int

10

0

N

N

0

Intelligent Recommendation

9

create_time

datetime

19

0

N

N

CURRENT_TIMESTAMP

creation time

10

update_time

timestamp

19

0

N

N

CURRENT_TIMESTAMP

Update time

Notice (announcement)

serial number

name

type of data

length

Decimal places

Allow null values

primary key

default value

illustrate

1

notice_id

mediumint

8

0

N

Y

Announcement id:

2

title

varchar

125

0

N

N

title:

3

content

longtext

2147483647

0

Y

N

text:

4

create_time

timestamp

19

0

N

N

CURRENT_TIMESTAMP

Creation time:

5

update_time

timestamp

19

0

N

N

CURRENT_TIMESTAMP

Update time:

Express praise (like)

serial number

name

type of data

length

Decimal places

Allow null values

primary key

default value

illustrate

1

praise_id

int

10

0

N

Y

Like ID:

2

user_id

int

10

0

N

N

0

Liked by:

3

create_time

timestamp

19

0

N

N

CURRENT_TIMESTAMP

Creation time:

4

update_time

timestamp

19

0

N

N

CURRENT_TIMESTAMP

Update time:

5

source_table

varchar

255

0

Y

N

Source table:

6

source_field

varchar

255

0

Y

N

Source field:

7

source_id

int

10

0

N

N

0

Source ID:

8

status

bit

1

0

N

N

1

Like status: 1 is like, 0 is canceled

Table regional_management (regional management)

serial number

name

type of data

length

Decimal places

Allow null values

primary key

default value

illustrate

1

regional_management_id

int

10

0

N

Y

Region management ID

2

region

varchar

64

0

Y

N

area

3

recommend

int

10

0

N

N

0

Intelligent Recommendation

4

create_time

datetime

19

0

N

N

CURRENT_TIMESTAMP

creation time

5

update_time

timestamp

19

0

N

N

CURRENT_TIMESTAMP

Update time

Table registered_user (registered user)

serial number

name

type of data

length

Decimal places

Allow null values

primary key

default value

illustrate

1

registered_user_id

int

10

0

N

Y

Register user ID

2

user_no

varchar

64

0

N

N

user ID

3

full_name

varchar

64

0

Y

N

Name

4

examine_state

varchar

16

0

N

N

passed

Approval Status

5

recommend

int

10

0

N

N

0

Intelligent Recommendation

6

user_id

int

10

0

N

N

0

User ID

7

create_time

datetime

19

0

N

N

CURRENT_TIMESTAMP

creation time

8

update_time

timestamp

19

0

N

N

CURRENT_TIMESTAMP

Update time

Table slides (carousel)

serial number

name

type of data

length

Decimal places

Allow null values

primary key

default value

illustrate

1

slides_id

int

10

0

N

Y

Carousel image ID:

2

title

varchar

64

0

Y

N

title:

3

content

varchar

255

0

Y

N

content:

4

url

varchar

255

0

Y

N

Link:

5

img

varchar

255

0

Y

N

Carousel image:

6

hits

int

10

0

N

N

0

Number of clicks:

7

create_time

timestamp

19

0

N

N

CURRENT_TIMESTAMP

Creation time:

8

update_time

timestamp

19

0

N

N

CURRENT_TIMESTAMP

Update time:

Table upload (file upload)

serial number

name

type of data

length

Decimal places

Allow null values

primary key

default value

illustrate

1

upload_id

int

10

0

N

Y

Upload ID

2

name

varchar

64

0

Y

N

file name

3

path

varchar

255

0

Y

N

access path

4

file

varchar

255

0

Y

N

file path

5

display

varchar

255

0

Y

N

display order

6

father_id

int

10

0

Y

N

0

Parent ID

7

dir

varchar

255

0

Y

N

folder

8

type

varchar

32

0

Y

N

file type

Table user (user account: used to save user login information)

serial number

name

type of data

length

Decimal places

Allow null values

primary key

default value

illustrate

1

user_id

mediumint

8

0

N

Y

User ID: [0,8388607] User obtains other user-related data

2

state

smallint

5

0

N

N

1

Account status: [0,10](1 available | 2 abnormal | 3 frozen | 4 logged out)

3

user_group

varchar

32

0

Y

N

User group: [0,32767] determines user identity and permissions

4

login_time

timestamp

19

0

N

N

CURRENT_TIMESTAMP

Last Login Time:

5

phone

varchar

11

0

Y

N

Mobile phone number: [0,11] User’s mobile phone number, used to retrieve password or log in

6

phone_state

smallint

5

0

N

N

0

Mobile phone certification: [0,1] (0 not certified | 1 under review | 2 certified)

7

username

varchar

16

0

N

N

Username: [0,16] The account name used by the user to log in

8

nickname

varchar

16

0

Y

N

Nickname: [0,16]

9

password

varchar

64

0

N

N

Password: [0,32] The password required for user login, consisting of 6-16 digits or English

10

email

varchar

64

0

Y

N

Email: [0,64] User's email, used to retrieve password or log in

11

email_state

smallint

5

0

N

N

0

Email certification: [0,1] (0 not certified | 1 under review | 2 certified)

12

avatar

varchar

255

0

Y

N

Avatar address: [0,255]

13

create_time

timestamp

19

0

N

N

CURRENT_TIMESTAMP

Creation time:

Table user_group (user group: used for user front-end identity and authentication)

serial number

name

type of data

length

Decimal places

Allow null values

primary key

default value

illustrate

1

group_id

mediumint

8

0

N

Y

User group ID: [0,8388607]

2

display

smallint

5

0

N

N

100

Display order: [0,1000]

3

name

varchar

16

0

N

N

Name: [0,16]

4

deion

varchar

255

0

Y

N

Description: [0,255] Describes the characteristics or scope of permissions of this user group

5

source_table

varchar

255

0

Y

N

Source table:

6

source_field

varchar

255

0

Y

N

Source field:

7

source_id

int

10

0

N

N

0

Source ID:

8

register

smallint

5

0

Y

N

0

Registration location:

9

create_time

timestamp

19

0

N

N

CURRENT_TIMESTAMP

Creation time:

10

update_time

timestamp

19

0

N

N

CURRENT_TIMESTAMP

Update time:

Table weather_forecast (weather forecast)

serial number

name

type of data

length

Decimal places

Allow null values

primary key

default value

illustrate

1

weather_forecast_id

int

10

0

N

Y

Weather forecast ID

2

date

date

10

0

Y

N

date

3

weather

varchar

64

0

Y

N

weather

4

region

varchar

64

0

Y

N

area

5

picture

varchar

255

0

Y

N

picture

6

highest_temperature

varchar

64

0

Y

N

highest temperature

7

lowest_temperature

varchar

64

0

Y

N

lowest temperature

8

temperature_difference

varchar

64

0

Y

N

temperature difference

9

air_quality_index

varchar

64

0

Y

N

Air quality index

10

humidity

varchar

64

0

Y

N

humidity

11

pulmonary_particulates

varchar

64

0

Y

N

Particles that can enter the lungs

12

one_week_weather_forecast

text

65535

0

Y

N

Weekly weather forecast

13

january_weather_forecast

text

65535

0

Y

N

January weather forecast

14

details

longtext

2147483647

0

Y

N

Details

15

hits

int

10

0

N

N

0

Clicks

16

recommend

int

10

0

N

N

0

Intelligent Recommendation

17

create_time

datetime

19

0

N

N

CURRENT_TIMESTAMP

creation time

18

update_time

timestamp

19

0

N

N

CURRENT_TIMESTAMP

Update time

  1. System implementation
    1. Implementation of database access layer

It can be seen from the principle of B/S architecture that the implementation of each major module of the meteorological data analysis system requires the operation of database data, including querying data, writing data, updating data and deleting data. Therefore, when developing each functional module Before, first create a file named "conn.Python". This file is mainly used to connect data. In the future, when the program needs to operate data, it may be called using the statement "<?Python reqiure_once('conn.Python');?>" That's it.

    1. Implementation of registration module

When the user fills in the data, it must match the verification on the registration page, otherwise the registration will fail. The form verification on the registration page is verified through JavaScript. The length of the user name must be between 6 and 18, and the email address must have the @ symbol. , the password and password confirmation must be the same. For the password you enter, the system will give a specified value based on the strength of the password you enter. The phone number and ID number must be input in a format consistent with life. When you pass the verification at the front desk, you Click register, and the form will pass the value you entered to the background through the name value and save it to the database.

The user registration flow chart is shown below.

Figure 5-1 User registration flow chart

The user registration interface is shown in the figure below.

Figure 5-2 User registration interface

    1. Implementation of login module

主要由两部分组成,登录前的登录界面以及登录后的用户功能界面。登录界面,要求用户输入用户名和密码,当用户名和密码其中一个输入为空时,给出提示“用户名,密码不能为空”。获取用户名和密码后到数据库中查找,如果用户名存在,以及对应的密码正确,则登录成功,否则登录失败。登录失败后给出提示,并把焦点停在文本框中。登录成功后将该次会话的全局变量username设置为用户名。登录成功后进入会员的功能模块,主要有会员基本信息修改,已经发布气象数据管理,发布信息,和退出功能。退出功能是清除全局变量username的值,并跳回到首页。

登录流程图如下图所示。

图5-3登录流程图

用户登录界面如下图所示。

图5-4用户登录界面

    1. 用户资料修改模块的实现

用户登录/注册成功之后可以修改自己的基本信息。修改页面的表单中每一个input的name值都要与实体类中的参数相匹配,在用户点击修改页面的时候,如果改后用户名与数据库里面重复了,页面会提示该用户名已经存在了,否则通过Id来查询用户,并将用户的信息修改为表单提交的数据。

    1. 气象资讯展示模块的实现

如果气象资讯的信息需要修改,管理员可以通过查询气象资讯的基本信息来查询气象资讯,查询气象资讯是通过ajax技术来进行查询的,需要传递气象资讯的标题、编号等参数然后在返回到该页面中,可以选中要修改或删除的那条信息,如果选中了超过一条数据,页面会挑一个窗口提醒只能选择一条数,如果没有选中数据会挑一个窗口题型必须选择一条数据。当选择确认修改的时候,后台会根据传过来的id到数据库查询,并将结果返回到修改页面中,可以在修改页面中修改刚刚选中的信息当点击确认的时候from表单会将修改的数据提交到后台并保存到数据库中,就是说如果提交的数据数据库中存在就修改,否则就保存。

气象资讯展示界面如下图所示。

图5-5气象资讯展示界面

    1. 天气预报模块的实现

此页面的关键是编写天气预报,包括天气编号,名称,详情等。单击提交按钮以完成信息的添加。如果未写入完整的天气预报,例如,如果未写入天气编号,系统将给出相应的错误提示,并且无法成功输入。数据以概念的形式以onsubmit =“return checkForm()”的形式写入以进行检查,checkForm()函数是一种用于写入数据的不同类型的校对方法,是不是为空也是经过form表单中的οnsubmit=”return checkForm()来检查。

管理员点击左侧菜单“天气预报管理”,页面跳转到天气预报管理外观,调用后台天气查询所有天气预报。并将信息密封到数据集合List,绑定到请求对象,然后页面跳转到相应的Python页面,显示出天气预报,单击删除按钮完成天气预报的删除。

天气预报流程图如下图所示。

图5-6天气预报流程图

天气预报添加界面如下图所示。

图5-7天气预报添加界面

天气预报界面如下图所示。

图5-8天气预报界面

  1. 系统测试
    1. 测试目的

对任何系统而言,测试都是必不可少的环节,测试可以发现系统存在的很多问题,所有的软件上线之前,都应该进行充足的测试之后才能保证上线后不会Bug频发,或者是功能不满足需求等问题的发生。下面分别从单元测试,功能测试和用例测试来对系统进行测试以保证系统的稳定性和可靠性。

    1. 功能测试

下表是系统登录功能测试用例,检测了用户名和密码的不同的输入情况,观察系统的响应情况。得出该功能达到了设计目标。

表6-1 系统登录功能测试用例

功能描述

用于系统登录

测试目的

检测登录时的合法性检查

测试数据以及操作

预期结果

实际结果

输入的用户名和密码带有非法字符

提示用户名或者密码错误

与预期结果一致

输入的用户名或者密码为空

提示用户名或者密码错误

与预期结果一致

输入的用户名和密码不存在

提示用户名或者密码错误

与预期结果一致

输入正确的用户名和密码

登录成功

与预期结果一致

下表是注册功能测试用例,检测了各种数据的输入情况,观察系统的响应情况。得出该功能达到了设计目标。

表6-2 注册功能测试用例

功能描述

用于用户注册

测试目的

检测用户注册时的合法性检查

测试数据以及操作

预期结果

实际结果

输入的手机号不合法

提示请输入正确的手机号码

与预期结果一致

输入的字段为空

提示必填项不能为空

与预期结果一致

输入的密码少于6位

提示密码必须为6-12位

与预期结果一致

输入的密码大于12位

提示密码必须为6-12位

与预期结果一致

下表是天气预报功能的测试用例,检测了天气预报中对天气预报的增加,删除,修改,查询操作是否成功运行。观察系统的响应情况,得出该功能也达到了设计目标,系统运行正确。

前置条件;用户登录系统。

表6-3 天气预报的测试用例

功能描述

用于天气预报

测试目的

检测天气预报时的各种操作的运行情况

测试数据以及操作

预期结果

实际结果

点击添加天气,必填项合法输入,点击保存

提示添加成功

与预期结果一致

点击添加天气,必填项输入不合法,点击保存

提示必填项不能为空

与预期结果一致

点击修改天气,必填项修改为空,点击保存

提示必填项不能为空

与预期结果一致

点击修改天气,必填项输入不合法,点击保存

提示必填项不能为空

与预期结果一致

点击删除天气,选择天气删除

提示删除成功

与预期结果一致

点击搜索天气,输入存在的天气名

查找出天气

与预期结果一致

点击搜索天气,输入不存在的天气名

不显示天气

与预期结果一致

    1. 性能测试

使用阿里云PTS(Performance Testing Service)性能测试服务对线上系统进行压力测试。线上服务器环境为:1核心CPU,1G内存,1Mbps公网带宽,Centos7.0操作系统。

压测过程中使用了2台并发机器,每台机器20个用户并发,对系统主页,登录,数据查询和数据维护等模块进行并发访问,测试结果是有40个用户并发时,数据管理相关页面的响应时间甚至达到了7s,通过查看服务器出网流量发现已经达到1381kb/s,可以看出服务器的带宽已经达到峰值,如果系统使用5Mbps的带宽,系统的响应时间和TPS将会大大增加。在整个测试的过程中,CPU的使用率占用仅8%,也提现出带宽瓶颈对系统的影响非常严重。

  1. 总结与展望

随着计算机互联网技术的迅猛发展,各行各业都已经实现采用计算机相关技术对日益放大的数据进行管理。该课题是气象数据分析系统为核心展开的,主要是为了实现气象数据化管理和用户浏览天气的需求。

气象数据分析系统的开发是以Python编程语言作为基础,在PythonStrom平台上完成编码工作,系统整体为B/S架构,数据库系统使用Mysql。文中详细分析了气象数据分析系统的研究背景、研究目的和意义、开发工具和相关技术以及系统需求、系统详细设计和系统测试等等一系列内容。系统实现了气象数据分析系统所需的一些基本功能,并通过测试对这些实现的功能进行了完善,进而提高了系统整体的实用性。整个系统的开发过程中大量使用了Python相关的知识以及前端开发使用的html和java等,同时涉及到了很多开源框架和组件,例如后台系统中运用的MVVM架构、Freemarker模板引擎等,前端运用的UI框架等。

系统投入运行时,各功能均运行正常。系统的每个界面的操作符合常规逻辑,对使用者来说操作简单,界面友好。整个系统的各个功能设计合理,体现了人性化。

但是由于自己在系统开发过程中对一些用到的相关知识和技术掌握不够牢固,再加上自身开发经验欠缺,因此系统在有些方面的功能还不够完善,考虑的不够全面,因此整个系统还有待日后逐步完善。

参考文献

[1]李同金.基于Python的端口扫描技术研究[J].电子世界,2022(02):38-39+42.DOI:10.19353/j.cnki.dzsj.2022.02.015.

[2]孙琳,徐文正.Python编程语言教学中问题分析能力培养的研究[J].产业与科技论坛,2022,21(04):182-183.

[3] Gao Wang. Practice of teaching reform of "Python language basics" under the background of new engineering [J]. Technology and Innovation, 2022(02):148-150+155.DOI:10.15913/j.cnki.kjycx.2022.02.044 .

[4] Peng Wenliang, Yu Yanhua. Adaptive classification method of text data streams based on Python language [J]. Journal of Ningxia Normal University, 2022, 43(01): 106-112.

[5] Wei Yiyang, Wu Yifan, Li Yongyuan. Research on the application of Python technology in data visualization [J]. Fujian Computer, 2022, 38(01): 27-31.DOI: 10.16707/j.cnki.fjpc.2022.01.007 .

[6] Dai Chengqiu. Design of hybrid teaching plan for Python language programming course [J]. Computer Education, 2022(01):162-166.DOI:10.16512/j.cnki.jsjjy.2022.01.037.

[7] Qin Weichao, Dai Xiaofen. Clever use of open source hardware to improve problem-solving abilities in Python programming learning [J]. China Information Technology Education, 2022(01):62-63.

[8] Xu Gaojian, Xu Haoyu. Design and implementation of C-like compiler based on Python language [J]. Journal of Luoyang Institute of Technology (Natural Science Edition), 2021, 31(04): 78-84.

[9] Xin Fengyang, Wang Zhongxin, Tian Fengliang, Zhao Ming, Zeng Xiangyu, Wang Jinjin. Design and implementation of an integrated management and control platform for continuous processes in open-pit mines based on BS architecture [J]. Metal Mining, 2021(12):177-182.DOI:10.19614 /j.cnki.jsks.202112027.

[10]Li Li. Employment Data Analysis based on Python Crawler Technology[C]//Proceedings of 4th International Workshop on Education Reform and Social Sciences (ERSS 2021).,2021:164-168.DOI:10.26914/c.cnkihy.2021.049800.

[11] Fang Sheng. Design and implementation of "Web" front-end based on "MVVM" model [J]. Computer Knowledge and Technology, 2021, 17(20): 147-149.DOI: 10.14004/j.cnki.ckt. 2021.2004.

[12] Kate Egan and Jeffrey Andrew Weinstock, And Now for Something Completely Different: Critical Approaches to Monty Python[J]. Journal of British Cinema and Television,2021,18(3):

[13]Melchert Oliver,Demircan Ayhan. pyGLLE: A Python toolkit for solving the generalized Lugiato–Lefever equation[J]. SoftwareX,2021,15:

[14] Liu Ziwei. Design and implementation of online education system based on MVVM design pattern [D]. Beijing University of Posts and Telecommunications, 2021.DOI:10.26969/d.cnki.gbydu.2021.002712.

[15] You Junhui. Comparison of three architectural models: MVC, MVP, and MVVM [J]. Office Automation, 2020, 25(22):11-12+27.

[16] Guo Minquan, Shen Hui, Mai Xiaoping, Ding Zhuoming, Zhang Lin, Qian Zhaojiang, Sun Qizhen. Application and testing of China's polar numerical meteorological data analysis system at the Antarctic Great Wall Station [J]. Ocean Forecast, 2021, 38(04) ):69-75.

[17] Sun Qiangqiang, Guo Jiajun. Design of meteorological data analysis system based on Android platform [J]. Yangtze River Information and Communications, 2021, 34(04): 106-109.

[18] Du Hui, Cai Zhanwen, Yin Qiyuan, Ye Zewen, Liu Sanmei, Jia Xiaohong, Shang Kezheng. Refined forecast of low cloud cover in the surrounding areas of Hetao based on the global meteorological data analysis system [J]. Journal of Lanzhou University (Natural Science Edition), 2021 ,57(02):176-184.DOI:10.13885/j.issn.0455-2059.2021.02.005.

[19] Wang Haiming. Europe and the United States will cooperate to establish an enhanced space weather data analysis system [J]. Journal of Space Science, 2019, 39(05):566.

[20] Pan Ningkang, Huang Zebin, Pan Yinhui, Chen Huanji, Wu Zehao, Lu Chunwan. Systematic review of meteorological data analysis based on big data platform [J]. Modern Information Technology, 2019, 3(06): 29-31.

Acknowledgments

This design took 3 months. In this graduation project, it is inseparable from the guidance of the instructor to make things basically smooth. The instructors gave me great help both in the graduation project experience and in completing the thesis. On the other hand, I have benefited a lot from the teacher's serious and responsible work attitude, cautious teaching spirit and solid theoretical standards. His diligent and cautious teaching, educating and learning attitude also left a very deep impression on me. I learned a lot from my teachers. My skills have been greatly improved in theory and practice. Here, I would like to express my sincere gratitude to the teachers.

After all the research and development of this graduation project, my system research and development has experienced special progress from requirement analysis to implementation of detailed functions to final testing and maintenance. It gave me a deeper understanding of system research and development. Now my hands-on ability to solve doubts independently has also been greatly improved. This is the best gain from this graduation project.

Finally, during the entire system development process, my classmates and friends around me gave me a lot of opinions, so I quickly confirmed the business idea of ​​the system. This time, I sincerely express my gratitude to them.

Please follow and like + privately message the blogger to receive the project source code for free

Guess you like

Origin blog.csdn.net/mynote/article/details/135200721