Computer major project report case 84: Design and implementation of a second-hand real estate statistical display system based on Python crawler

If you need the source code, drop me a message

Table of contents

1. Research background

With the continuous development of society, the Internet era is also developing continuously. Various products derived from the Internet are widely used in various fields. At the same time, with the popularity of smart real estate and the growth of consumer demand for mobile Internet, competition in the real estate market has become more intense, and the sales data of various real estate developers has also become an important competitive indicator. Through the analysis and visualization of real estate sales data, we can better grasp the development trends of real estate and consumer needs, thereby providing important reference for the formulation of corporate marketing strategies. At the same time, with the continuous development of the Internet and big data technology, the collection and processing of real estate sales data have become more convenient and effective. At the same time, the application of data visualization technology is becoming increasingly widespread. By converting data into visual graphics or pictures, the distribution and changes of data can be more intuitively displayed, thereby achieving a deeper understanding of the market and consumers. The system uses the computer language Python as a tool to crawl real estate sales data and analyze it from multiple angles, allowing users to intuitively understand the current sales and performance of each property. Therefore, visual analysis of real estate sales data is an important topic in today's information age and has a significant impact on the competition and development of the real estate industry. ,

2. Main content

This article uses software engineering methods to conduct a detailed design and introduction of a second-hand real estate statistical display system based on Python crawlers. By collecting, cleaning, and analyzing data, and migrating the cleaned data to MySQL, Django is used to build a sales data visualization interface and create a large visual screen. Using the visualization of real estate data, users can see the results of data analysis more intuitively and clearly, so as to synthesize personal needs and make correct purchasing decisions.

The visual demonstration of this system consists of six parts. The functions of each module are shown below:

(1) The homepage displays the detailed information of Anjuke’s second-hand houses, which includes the property’s name, unit type, area, orientation, floor, release time, community name, community label, total price, unit price, etc.

(2) Display the number of listings of second-hand properties, the number of transactions, the settlement rate of quantity, and the settlement rate of amount, and display it in a donut-shaped circular chart;

(3) Calculate the number of listed properties according to type, and use a bar chart for statistics; the y-axis represents the number of buildings, and the x-axis represents the type of house and the information about the number of rooms and living rooms.

(4) A circular pie chart is drawn to count the number of each room type. Different room types are represented by different colors. A donut-shaped pie chart showing the popular programs, counting the number of units sold according to the popularity of the program;

(5) The house overview module displays the real estate information released by users, using an up and down scrolling method for carousel display;

(6) Count the number of properties according to the orientation of the houses. The visual graphics are displayed with bar charts. The y-axis represents the counted quantity, and the x-axis represents the orientation of each type of house;

(7) The month-on-month number of transactions module is displayed in the form of a line chart. The y-axis represents the number of transactions and the x-axis represents the month of the year.

3. Key issues

This paper focuses on the majority of real estate users and collects and analyzes the real estate sales data of the Anjuke platform to provide an effective information display and support for people to understand the current sales status of the most popular real estate products [9]. For this purpose, the problem can be analyzed from three perspectives before visualizing the data.

(1) How to obtain data: write code on the Pycharm platform, obtain the URL from the Anjuke webpage, and make a request;

(2) Data analysis method: Use the Pandas library and PyQuery in Python for data clarity and analysis, and store it in MySQL data;

(3) Data display method: Use Django to build a sales data visualization interface, integrate all the functions of the Django website, and combine it with ECharts to create a large visual screen.

4. Development languages and tools

This system uses HTML5, JavaScript, CSS and other technologies to realize the display of front-end web pages, and then uses Django framework, data crawler technology and python technology, Echart visual component library and mysql as the data repository to achieve back-end management.

5. References

[1] Zhu Xiaoqin . Application of Apriori improved algorithm and Django framework in accurate recommendation system for college books [J]. Journal of Ezhou University , 2022(001):029.

[2] Tu Jin , Jiang Wanchen , Leng Zhengxing . Differentiated research on factors affecting housing prices in cities in China - Analysis based on big data of the second-hand housing market in Chengdu [J]. Price Theory and Practice , 2021(10):4.

[3] Li Ying , Wang Jinsheng , Zhang Zhuang . Real-time monitoring and management of second-hand real estate data using computer network [J]. Chinese Journal of Hospital Infectious Diseases , 2002, 12(9):2.

[4] Jia Yanping , Zhai Jingang . Visual analysis of tourist review data based on Python crawler technology [J]. Journal of Anyang Normal University , 2021(5):51-54.

[5] Wang Jianwei . An automatic crawler management method based on microservices :, CN112765438A[P]. 2021.

[6] Yang Yan, Jiang Jingyi, Zhao Yin, Zheng Chuanxing and Ao Jin . Application of house rental data analysis based on Python [J]. Information Technology and Standardization , 2021, 000(009):75-78.

[7] Guan Fei , Zhang Han . Prediction of second-hand housing prices in Zhengzhou based on data mining [J]. 2021.

[8] Wang Yanya . Crawling and analyzing second-hand housing data in Langfang City based on Python [J]. 2021.

[9] Wei Yiyang , Wu Yifan , Li Yongyuan . Research on the application of Python technology in data visualization [J]. Fujian Computer , 2022, 38(1):27-31.

[10] Ji Zhengxiao . Crawling and analyzing Wuhan second-hand housing information based on Python [J]. Information and Computers , 2022, 34(16):195-199.

[11] Chen Zhiqiang . Feasibility analysis of heat recovery in boiler slag cooler cooling water system [J]. 2022(13).

[12] Zhang Ruiming . Feasibility analysis of the main transformer cooler control system of a thermal power plant based on DCS [J]. Northeast Electric Power Technology , 2021, 42(8):3.