In order to buy a house, I grabbed the transaction data of a certain house (analysis)

1. Dependence

Recently, interest rates on loans have been dropping, and when I get married, I plan to buy a suite as a wedding house. Otherwise, I will have to rent a house after getting married. I can't bear to keep moving.

After making up my mind (after all, I have to bear a loan of several million, so I have done a lot of psychological construction for myself, haha), so I joined the big team of house inspections. With the continuous increase in the accumulated experience of viewing houses, I feel more and more that the industry of intermediaries is very deep, and they can use any means to make a deal. Of course, there are also very professional intermediaries who have given a lot of objective opinions, but they are still a minority

Considering that there is no way to keep an eye on the changes of housing listings, and the housing listings in Beijing do not display transaction prices, as a senior bald programmer, he must be thinking about whether there is any magical operation that can solve these two problems, So the goal to be achieved is just around the corner

1. Listing change daily report
2. Catch the real estate transaction price

After careful analysis (looked at while waiting to load the game), I found that it could be achieved, so I took action quickly (after playing N games)

This series is divided into two articles "In order to buy a house, I grabbed the data of a certain house - analysis", "In order to buy a house, I grabbed the data of a certain house - actual combat", this article is the principle Analysis article, it is better to eat after paying attention~

2. Effect display

In order to prove the strength of Ben Diao, the effect must be applied first, let’s take a look at the picture below

Since Ben Diao is a back-end coder, he is really not good at beautifying the interface, so everyone will just wait and see

The data is mainly divided into two blocks, with daily changing data and all data

The yellow title represents the data that has changed today . There are sold listings, closed listings, and new listings will also be displayed

All data refers to all the listing data details from crawling to the present

And if the listing has been sold, it will display the [Signing Date], [Signing Price] and [Close Period] of the listing.

3. Analysis

3.1 Bearer mode of a certain business

After looking at the renderings, let’s analyze this requirement and how to achieve it

The first thing that needs to be understood is the carrying form of a certain business and the difficulty of data acquisition, mainly including:

1. A certain app. The mobile APP can do many functions, and generally has the functions of anti-capture and data encryption, and the difficulty of data acquisition is high.
2. a website. It can be accessed and captured on the browser, and data acquisition is relatively easy
3. a small program. Accessing on WeChat requires tools to capture packets, and the difficulty of data acquisition is average

According to the difficulty of data acquisition, a website that can be directly opened on the browser is undoubtedly the first choice.

So I analyzed a certain website. This process is very important, and it is necessary to guard against arrogance and impetuosity.

3.2 Analysis of data acquisition process

The first thing is, how to obtain all the housing information of the favorite community?

Lianjia provides the function of searching houses in a specified community, as shown in the figure below, you can get the general information and house information of the community, if you can’t fit in one page, you need to turn the page, as shown in the figure below

With the above data, you can get the newly listed housing information, the formula is: 今天新上架的房源 = 今天的所有房源 - 昨天所有的房源, the complexity of this formula makes me call myself a little genius again and again

Lianjia has made certain restrictions on the listings that have been taken off the shelves/stopped sale and the listings that have been sold. It is necessary to pay attention to the listings and wait until it is [off the shelves/stopped sale] or [closed] before changing the listing in the watch list. Source status information, as follows:

Listings that have been discontinued from the watchlist

Listings sold in the watch list

So in theory, we only need to pay attention to all the listings in this community, and then continuously traverse their status to get the information about the listings that have been removed from the shelves and sold out.

Simple to say, simple to do, it involves two points:

1. Log in
2. Follow listings

Most websites save the login state by borrowing the function of cookies. We can just throw all the cookies into it when accessing the interface. I will introduce these technical details in detail in the implementation chapter, so I don’t need to pay attention here for the time being.

For paying attention to the listings, pay attention to it, grab a bag and it’s OK, it looks quite simple, let’s meet these details for the actual combat~

Four. Summary

This article mainly analyzes Lianjia’s website and finds suitable data information. The following practical article will introduce how to capture packets, analyze the data you need, and send email notifications

This series of articles is for learning purposes only, and shall not be used for illegal or criminal acts