Multi-process crawling Shanghai housing prices and drawing heat map analysis

1. Purpose of Analysis

1. Explore the regional distribution of housing prices in Shanghai

2. See where homebuyers like to buy

2. Data collection

Collect the data of 12,000 residential areas in the Shanghai area of I Love My Home. The collected fields include residential area, location, number of the last 30 transactions, on sale, rent, average transaction price, total transaction price, and detailed introduction of the community. Go directly to the code:

The collection looks like this:

Then use the Gaode API to convert the cell into latitude and longitude coordinates, this code will not be posted. After conversion it becomes like this:

3. Data visualization

Generate heatmaps using Baidu Maps API.

<!DOCTYPE html>
<html>
<head>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
    <meta name="viewport" content="initial-scale=1.0, user-scalable=no" />
    <script type="text/javascript" src="http://api.map.baidu.com/api?v=2.0&ak=ie1pboEhSttv7biL1iYj6kUI"></script>
    <script type="text/javascript" src="http://api.map.baidu.com/library/Heatmap/2.0/src/Heatmap_min.js"></script>
    <title>Example of heat map function</title>
    <style type="text/css">
		ul,li{list-style: none;margin:0;padding:0;float:left;}
		html{height:100%}
		body{height:100%;margin:0px;padding:0px;font-family:"Microsoft Yahei";}
		#container{height:500px;width:100%;}
		#r-result{width:100%;}
    </style>	
</head>
<body>
	<div id="container"></div>
	<div id="r-result">
		<input type="button" onclick="openHeatmap();" value="Show Heatmap"/><input type="button" onclick="closeHeatmap();" value="Close Heatmap"/>
	</div>
</body>
</html>
<script type="text/javascript">
    var map = new BMap.Map("container"); // create a map instance

    var point = new BMap.Point(121.409868, 31.208977);
    map.centerAndZoom(point, 15); // Initialize the map, set the center point coordinates and map level
    map.enableScrollWheelZoom(); // Enable scroll wheel zoom
  
    var points =[
{"lng":121.524245,"lat":31.234472,"count":5},
{"lng":121.525868,"lat":31.225027,"count":10},
{"lng":121.439535,"lat":31.133423,"count":5},
{"lng":121.511138,"lat":31.231639,"count":1}

];
   
    if(!isSupportCanvas()){
    	alert('The heat map currently only supports browsers with canvas support, the browser you are using cannot use the heat map function~')
    }
	//For detailed parameters, you can check the documentation of heatmap.js https://github.com/pa7/heatmap.js/blob/master/README.md
	//The parameter description is as follows:
	/* visible whether the heat map is displayed, the default is true
     * opacity thermal transparency, 1-100
     * radius size of the radius of each point of the influence map   
     * gradient {JSON} The gradient interval of the heatmap. The gradient is as follows
     *	{
			.2:'rgb(0, 255, 255)',
			.5:'rgb(0, 110, 255)',
			.8:'rgb(100, 0, 255)'
		}
		Where key represents the position of interpolation, 0~1.
		    value is the color value.
     */
	heatmapOverlay = new BMapLib.HeatmapOverlay({"radius":50});
	map.addOverlay(heatmapOverlay);
	heatmapOverlay.setDataSet({data:points,max:11});
	//whether to display the heatmap
    function openHeatmap(){
        heatmapOverlay.show();
    }
	function closeHeatmap(){
        heatmapOverlay.hide();
    }
	closeHeatmap();
    function setGradient(){
     	/*The format is as follows:
		{
	  		0:'rgb(102, 255, 0)',
	 	 	.5:'rgb(255, 170, 0)',
		  	1:'rgb(255, 0, 0)'
		}*/
     	var gradient = {};
     	var colors = document.querySelectorAll("input[type='color']");
     	colors = [].slice.call(colors,0);
     	colors.forEach(function(ele){
			gradient[ele.getAttribute("data-key")] = ele.value;
     	});
        heatmapOverlay.setOptions({"gradient":gradient});
    }
	/ / Determine whether the browser area supports canvas
    function isSupportCanvas(){
        var elem = document.createElement('canvas');
        return !!(elem.getContext && elem.getContext('2d'));
    }
</script>

Replace the data in point with your own coordinates, modify the radius and maximum value, save it locally, modify it to index.html, put it in an easy-to-find location, and then use python to mount a simple server, in the cmd command window, cd to index.html corresponds to the folder, and then enter

python -m http.server 80

Then open http://localhost:80 with your browser

You can see the heatmap. Let's take a look at where the houses that have been sold in the last 30 days are:

It can be seen that the houses that have recently been transacted are mainly concentrated next to the Huangpu River. It seems that everyone prefers houses near the river. The most transactions of houses are in the Xinzhuang area of Minhang (it is estimated that the house price is slightly cheaper) and near the Bund in Pudong (it is estimated that Financial practitioners have relatively strong purchasing power).

Multi-process crawling Shanghai housing prices and drawing heat map analysis

Guess you like