Land use management is a key component of natural resource management, and land use investigation, change detection and evaluation have important guiding significance for territorial and spatial planning. However, the processing methods of land use data are complex, and the performance of large-scale data processing is poor, which makes it difficult to guarantee the timeliness of processing results. Industry application personnel urgently need simpler processing methods and higher performance processing technologies to realize massive land use data. Efficient management and application.
Optimization of land use data processing plan
• Distributed GIS technology as support
SuperMap's distributed spatial data engine technology and distributed spatial analysis technology provide powerful technical support for the high-performance processing and analysis of large-scale land use data. By introducing distributed storage and storing land use data into distributed spatial databases such as HBase or DSF, the efficiency of data access during processing is greatly improved. The distributed analysis technology based on the Spark distributed computing framework can realize the full and efficient processing and calculation of massive land use data.
Based on the above-mentioned distributed storage and analysis technology ideas, in the past, we needed to complete the entire land use data processing business process based on the SuperMap iObjects Java for Spark component and through the writing of program codes. However, this requires industry application personnel to have code writing experience and a certain degree of knowledge of distributed technology. Therefore, the threshold for use is relatively high. In addition, in the face of business process adjustments, the reuse rate of original business codes is low, and a more convenient and efficient means to implement business process construction is urgently needed.
• Convenient and efficient new method
The geoprocessing modeling technology introduced by SuperMap SuperMap GIS 10i (2020) provides a wealth of distributed analysis and result publishing and other predefined tools for large-scale land use data processing services. Using tools to visually build a business model with zero code is not only simple to operate, but also able to flexibly respond to complex business needs, and walk through the complete process of data processing and result display.
Rich vector data distributed analysis tools
The constructed geoprocessing model supports cluster mode operation. Distributed cluster resource scheduling is performed through the setting of environmental parameters. The high-performance memory cache mechanism can effectively avoid the disk read and write of intermediate data, greatly improving the processing efficiency of massive land use data.
In order to realize the sharing of land use business models, geoprocessing modeling supports publishing the model as a tool. After publishing, it supports tool invocation through the REST API interface, making model reuse more convenient and efficient. In the face of business process adjustment, only by adding, deleting or replacing some geoprocessing tools in the existing business model, the business model can be rebuilt quickly.
Facing the special needs of business, such as business calculation, result assembly, etc., geoprocessing modeling supports users to expand and develop custom tools. Custom tools and pre-defined tools are connected to each other and participate in distributed analysis together to achieve high-performance large-scale land use data processing.
Land use big data analysis process based on geoprocessing modeling
Land use change detection and cultivated land quality analysis are common cases of large-scale land use data processing. Next, we take land use change detection and cultivated land quality analysis based on geoprocessing modeling as an example, and invite you to experience the geographical distribution of distributed GIS Deal with the practice of modeling in real business.
• Land use change detection
In the land and resources industry, land use change detection analyzes the land use change in the same area to determine the law of land change in the region, and then analyze the impact of human production, life and environmental changes on land use.
Land use change detection can be summarized into three major steps: land parcel change detection, change data aggregation, and display of change results. After importing the land survey data into HBase or HDFS, based on geoprocessing modeling, we select geoprocessing tools such as overlay analysis, attribute summary, and publishing map services, and connect the tools in the canvas according to the analysis steps to complete the land change detection setup model.
Land change detection model
One-click execution of the model, automatic land use change detection according to the process, get the data of the land parcels where the land use type changes during the two land surveys and the summary results of the land area of each land use type, and finally load the land use change land data The pre-prepared thematic map template will be published for browsing and sharing.
We can further publish the model as a server-side geoprocessing tool to facilitate subsequent calls to the model interface and sharing with other users. Build a personalized front-end display page based on the model, and then perform analysis by calling the interface of land change detection, and finally obtain and display the analysis results. The original complex and tedious land change detection process is simplified to be completed efficiently through a single page.
The distributed analysis of land use change detection through geoprocessing modeling is not only simple and convenient, but also realizes the automation of the detection process, which effectively solves the cumbersome processing steps of traditional spatial analysis tools and the problem of data processing step by step. Improved the analysis performance of land change detection.
• Analysis of the
quality of cultivated land The quality of cultivated land is related to the ability of grain output. In recent years, the frequency of cultivated land quality analysis in the land and resources industry has increased. Carrying out the evaluation and analysis of the quality of cultivated land can provide a scientific basis for comprehensive management of cultivated land.
In the Guizhou Province Land Use Big Data Analysis Platform, which won the Gold Award for the Geographic Information Industry, geoprocessing modeling provides pre-defined tools such as overlay analysis for the cultivated land quality analysis function under the land thematic analysis, and the two-level analysis of business calculations and cultivated land quality levels Situation analysis and other tools are customized and extended by the project team based on the actual situation of the cultivated land data and computing requirements.
In order to improve the analysis performance, the project team also expanded and developed a custom database reading tool, which is connected with the predefined tools such as constructing DSF based on the index to read and convert the data of the database into a geographic region feature data set (DSFFeatureRDD). This data set format optimized by distributed computing can significantly improve the computing performance of big data, and efficiently perform vector superposition operations of tens of millions and above. At the same time, the newly-added high-performance memory cache mechanism is used to cache the intermediate results using tools such as the cache DSF data set to avoid data storage and redundancy, and the analysis performance is further improved.
The custom tools and predefined tools developed by the project team jointly participate in the construction of the business model, connect with each other, and finally complete the entire farmland quality analysis process.
Cultivated land quality analysis process
Next, the project team will publish the built model as a geoprocessing tool, and the front end of the Guizhou Provincial Land Use Big Data Analysis Platform can directly call the tool through the REST API. Users can complete high-performance farmland quality analysis through simple page operations, which not only has an excellent user experience, but also greatly improves work efficiency.
According to the field test of the project team, using the old data platform of Guizhou Province to analyze the land quality of 5W pieces of data, it took several hours; using SuperMap's traditional Java component tools to analyze the 5W pieces of data, it took half an hour; and Using distributed GP modeling and analysis, the amount of data is increased by 5 times, and the time for 25W data is only three minutes, and the analysis is improved by 50 times.
SuperMap GIS 10i (2020)
geoprocessing modeling technology starts from a zero-code visualized construction model, provides a wealth of predefined tools, and supports the expansion and development of custom tools. Users can flexibly use tools to build business models according to actual analysis needs. It provides a great help for the automation and high-performance data processing of large-scale land use data. At the same time, the cluster mode of distributed geoprocessing tools and the newly-added memory cache mechanism escort geoprocessing performance. The more complex the processing and the larger the amount of data, the more obvious the performance improvement.The era of big data is coming, and the construction of various big data platforms is in full swing. This article brings new thinking to the realization of platform functions, and also provides technical solutions for large-scale data complex services.