With the continuous development of multiple remote sensing platforms such as aviation, aerospace, and near-Earth space, remote sensing technology has advanced by leaps and bounds in recent years. As a result, the spatial, temporal, and spectral resolution of remote sensing data has been continuously improved, and the amount of data has also increased significantly, making it more and more characteristic of big data. For related research, the emergence of remote sensing big data provides unprecedented opportunities, but it also poses huge challenges. Traditional workstations and servers have been unable to meet the needs of large-scale, multi-scale massive remote sensing data processing.
To solve this problem, many global-scale earth science data (especially satellite remote sensing data) online visualization computing and analysis cloud platforms have emerged at home and abroad, such as Google Earth Engine (GEE), Aerospace Hongtu's PIE Engine and Ali's AI Earth, etc. Among them, Earth Engine is the most powerful, capable of accessing and synchronizing satellite images such as MODIS, Landsat, and Sentinel, and meteorological reanalysis data sets such as NCEP, which are currently commonly used in the field of remote sensing. These data are processed. At present, Earth Engine contains more than 900 public datasets, and about 2 PB of data are added every month, with a total capacity of more than 80 PB. Compared with traditional image processing tools (such as ENVI), Earth Engine has incomparable advantages in processing massive remote sensing data. On the one hand, it provides abundant computing resources; on the other hand, its huge cloud storage capacity saves researchers a lot of data download and preprocessing time. It can be said that Earth Engine represents the most cutting-edge level in the field of remote sensing data calculation and analysis visualization, and is a revolution in the field of remote sensing.
Today, with its powerful functions, Earth Engine is attracting the attention of more and more scientific and technological workers at home and abroad, and its application scope is also expanding. This course is dedicated to helping researchers master the practical application ability of Earth Engine. Based on the Python programming language, it will explain platform construction, image data analysis, classic application cases, local and cloud data management, and cloud data paper publication-level visualization with examples. and other advanced skills.
Click to view the original text
Chapter 1, Theoretical Basis
1. Introduction of Earth Engine platform and application, main data resources
2. Important concepts, data types and objects of Earth Engine remote sensing cloud
3. Comparison and selection of JavaScript and Python remote sensing cloud programming
4. Python basics (syntax, data types and program control structures, functions, classes and objects, etc.)
5. Introduction to common Python software packages ((pandas, numpy, os, etc.) and demonstration of basic functions (Excel/csv data file reading and data processing, directory operations, etc.)
6. Differences between JavaScript and Python remote sensing cloud APIs, learning methods and resource recommendations
7. Introduction to AI natural language models such as ChatGPT and Wenxin Yiyan and their applications in the field of remote sensing
Chapter 2, Development Environment Construction
1. Introduction to the local and cloud Python remote sensing cloud development environment
2. Build the local development environment
1) Anaconda installation, pip/conda package installation method and virtual environment creation, etc.;
2) Installation of necessary software packages such as earthengine-api and geemap;
3) Remote sensing cloud local authorization management;
4) Jupyter Notebook/Visual Studio Code installation and running debugging.
3. Cloud Colab development environment construction
4. Geemap introduction and common function demonstration
5. ChatGPT, Wenxin Yiyan account application and main function demonstration, such as remote sensing knowledge answering, data analysis and processing code generation, solution framework consultation, etc.
Chapter 3, the basis of remote sensing big data processing and the interaction with AI models such as ChatGPT
1. Introduction to the image data analysis and processing process of the remote sensing cloud platform: introduce the basic framework of the image data analysis and processing process of the remote sensing cloud platform, including data acquisition, data preprocessing, algorithm development, visualization, etc.
2. Object display and attribute field exploration of elements and images: Introduce how to display and explore attribute fields of elements and images on the remote sensing cloud platform, including how to select elements and image objects, view attribute information, filter data, etc.
3. Time, space and attribute filtering method of image/element set: Introduce how to filter image/element set in time, space and attribute, including how to select time period, geographical area and attribute conditions to achieve more accurate data analysis.
4. Band operation, conditional operation, vegetation index calculation, clipping and mosaic, etc.: Introduce how to perform band operation, conditional operation, vegetation index calculation, clipping and mosaic operations on the remote sensing cloud platform to achieve more in-depth data analysis.
5. Declouding of commonly used optical images such as Landsat/Sentinel-2: Introduce how to use different methods to remove clouds from commonly used optical images such as Landsat/Sentinel-2 on the remote sensing cloud platform to improve the quality of image data.
6. Iterative cycle of images and feature sets: Introduce how to use the iterative cycle function of the remote sensing cloud platform to batch process images and feature sets to improve data analysis efficiency.
7. Image data integration (Reducer): Introduce how to use the Reducer function of the remote sensing cloud platform to integrate multiple image data into one data set to facilitate subsequent data analysis.
8. Neighborhood analysis and spatial statistics: Introduce how to perform neighborhood analysis and spatial statistics on the remote sensing cloud platform to obtain more in-depth spatial information.
9. Common errors and code optimization: Introduce common errors in the data analysis process of the remote sensing cloud platform and how to optimize the code to improve the efficiency and accuracy of data analysis.
10. Python remote sensing cloud data analysis exclusive package construction: Introduce how to use Python to build a data analysis exclusive package on the remote sensing cloud platform to facilitate multiple use and share analysis codes.
Chapter Four, Typical Case Operation Practice
11. Machine learning classification algorithm case: This case combines Landsat and other long-term image series and machine learning algorithms to demonstrate the basic remote sensing classification process at the national scale. The specific content includes image statistics in the study area, spatially stratified random sampling, random sample segmentation, time series image preprocessing and synthesis, machine learning algorithm application, classification postprocessing and accuracy evaluation, etc.
12. Decision tree forest classification algorithm case: In this case, combined with L-band radar and Landsat optical time series images, the decision tree classification algorithm is used to extract the forest distribution map of the designated area from 2007 to 2020, and the spatial comparison is made with JAXA annual forest products. The case involves joint use of multi-source data, construction of decision tree classification algorithm, dynamic optimization of threshold, spatial analysis of classification results, etc.
13. Flood disaster monitoring case: This case is based on Sentinel-1 C-band radar and other images to monitor provincial-level torrential rain disasters. The case content includes Sentinel-1 C image processing, construction of various water body recognition algorithms, image difference analysis, and result visualization. .
14. Drought remote sensing monitoring case: This case uses 40-year-old satellite remote sensing rainfall data products such as CHIRPS to monitor the extreme drought at the provincial scale. The case content includes basic processing of meteorological data, integration of annual and monthly scale data, calculation of long-term average LPA/deviation, and visualization of data results.
15. Case analysis of phenological characteristics: This case is based on time series images such as Landsat and MODIS, and analyzes the phenological differences (sampling point scale) and large-scale (such as China) phenological spatial variation characteristics of typical surface vegetation for many years through the change of vegetation index. The case content includes time series image synthesis, image smoothing (Smoothing) and gap filling (Gap-filling), result visualization, etc.
16. Case of forest vegetation health monitoring: This case uses 20 years of MODIS vegetation index to monitor the forest in a selected area for a long time, and analyze the greening or browning of forest vegetation. Methods for joining and compositing imagery, trend analysis, spatial statistics, and visualization are involved.
17. Case of dynamic monitoring of ecological environment quality: This case uses RSEI remote sensing ecological index and Landsat series images to quickly monitor the ecological status of selected cities. The main technologies involved include the calculation of vegetation index, the extraction of surface temperature, the normalization of data, the PCA analysis of principal components, the construction of RSEI ecological index and the visualization of the results, etc.
Chapter 5, Input and Output and Efficient Management of Data Assets
1. Interaction between local data and cloud: Introduce how to convert local csv, kml, vector and raster data to cloud data, and explain how to export data.
2. Batch download of server-side data: including direct local download, batch download of image sets, and how to quickly download large-scale and long-term data products, such as global forest products and 20-year MODIS data products. .
3. Local data upload and attribute setting: including the use of earthengine commands, how to upload a small amount of local vector and raster data and set attributes (small files), and how to upload data in batches and automatically set attributes, and also introduce how to use the fast Upload skills Upload very large image files, such as domestic high-score images.
4. Personal data asset management : Introduce how to use Python and earthengine command line to manage personal data assets, including creating, deleting, moving, renaming and other operations, and also explain how to cancel upload/download tasks in batches.
Chapter 6, Publication-level Visualization of Cloud Data Papers
1. Introduction to Python visualization and main software packages: Introduce matplotlib and seaborn visualization packages, explain basic graphics concepts, graphics composition, and quick drawing of commonly used graphics.
2. Drawing of topography and plot distribution map of the research area: Combine local or cloud vector files, cloud topographic data, etc. to draw a schematic diagram of the research area. It involves the drawing process, Chinese display, color matching and beautification, etc. It will also introduce the online download and local application of cpt-city exquisite palette palette.
3. Image coverage statistics and mapping in the research area: make statistics on the coverage quantity and cloud-free image coverage of Landsat and Sentinel series images in the designated area, and draw regional image statistical maps or pixel-level cloud-free image coverage thematic maps.
4. Analysis and drawing of sample spectral characteristics and phenological characteristics: Quickly draw spectral and phenological characteristics of different types of plots, dynamically download and integrate thumbnails and vegetation index time series of sampling points in the past 30 years.
5. Thematic map drawing of classification results and Timelapse production of space-time dynamic time-lapse photography: single or multiple classified thematic map drawing and color matching beautification, making Timelapse with clear land use changes, and introducing animation text addition and other content.
6. Area statistics and drawing of classification results: Based on the classification results and vector boundary files in the cloud, count the areas of different land types in different regions, extract the statistical results, and display the statistical areas in different graphics; make statistical drawings of land use changes, etc.
