Continuing from the above, since NASA officially abandoned FTP and switched to HTTPS service, the download method of MODIS data has changed. The full series of blogs can be found at the link below. This article mainly follows the script download introduction of the application key that was not finished in the previous article.
Introduction and download of MODIS data (1) - Introduction to MODIS data
Introduction and download of MODIS data (2) - MODIS data download method (FTP)
Introduction and Download of MODIS Data (Extra) - Python Client Application of MODIS Web Service
Introduction and download of MODIS data (4) - HTTPS service download instructions
1 Official Tutorial and Instructions
Continuing from the above section, the previous blog has already talked about a small number of application keys. This article mainly focuses on the following parts (red box part).
1 Application key
1 Apply for an application key
Anyone who has signed up for an Earthdata account ( signup link ) can apply for an app key. Follow the steps below to apply for an application key.
1. First go to the corresponding page: LAADS DAAC and log in to Earthdata. 2. Then click Profile→App Keys (see screenshot). 3. Create a new App Key by entering your description of the keys and clicking the "Create New App Key" button.
Of course, if you have an application key before, but you have forgotten it, follow the steps below: 1. Log in to Earthdata (same as above). 2. Then click Profile→App Keys (same as above). 3. Copy your key.
In general, this operation is not much different from the domestic Baidu map and AutoNavi map API.
2 After I have an application key, what should I do?
Application keys can request data via the HTTPS GET service. Regarding the GET and POST services of HTTPS, you can take a look at a few posts I found. Those who have done crawlers and called APIs should be familiar with them.
HTTP method: GET vs POST Zhihu: What is the difference between get and post?
The official example is to use the "curl" command-line tool to create a URL with a request.
https://ladsweb.modaps.eosdis.nasa.gov/PATH_TO_MY_FILE
curl -v -H 'Authorization: Bearer MY_APP_KEY' 'https://ladsweb.modaps.eosdis.nasa.gov/PATH_TO_MY_FILE' > result
-v and -H are additional setup commands.
curl is a command line tool for all operating systems. Introduction to curl , that is to say, the data corresponding to the order can be downloaded through curl.
It gives some points for downloading using this method:
1. All strings are important, including dashes, colons and quotes; 2. Replace 'MY_APP_KEY' with your application key; 3. Replace "PATH_TO_MY_FILE" with the path of the file you need. 4. Usually the file path of LAADS DAAC is like the following form:
archive/allData/COLLECTION/PRODUCT/YEAR/DAY_OF_YEAR/FILENAME
Here is an example of a URL:
After sending the request, it will return you a top-atmospheric reflectance product of MODIS Terra250m on the 18th day of 2007.
The author's commonly used systems are Ubuntu and Windows 10. Here I will demonstrate how to use curl to download data (using Ubuntu as an example). Take the given URL as an example.
On Ubuntu, you can install curl directly with the apt-get install command. There are posts on the Internet, so I won't go into details here. Then rewrite the command line according to the above, if you don't need "> result", download it according to the original file name.
downloading.
result data.
Attempts under Windows were not very pleasant. Of course, curl is not the main download method, so I will not continue to explore, if there is a chance to talk about this later.
2 Automation
If the data you need is a single file and you know where it is located in the LAADS data archive, it's easy to click and download it. If you need to download a lot of files (such as last month's VIIRS data for the entire month), you may prefer to use a script to download. So here are some code examples: Shell scripts, Perl and Python versions. Two warnings: 1. Don't download all the data to your hard drive. 2. Try to avoid errors in your script as much as possible, to prevent the IP from being blocked due to too many downloads.
3 Code Examples
HTTPS communication is possible in most languages, some examples are below. The way to use it is to click "Download Source" to download or copy the code and paste it into a file that reflects the language (.sh for Shell scripts, .p1. for Perl, .py for Python). Make sure execute permissions are set for the file. Finally, open a terminal or execute the file with your preferred runtime.
Example:
perl laads-data-download.pl
I don't know much about Perl. This article mainly introduces Python scripting, and Shell scripting will also be mentioned.
2 Download using Python script
First download laads-data-download.py, then put it in a folder, then open cmd and enter the following command.
python laads-data-download.py -h
The meaning of this code is that -h refers to help, that is to say, the instructions for the use of this Python function.
laads-data-download.py [-h] -s URL -d DIR -t TOK
Simply put, this function has several parameters that need to be passed in, -s is the download source, URL is the URL path of the data you want to download, -d is the download path, which is the path to which the data should be downloaded, and -t is the token , tokens, which are actually your app keys. That is to say, the complete running code should be as follows:
python laads-data-download.py -s https://ladsweb.modaps.eosdis.nasa.gov/archive/orders/YOUR ORDERS ID -d Paht TO MY FILE -t MY_APP_KEYS
The first red box is the ID of your order data (as shown below).
The second red box is the path where your data is stored in the computer.
The third red box is your app keys.
Then you just need to wait for the data to download.
By the way, the Shell script is also downloaded from the laads-data-download.sh file. In fact the syntax is the same. But the shell script also needs to rely on 'jq' to download.
./laads-data-download.sh [-h] -s URL -d PATH -t TOKEN
'jq' can be installed with the command.
apt-get install jq
Wait for the download to finish.
So in fact, the Python script download is actually not difficult. Just replace the corresponding URL, path, and App Keys. The same goes for the others, if you get the hang of it you can start experimenting. Of course, the author has also used another method to perform batch downloads before. This part may be introduced later. Another point is that the author's test results show that the download speed of Shell scripts and Ubuntu is significantly faster than that of Python scripts and Windows.