MaxCompute Studio has been released for a long time, but I have not been able to experience it well. Recently, MaxCompute Studio has successively launched many useful functions. Today, I will share with you the experience of using it.
Students who have used the big data development kit will encounter the following problems when uploading and downloading local data:
- 'Import local data' function, limit the maximum size of local data files to 10MB;
- When the query results are exported locally, due to the limitation of the maximum number of rows returned by the select statement, only 1W of data can be exported at most;
- The function of exporting data to local files is only 'export query results to local'.
To solve the above problems, you usually need to install the console client yourself, and import and export through the tunnel command .
Now let's share the tool interface of MaxCompute Studio, which can easily complete the import and export of local data that can be done by the tunnel command .
For the installation of MaxCompute Studio tools, please refer to the documentation to install IntelliJ IDEA and install MaxCompute Studio .
local data import
Prerequisites: The table and partition already exist, and the operator has permission to write data to the table.
Go to the Project Explorer window of MaxCompute Studio, expand Data Preview, right-click the table where data needs to be imported, and select Import data into table
In the pop-up box, select the uploaded local data file path, partition (non-partitioned table does not need to be filled), separator, line limit, size limit, etc.:
Note: Size Limit is not limited to 10MB.
After clicking OK, you can see the progress bar in the lower right corner, and you can view the relevant logs in the Event Log if the event is successful or not. The import success log will show the number of successfully written rows and the number of unsuccessfully written rows. Such as:
上午10:35 Success: Import to table tablename from file /Users/.../Documents/.../t_data.txt finished, success [ 220977 ], failed [ 0 ].
Data export locally
Prerequisite: The operator has permission to export data from the table.
Go to the Project Explorer window of MaxCompute Studio, expand Data Preview, right-click the table where data needs to be imported, and select Export data from table
Enter the export file address in the pop-up box, including the file name and file type (CSV or TXT), selecting the partition to be downloaded (non-partitioned table directly downloads the entire table), export row limit, size limit, etc.
Note: The number of rows is not limited to 10,000 rows, and the size is not limited to 10MB. You can download the entire non-partitioned table or the entire partition of the partitioned table.
After clicking OK, you can see the progress bar in the lower right corner, and you can view the relevant logs in the Event Log if the event is successful or not. The export success log will show the number of rows that were successfully exported and the number of rows that failed to be exported. Such as:
上午10:47 Success: Export from table tablename to file /Users/..../Desktop/... finished, success [ 220977 ], failed [ 0 ].
Conclusion: The above-mentioned local data import and export functions in MaxCompute Studio actually use the MaxCompute Tunnel service, and the functions are the same as those of the Tunnel command .