Getting Started Guide for MaxCompute (Formerly ODPS) Development - Data Migration to the Cloud

Full text link http://click.aliyun.com/m/22750/

MaxCompute (formerly ODPS) Development Getting Started Guide - Data on the Cloud
Write at the top
>>> Go to learn more >>> Aliyun Data Plus · MaxCompute Big data computing service.
According to the understanding of "MaxCompute (formerly ODPS) Development Beginner Guide - Metering and Billing", everyone knows what MaxCompute can do and what the billing model is. Presumably everyone has also activated MaxCompute and wants to conduct a POC, but The first question everyone encounters must be how do I transfer my data to the cloud?

SQL Cost

can flow data into MaxCompute in various ways.
MaxCompute (formerly ODPS) provides various channels for data to flow into MaxCompute, including MaxCompute's native Tunnel, as well as Log, DataX and data integration in Alibaba Cloud system, and Flume in the open source ecosystem. , Logstash and Fluentd. The following is for everyone to choose one by one according to their own situation.

① MaxCompute Tunnel command Use the Tunnel command to upload data

directly on the MaxCompute client. The data is encrypted and transmitted during the transmission process, which is suitable for one-time batch data import.

Note that each upload only supports uploading data to one table or one partition of the table. For tables with partitions, the uploaded partition must be specified.
The data upload command is as follows:

tunnel upload log.txt test_project.test_table/p1="b1",p2="b2";



Tunnel command to import data
More about Tunnel upload command
SQL Cost

② MaxCompute Tunnel SDK

uses the SDK provided by TUNNEL to write Java tools.

About how to use the tunnel SDK to upload data, the following will also introduce the scene. Scenario description: Upload data to MaxCompute, where the project space is "odps_public_dev", the table name is "tunnel_sample_test", and the partition is "pt=20150801,dt=hangzhou".

Detailed Tunnel SDK case teaching

③ LOG (Alibaba Cloud Log Analysis Product)

to learn more about log analysis products.
Log Service (Log for short) is a one-stop service for log data, which has been tempered by Alibaba Group through a large number of big data scenarios. You can quickly complete log data collection (Logtail), consumption, delivery (LogShipper), query analysis (LogSearch) and other functions without development, improve operation and maintenance, operation efficiency, and establish the massive log processing capability in the DT era.
Full text link http://click.aliyun.com/m/22750/