Summary of Common Problems Using MaxCompute

Abstract: A summary of common problems with MaxCompute, which is convenient for users to quickly troubleshoot problems.

Billing related

Storage billing: Step-by-step billing is performed according to the capacity of the data stored in MaxCompute.

Calculation and billing: MaxCompute has two calculation and billing methods: post-payment and pre-payment by CU.

Pay-As-You-Go: Pay-As-You-Go is billed for SQL tasks and MapReduce tasks.

Pay-as-you-go for SQL tasks: SQL tasks are paid after I/O.

MapReduce pay-as-you-go: that is, MapReduce tasks are billed on a per-volume basis.

Prepaid by CU: This method is only available on Alibaba Cloud big data platform.

Download billing: MaxCompute will charge according to the size of the downloaded data.
Note that the public network download is charged, and the intranet download is not charged. Please
refer to https://help.aliyun.com/document_detail/27989.html?spm=5176.doc45389. 6.560 . The

difference between LYjhcl SQL syntax and standard SQL

1, no transaction control
2, temporarily do not support update/delete
3, do not support modifying field types, do not support deleting fields
4, do not support select from multiple tables
5, other hive benchmarking functions, The official documents are not supported temporarily. For
more reference
https://help.aliyun.com/document_detail/50985.html?spm=5176.doc27862.6.757.lSC74f

SQL limits

select to display a maximum of 10,000 rows. If

you want to export more than 10,000 rows, please refer to
https://help.aliyun.com/document_detail/53093.html?spm=5176.7751065.6.751.iLm6hF
2, and the maximum string field is 8M. For
more other restrictions, please refer to
https:/ /help.aliyun.com/document_detail/51823.html?spm=5176.doc53093.6.618.R37r8p

java sandbox restrictions

users often ask:
the local test is normal, but it fails after submitting to odps, the code is the same
https:/ /help.aliyun.com/document_detail/27967.html?spm=5176.doc51823.6.647.Rt0efa
Reason: We
have restricted some third-party jar packages for security reasons.
Generally , the fastjson package is more common.

The java sandbox is restricted, and some third-party jar packages are restricted, and the fastjson jar package is restricted and cannot be used because of the security sandbox.
https://help.aliyun.com/document_detail/27967.html

It is recommended to use gson
import com.google.gson.Gson; try this package.
Other jars are restricted. Users who encounter similar errors can submit work orders for consultation

The following figure is an example of a common error.
Picture description If the

new type flag is not enabled

, if you want to use a new data type, you need to add a statement before the SQL statement: set odps.sql.type.system.odps2=true;

the current situation is that you need to add this set, follow-up Will save
https://help.aliyun.com/document_detail/27821.html?spm=5176.doc27967.6.546.XhnAGU

new data type, and new function, function return value is a new data type need to set
if there is no set, Error
com.aliyun.odps.OdpsException: ODPS-0130071:[1,8] Semantic analysis exception - function or view udtf_recom cannot be resolved The
function or method

oss ​​and ots external table cannot be created, the query fails

1, check whether the following Document authorization
https://help.aliyun.com/document_detail/45389.html?spm=5176.doc27821.6.691.5g0yGt
2, the error is similar to
DDL syntax error:ODPS-0130071:Semantic analysis exception - unrecognized option 'EXTERNAL'.
Check : Whether a new type is referenced, set
3 is also required, if the query fails after the table is created
Check whether the amount of user oss or ots data is huge,
whether the number of objects in oss is large, and whether a
single object is large
. Refer to the document https://www.atatech.org/articles/67977 for
understanding . The example of error reporting is not authorized

. Image description

Client odpscmd reports an error

. Connect to the client. ／For accessing the domain name, first look at the domain name
https://help.aliyun.com/document_detail/34951.html?spm=5176.doc48950.6.572.58H50H
Project name Ak
information Environment (java9 is not supported for the time being), it is recommended that java 7 and 8 tasks run slowly 1. If it is pay-as-you-go, please submit a work order. If it is pre-paid, you can first check whether the project corresponding to cu has a large number of jobs or is relatively large. The work has not been completed for a long time. You can use the master account to execute the command: TOP INSTANCE; this way you can view the currently running tasks (running on the client (version 0.29.0 or above).) Download the latest version of the client: https://help.aliyun. com/document_detail/27804.html? You can also use this tool to view the status of running tasks under the project https://yq.aliyun.com/articles/247739?spm=5176.8091938.0.0.7Sf007










If you are a prepaid cu user, you can use this tool to check the cu usage
https://help.aliyun.com/document_detail/66565.html?spm=a2c4g.11186623.6.713.QARDJv
2, data skew
https://help.aliyun .com/document_detail/51020.html?spm=5176.doc34951.6.754.iMKNUT

Original link: https://yq.aliyun.com/articles/536451?spm=a2c41.11181499.0.0