Apache Sqoop Cookbook in English - translation learn !!
More information https://blue-shadow.top/
problem
Sqoop default type mappings between relational databases and Hadoop provides generally works fine. But sometimes you need to change the type of installation.
solution
Parameter --map-column-Java , Sqoop type conversion may be performed. Example: The conversion type id for the class type long java
sqoop import \
--connect jdbc:mysql://mysql.example.com/sqoop \
--username sqoop \
--table cities \
--map-column-java id=long
discuss
This parameter takes a comma-separated list of key-queue, an equal sign for keys to be represented explicitly column name as the key, as the value for the type of JAVA. For example, if you need to change the three c1, c2 and c3 are data types to float, string and string, so what will be included in the fragments sqoop command line:
sqoop import --map-column-java c1=Float,c2=String,c3=String ...
When the main key to a defined type unsigned int MySQL table column value is greater than this and 2147483647 when using this parameter can easily handle this situation.
In this particular case, MySQL will appear in this column is the type integer, even though the actual type of this column is unsigned int type, the largest of this type 4294967295 Because the display of
type int, sqoop will use the java int type, but this type can not be saved is greater than the value of 2147483647, you need to manually provide a more appropriate type conversion.
Using this parameter is not limited to unsigned type problem for MySQL, this also applies to the default type sqoop does not fit the case when your environment, sqoop data from the database structure
does not require data storage and contact occurs while getting all metadata , so the need for additional information and data are stored separately, especially if you want to fully use the data
as you want to use blob or binary type for storing text data in order to avoid coding problems, you can use --column-map-JAVA column parameters to reset type, and import of text data.
Reproduced in: https: //www.jianshu.com/p/70032ec77e3c