版权声明：本文为博主原创文章，未经博主允许不得转载。 https://blog.csdn.net/qq_14959801/article/details/77888456
CREATE TABLE pi_cust_item_recommend8(
> ADVISE_ID VARCHAR(30),
> CUST_CODE VARCHAR(30),
> CGT_CODE VARCHAR(30),
> CRT_DATE VARCHAR(8),
> NOTICE VARCHAR(3),
> USER_ID VARCHAR(30),
> ADVISE_LEVEL decimal(5,3)
> )
> partitioned by (ymday string)
> clustered by (CUST_CODE) into 12 buckets row format delimited fields terminated by '\001';
OK
Time taken: 0.312 seconds
hive> describe pi_cust_item_recommend8;
OK
advise_id varchar(30)
cust_code varchar(30)
cgt_code varchar(30)
crt_date varchar(8)
notice varchar(3)
user_id varchar(30)
advise_level decimal(5,3)
ymday string
# Partition Information
# col_name data_type comment
ymday string
Time taken: 0.172 seconds, Fetched: 13 row(s)
hive> SELECT CAST(int(rand()*1000000) as varchar(30)),CO_CUST.nation_cust_code,PLM_ITEM.pack_bar,'20170905','10','TCSADMIN',pi_cust_item_recommend2.advise_level FROM pi_cust_item_recommend2 INNER JOIN CO_CUST ON pi_cust_item_recommend2.cust_id = CO_CUST.cust_id inner join PLM_ITEM on pi_cust_item_recommend2.item_id=PLM_ITEM.item_id limit 3;
Found ticket for [email protected] to go to krbtgt/[email protected] expiring on Fri Sep 08 10:48:34 CST 2017
Entered Krb5Context.initSecContext with state=STATE_NEW
Service ticket not found in the subject
Session re-established.
Status: Running (Executing on YARN cluster with App id application_1503626611027_0229)
----------------------------------------------------------------------------------------------
VERTICES MODE STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
----------------------------------------------------------------------------------------------
Map 1 .......... container SUCCEEDED 2 2 0 0 0 0
Map 2 .......... container SUCCEEDED 2 2 0 0 0 0
Map 3 .......... container SUCCEEDED 1 1 0 0 0 0
----------------------------------------------------------------------------------------------
VERTICES: 03/03 [==========================>>] 100% ELAPSED TIME: 7.69 s
----------------------------------------------------------------------------------------------
OK
215067 110102206466 6901028940689 20170905 10 TCSADMIN 60.72310558804653
270954 110102206466 6901028118187 20170905 10 TCSADMIN 9.942863654959975
569210 110102206466 6901028316866 20170905 10 TCSADMIN 10.621125508964525
Time taken: 15.388 seconds, Fetched: 3 row(s)
hive> SELECT CAST(int(rand()*1000000) as varchar(30)),CO_CUST.nation_cust_code,PLM_ITEM.pack_bar,'20170905','10','TCSADMIN',pi_cust_item_recommend2.advise_level FROM pi_cust_item_recommend2 INNER JOIN CO_CUST ON pi_cust_item_recommend2.cust_id = CO_CUST.cust_id inner join PLM_ITEM on pi_cust_item_recommend2.item_id=PLM_ITEM.item_id limit 31;
WARNING: Comparing a bigint and a string may result in a loss of precision.
Query ID = superadmin_20170907223434_35102611-b531-4b7f-bfb0-2bf216d7ac6d
Total jobs = 1
Launching Job 1 out of 1
Status: Running (Executing on YARN cluster with App id application_1503626611027_0229)
----------------------------------------------------------------------------------------------
VERTICES MODE STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
----------------------------------------------------------------------------------------------
Map 1 .......... container SUCCEEDED 2 2 0 0 0 0
Map 2 .......... container SUCCEEDED 2 2 0 0 0 0
Map 3 .......... container SUCCEEDED 1 1 0 0 0 0
----------------------------------------------------------------------------------------------
VERTICES: 03/03 [==========================>>] 100% ELAPSED TIME: 7.93 s
----------------------------------------------------------------------------------------------
OK
722145 110102206466 6901028940689 20170905 10 TCSADMIN 60.72310558804653
930616 110102206466 6901028118187 20170905 10 TCSADMIN 9.942863654959975
415818 110102206466 6901028316866 20170905 10 TCSADMIN 10.621125508964525
62395 110102206466 6901028315012 20170905 10 TCSADMIN 10.748528216651824
349555 110102206466 6901028936750 20170905 10 TCSADMIN 10.756672311271405
49636 110102206466 6901028084864 20170905 10 TCSADMIN 11.134520882990092
544648 110102206466 6901028080446 20170905 10 TCSADMIN 11.153070464110803
68224 110102206466 6901028138536 20170905 10 TCSADMIN 11.366099729791326
975291 110102206466 6901028058148 20170905 10 TCSADMIN 11.603361435468745
220807 110102206466 6901028150378 20170905 10 TCSADMIN 11.689022768057503
141702 110102206466 6901028005036 20170905 10 TCSADMIN 11.882462449560231
269026 110102206466 6901028177153 20170905 10 TCSADMIN 11.979572435677673
463826 110102206466 6901028001687 20170905 10 TCSADMIN 12.160150106222419
847827 110102206466 6901028157629 20170905 10 TCSADMIN 12.827278813265025
750240 110102206466 6901028024976 20170905 10 TCSADMIN 13.198598635991594
152545 110102206466 6901028311168 20170905 10 TCSADMIN 14.145684045570716
462842 110102206466 6901028084895 20170905 10 TCSADMIN 14.543860630556074
998752 110102206466 6901028133487 20170905 10 TCSADMIN 14.69225879590099
715950 110102206466 6901028084741 20170905 10 TCSADMIN 17.126573594234525
181834 110102206466 6901028075039 20170905 10 TCSADMIN 18.50133249391921
935074 110102206466 6901028317269 20170905 10 TCSADMIN 18.681565978743667
451227 110102206466 6901028048828 20170905 10 TCSADMIN 19.414165825698127
888096 110102206466 6901028208352 20170905 10 TCSADMIN 19.750765686500053
46792 110102206466 6901028186582 20170905 10 TCSADMIN 19.917558066949805
687769 110102206466 6901028119009 20170905 10 TCSADMIN 20.44004660243949
396555 110102206466 6901028104050 20170905 10 TCSADMIN 21.493937559219983
162789 110102206466 6901028124942 20170905 10 TCSADMIN 21.532472887028405
160869 110102206466 6901028143059 20170905 10 TCSADMIN 22.578897017969844
284319 110102206466 6901028126687 20170905 10 TCSADMIN 23.66996101158705
633708 110102206466 4711588110956 20170905 10 TCSADMIN 38.67483685224603
710439 110112103885 6901028157629 20170905 10 TCSADMIN 76.04121085665929
Time taken: 9.457 seconds, Fetched: 31 row(s)
hive> insert into table pi_cust_item_recommend8 PARTITION (ymday='"20170905"') SELECT CAST(int(rand()*1000000) as varchar(30)),CO_CUST.nation_cust_code,PLM_ITEM.pack_bar,'20170905','10','TCSADMIN',pi_cust_item_recommend2.advise_level FROM pi_cust_item_recommend2 INNER JOIN CO_CUST ON pi_cust_item_recommend2.cust_id = CO_CUST.cust_id inner join PLM_ITEM on pi_cust_item_recommend2.item_id=PLM_ITEM.item_id ;
Found ticket for [email protected] to go to krbtgt/[email protected] expiring on Fri Sep 08 10:48:34 CST 2017
Entered Krb5Context.initSecContext with state=STATE_NEW
Service ticket not found in the subject
Status: Running (Executing on YARN cluster with App id application_1503626611027_0229)
----------------------------------------------------------------------------------------------
VERTICES MODE STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
----------------------------------------------------------------------------------------------
Map 1 .......... container SUCCEEDED 2 2 0 0 0 0
Map 2 .......... container SUCCEEDED 2 2 0 0 0 0
Map 4 .......... container SUCCEEDED 1 1 0 0 0 0
Reducer 3 ...... container SUCCEEDED 12 12 0 0 0 0
----------------------------------------------------------------------------------------------
VERTICES: 04/04 [==========================>>] 100% ELAPSED TIME: 9.29 s
----------------------------------------------------------------------------------------------
Loading data to table yxpt.pi_cust_item_recommend8 partition (ymday="20170905")
Failed with exception java.util.ConcurrentModificationException
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask. java.util.ConcurrentModificationException
Found ticket for [email protected] to go to krbtgt/[email protected] expiring on Fri Sep 08 10:48:34 CST 2017
Entered Krb5Context.initSecContext with state=STATE_NEW
Service ticket not found in the subject
hive> select * from pi_cust_item_recommend8 limit 31;
OK
Time taken: 0.291 seconds
hive>
>
> insert into table pi_cust_item_recommend8 PARTITION (ymday='"20170905"') SELECT CAST(int(rand()*1000000) as varchar(30)),CO_CUST.nation_cust_code,PLM_ITEM.pack_bar,'20170905','10','TCSADMIN',pi_cust_item_recommend2.advise_level FROM pi_cust_item_recommend2 INNER JOIN CO_CUST ON pi_cust_item_recommend2.cust_id = CO_CUST.cust_id inner join PLM_ITEM on pi_cust_item_recommend2.item_id=PLM_ITEM.item_id ;
Found ticket for [email protected] to go to krbtgt/[email protected] expiring on Fri Sep 08 10:48:34 CST 2017
Entered Krb5Context.initSecContext with state=STATE_NEW
Service ticket not found in the subject
Status: Running (Executing on YARN cluster with App id application_1503626611027_0229)
----------------------------------------------------------------------------------------------
VERTICES MODE STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
----------------------------------------------------------------------------------------------
Map 1 .......... container SUCCEEDED 2 2 0 0 0 0
Map 2 .......... container SUCCEEDED 2 2 0 0 0 0
Map 4 .......... container SUCCEEDED 1 1 0 0 0 0
Reducer 3 ...... container SUCCEEDED 12 12 0 0 0 0
----------------------------------------------------------------------------------------------
VERTICES: 04/04 [==========================>>] 100% ELAPSED TIME: 11.27 s
----------------------------------------------------------------------------------------------
Loading data to table yxpt.pi_cust_item_recommend8 partition (ymday="20170905")
Failed with exception java.io.IOException: rename for src path: hdfs://bjychdfs/apps/hive/warehouse/yxpt.db/pi_cust_item_recommend8/ymday=%2220170905%22/.hive-staging_hive_2017-09-07_22-40-50_660_8567272627820500626-1/-ext-10000/000000_0 to dest path:hdfs://bjychdfs/apps/hive/warehouse/yxpt.db/pi_cust_item_recommend8/ymday=%2220170905%22/000000_0 returned false
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask. java.io.IOException: rename for src path: hdfs://bjychdfs/apps/hive/warehouse/yxpt.db/pi_cust_item_recommend8/ymday=%2220170905%22/.hive-staging_hive_2017-09-07_22-40-50_660_8567272627820500626-1/-ext-10000/000000_0 to dest path:hdfs://bjychdfs/apps/hive/warehouse/yxpt.db/pi_cust_item_recommend8/ymday=%2220170905%22/000000_0 returned false
Found ticket for [email protected] to go to krbtgt/[email protected] expiring on Fri Sep 08 10:48:34 CST 2017
Entered Krb5Context.initSecContext with state=STATE_NEW
Service ticket not found in the subject
hive> select * from pi_cust_item_recommend8 limit 3;
OK
Time taken: 0.29 seconds
hive>
>
> insert into table pi_cust_item_recommend8 PARTITION (ymday='"20170905"') SELECT CAST(int(rand()*1000000) as varchar(30)),CO_CUST.nation_cust_code,PLM_ITEM.pack_bar,'20170905','10','TCSADMIN',pi_cust_item_recommend2.advise_level FROM pi_cust_item_recommend2 INNER JOIN CO_CUST ON pi_cust_item_recommend2.cust_id = CO_CUST.cust_id inner join PLM_ITEM on pi_cust_item_recommend2.item_id=PLM_ITEM.item_id ;
Found ticket for [email protected] to go to krbtgt/[email protected] expiring on Fri Sep 08 10:48:34 CST 2017
Entered Krb5Context.initSecContext with state=STATE_NEW
Service ticket not found in the subject
Session re-established.
Status: Running (Executing on YARN cluster with App id application_1503626611027_0230)
----------------------------------------------------------------------------------------------
VERTICES MODE STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
----------------------------------------------------------------------------------------------
Map 1 .......... container SUCCEEDED 2 2 0 0 0 0
Map 2 .......... container SUCCEEDED 2 2 0 0 0 0
Map 4 .......... container SUCCEEDED 1 1 0 0 0 0
Reducer 3 ...... container SUCCEEDED 12 12 0 0 0 0
----------------------------------------------------------------------------------------------
VERTICES: 04/04 [==========================>>] 100% ELAPSED TIME: 11.08 s
----------------------------------------------------------------------------------------------
hive> select * from pi_cust_item_recommend8 limit 3;
OK
Time taken: 0.29 seconds
hive> insert into table pi_cust_item_recommend8 PARTITION (ymday='"20170907"') SELECT CAST(int(rand()*1000000) as varchar(30)),CO_CUST.nation_cust_code,PLM_ITEM.pack_bar,'20170905','10','TCSADMIN',pi_cust_item_recommend2.advise_level FROM pi_cust_item_recommend2 INNER JOIN CO_CUST ON pi_cust_item_recommend2.cust_id = CO_CUST.cust_id inner join PLM_ITEM on pi_cust_item_recommend2.item_id=PLM_ITEM.item_id ;
Found ticket for [email protected] to go to krbtgt/[email protected] expiring on Fri Sep 08 10:48:34 CST 2017
Entered Krb5Context.initSecContext with state=STATE_NEW
Service ticket not found in the subject
Status: Running (Executing on YARN cluster with App id application_1503626611027_0230)
----------------------------------------------------------------------------------------------
VERTICES MODE STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
----------------------------------------------------------------------------------------------
Map 1 .......... container SUCCEEDED 2 2 0 0 0 0
Map 2 .......... container SUCCEEDED 2 2 0 0 0 0
Map 4 .......... container SUCCEEDED 1 1 0 0 0 0
Reducer 3 ...... container SUCCEEDED 12 12 0 0 0 0
----------------------------------------------------------------------------------------------
VERTICES: 04/04 [==========================>>] 100% ELAPSED TIME: 8.86 s
----------------------------------------------------------------------------------------------
Loading data to table yxpt.pi_cust_item_recommend8 partition (ymday="20170907")
Failed with exception java.util.ConcurrentModificationException
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask. java.util.ConcurrentModificationException
Found ticket for [email protected] to go to krbtgt/[email protected] expiring on Fri Sep 08 10:48:34 CST 2017
Entered Krb5Context.initSecContext with state=STATE_NEW
Service ticket not found in the subject
hive> select * from pi_cust_item_recommend8 limit 3;
OK
Time taken: 0.307 seconds
hive> insert into table pi_cust_item_recommend8 PARTITION (ymday='"20170905"') SELECT CAST(int(rand()*1000000) as varchar(30)),CO_CUST.nation_cust_code,PLM_ITEM.pack_bar,'20170905','10','TCSADMIN',pi_cust_item_recommend2.advise_level FROM pi_cust_item_recommend2 INNER JOIN CO_CUST ON pi_cust_item_recommend2.cust_id = CO_CUST.cust_id inner join PLM_ITEM on pi_cust_item_recommend2.item_id=PLM_ITEM.item_id ;
Found ticket for [email protected] to go to krbtgt/[email protected] expiring on Fri Sep 08 10:48:34 CST 2017
Entered Krb5Context.initSecContext with state=STATE_NEW
Service ticket not found in the subject
Status: Running (Executing on YARN cluster with App id application_1503626611027_0230)
----------------------------------------------------------------------------------------------
VERTICES MODE STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
----------------------------------------------------------------------------------------------
Map 1 .......... container SUCCEEDED 2 2 0 0 0 0
Map 2 .......... container SUCCEEDED 2 2 0 0 0 0
Map 4 .......... container SUCCEEDED 1 1 0 0 0 0
Reducer 3 ...... container SUCCEEDED 12 12 0 0 0 0
----------------------------------------------------------------------------------------------
VERTICES: 04/04 [==========================>>] 100% ELAPSED TIME: 10.94 s
----------------------------------------------------------------------------------------------
Loading data to table yxpt.pi_cust_item_recommend8 partition (ymday="20170905")
Failed with exception java.io.IOException: rename for src path: hdfs://bjychdfs/apps/hive/warehouse/yxpt.db/pi_cust_item_recommend8/ymday=%2220170905%22/.hive-staging_hive_2017-09-07_22-51-17_746_1031556558801321637-1/-ext-10000/000000_0 to dest path:hdfs://bjychdfs/apps/hive/warehouse/yxpt.db/pi_cust_item_recommend8/ymday=%2220170905%22/000000_0 returned false
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask. java.io.IOException: rename for src path: hdfs://bjychdfs/apps/hive/warehouse/yxpt.db/pi_cust_item_recommend8/ymday=%2220170905%22/.hive-staging_hive_2017-09-07_22-51-17_746_1031556558801321637-1/-ext-10000/000000_0 to dest path:hdfs://bjychdfs/apps/hive/warehouse/yxpt.db/pi_cust_item_recommend8/ymday=%2220170905%22/000000_0 returned false
Found ticket for [email protected] to go to krbtgt/[email protected] expiring on Fri Sep 08 10:48:34 CST 2017
Entered Krb5Context.initSecContext with state=STATE_NEW
Service ticket not found in the subject
Entered Krb5Context.initSecContext with state=STATE_IN_PROCESS
>>> EType: sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType
Krb5Context setting peerSeqNumber to: 791237406
Krb5Context.unwrap: token=[05 04 01 ff 00 0c 00 00 00 00 00 00 2f 29 53 1e 01 01 00 00 33 9a 31 10 e0 8f 23 5c d2 b7 90 e1 ]
Krb5Context.unwrap: data=[01 01 00 00 ]
Krb5Context.wrap: data=[01 01 00 00 ]
Krb5Context.wrap: token=[05 04 00 ff 00 0c 00 00 00 00 00 00 19 12 e9 1e 01 01 00 00 4b 17 73 21 63 5c d9 d1 12 14 4a d3 ]
hive>
>
> select * from pi_cust_item_recommend8 limit 3;
OK
Time taken: 0.282 seconds
hive>
>
> insert into table pi_cust_item_recommend8 PARTITION (ymday='20170907') SELECT CAST(int(rand()*1000000) as varchar(30)),CO_CUST.nation_cust_code,PLM_ITEM.pack_bar,'20170905','10','TCSADMIN',pi_cust_item_recommend2.advise_level FROM pi_cust_item_recommend2 INNER JOIN CO_CUST ON pi_cust_item_recommend2.cust_id = CO_CUST.cust_id inner join PLM_ITEM on pi_cust_item_recommend2.item_id=PLM_ITEM.item_id ;
Found ticket for [email protected] to go to krbtgt/[email protected] expiring on Fri Sep 08 10:48:34 CST 2017
Entered Krb5Context.initSecContext with state=STATE_NEW
Service ticket not found in the subject
Status: Running (Executing on YARN cluster with App id application_1503626611027_0230)
----------------------------------------------------------------------------------------------
VERTICES MODE STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
----------------------------------------------------------------------------------------------
Map 1 .......... container SUCCEEDED 2 2 0 0 0 0
Map 2 .......... container SUCCEEDED 2 2 0 0 0 0
Map 4 .......... container SUCCEEDED 1 1 0 0 0 0
Reducer 3 ...... container SUCCEEDED 12 12 0 0 0 0
----------------------------------------------------------------------------------------------
VERTICES: 04/04 [==========================>>] 100% ELAPSED TIME: 9.06 s
----------------------------------------------------------------------------------------------
Loading data to table yxpt.pi_cust_item_recommend8 partition (ymday=20170907)
Failed with exception java.lang.ArrayIndexOutOfBoundsException: -3
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask. java.lang.ArrayIndexOutOfBoundsException: -3
hive> select * from pi_cust_item_recommend8 limit 3;
OK
Time taken: 0.282 seconds
hive> insert into table pi_cust_item_recommend8 PARTITION (ymday='20170907') SELECT CAST(int(rand()*1000000) as varchar(30)),CO_CUST.nation_cust_code,PLM_ITEM.pack_bar,'20170905','10','TCSADMIN',pi_cust_item_recommend2.advise_level FROM pi_cust_item_recommend2 INNER JOIN CO_CUST ON pi_cust_item_recommend2.cust_id = CO_CUST.cust_id inner join PLM_ITEM on pi_cust_item_recommend2.item_id=PLM_ITEM.item_id ;hive> use yxpt;
OK
Time taken: 0.274 seconds
hive> select * from pi_cust_item_recommend2 limit 3;
Found ticket for [email protected] to go to krbtgt/[email protected] expiring on Fri Sep 08 10:48:34 CST 2017
Entered Krb5Context.initSecContext with state=STATE_NEW
Service ticket not found in the subject
Entered Krb5Context.initSecContext with state=STATE_IN_PROCESS
>>> EType: sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType
Krb5Context setting peerSeqNumber to: 188448695
Krb5Context.unwrap: token=[05 04 01 ff 00 0c 00 00 00 00 00 00 0b 3b 7f b7 01 01 00 00 59 53 44 b9 a2 17 a4 04 5a ba 66 04 ]
Krb5Context.unwrap: data=[01 01 00 00 ]
Krb5Context.wrap: data=[01 01 00 00 ]
Krb5Context.wrap: token=[05 04 00 ff 00 0c 00 00 00 00 00 00 32 30 5e b9 01 01 00 00 a6 b7 c7 38 0c 5f 22 81 28 31 dd 87 ]
OK
110101104163 42010504 72.25313646387272 20170905
110101104163 90180101 22.481733024615185 20170905
110101104163 44510301 21.791753251469157 20170905
Time taken: 1.343 seconds, Fetched: 3 row(s)
hive>
> describe pi_cust_item_recommend7 ;
OK
cust_id string
item_id bigint
advise_level double
cust_code string
cgt_code string
notice varchar(3)
advise_id varchar(30)
Time taken: 0.181 seconds, Fetched: 7 row(s)
hive> exit;
Found ticket for [email protected] to go to krbtgt/[email protected] expiring on Fri Sep 08 10:48:34 CST 2017
Entered Krb5Context.initSecContext with state=STATE_NEW
Service ticket not found in the subject
hive> use yxpt;
OK
Time taken: 0.261 seconds
hive> select * from pi_cust_item_recommend8 limit 3;
OK
Time taken: 1.168 seconds
hive>
>
> insert into table pi_cust_item_recommend8 PARTITION (ymday='20170907') SELECT CAST(int(rand()*1000000) as varchar(30)),CO_CUST.nation_cust_code,PLM_ITEM.pack_bar,'20170905','10','TCSADMIN',pi_cust_item_recommend2.advise_level FROM pi_cust_item_recommend2 INNER JOIN CO_CUST ON pi_cust_item_recommend2.cust_id = CO_CUST.cust_id inner join PLM_ITEM on pi_cust_item_recommend2.item_id=PLM_ITEM.item_id ;
Found ticket for [email protected] to go to krbtgt/[email protected] expiring on Fri Sep 08 22:50:06 CST 2017
Entered Krb5Context.initSecContext with state=STATE_NEW
Service ticket not found in the subject
Session re-established.
Status: Running (Executing on YARN cluster with App id application_1503626611027_0232)
----------------------------------------------------------------------------------------------
VERTICES MODE STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
----------------------------------------------------------------------------------------------
Map 1 .......... container SUCCEEDED 2 2 0 0 0 0
Map 2 .......... container SUCCEEDED 2 2 0 0 0 0
Map 4 .......... container SUCCEEDED 1 1 0 0 0 0
Reducer 3 ...... container SUCCEEDED 12 12 0 0 0 0
----------------------------------------------------------------------------------------------
VERTICES: 04/04 [==========================>>] 100% ELAPSED TIME: 13.52 s
----------------------------------------------------------------------------------------------
Loading data to table yxpt.pi_cust_item_recommend8 partition (ymday=20170907)
Failed with exception java.io.IOException: rename for src path: hdfs://bjychdfs/apps/hive/warehouse/yxpt.db/pi_cust_item_recommend8/ymday=20170907/.hive-staging_hive_2017-09-07_22-59-13_617_377247445961069214-1/-ext-10000/000000_0 to dest path:hdfs://bjychdfs/apps/hive/warehouse/yxpt.db/pi_cust_item_recommend8/ymday=20170907/000000_0 returned false
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask. java.io.IOException: rename for src path: hdfs://bjychdfs/apps/hive/warehouse/yxpt.db/pi_cust_item_recommend8/ymday=20170907/.hive-staging_hive_2017-09-07_22-59-13_617_377247445961069214-1/-ext-10000/000000_0 to dest path:hdfs://bjychdfs/apps/hive/warehouse/yxpt.db/pi_cust_item_recommend8/ymday=20170907/000000_0 returned false
Found ticket for [email protected] to go to krbtgt/[email protected] expiring on Fri Sep 08 22:50:06 CST 2017
Entered Krb5Context.initSecContext with state=STATE_NEW
Service ticket not found in the subject
Time taken: 0.284 seconds
hive> select * from pi_cust_item_recommend8 limit 3;
OK
Time taken: 0.282 seconds
hive> drop table pi_cust_item_recommend8 ;
OK
Time taken: 2.087 seconds
hive> select * from pi_cust_item_recommend8 limit 3;
FAILED: SemanticException [Error 10001]: Line 1:14 Table not found 'pi_cust_item_recommend8'
hive>
>
> CREATE TABLE pi_cust_item_recommend8(ADVISE_ID VARCHAR(30),CUST_CODE VARCHAR(30),CGT_CODE VARCHAR(30),CRT_DATE VARCHAR(8),NOTICE VARCHAR(3),USER_ID VARCHAR(30),ADVISE_LEVEL decimal(5,3));
OK
Time taken: 0.191 seconds
hive> describe pi_cust_item_recommend8;
OK
advise_id varchar(30)
cust_code varchar(30)
cgt_code varchar(30)
crt_date varchar(8)
notice varchar(3)
user_id varchar(30)
advise_level decimal(5,3)
Time taken: 0.162 seconds, Fetched: 7 row(s)
hive>
>
> SELECT CAST(int(rand()*1000000) as varchar(30)),CO_CUST.nation_cust_code,PLM_ITEM.pack_bar,'20170905','10','TCSADMIN',pi_cust_item_recommend2.advise_level FROM pi_cust_item_recommend2 INNER JOIN CO_CUST ON pi_cust_item_recommend2.cust_id = CO_CUST.cust_id inner join PLM_ITEM on pi_cust_item_recommend2.item_id=PLM_ITEM.item_id limit 31 ;
Found ticket for [email protected] to go to krbtgt/[email protected] expiring on Fri Sep 08 22:50:06 CST 2017
Entered Krb5Context.initSecContext with state=STATE_NEW
Service ticket not found in the subject
Status: Running (Executing on YARN cluster with App id application_1503626611027_0233)
----------------------------------------------------------------------------------------------
VERTICES MODE STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
----------------------------------------------------------------------------------------------
Map 1 .......... container SUCCEEDED 2 2 0 0 0 0
Map 2 .......... container SUCCEEDED 2 2 0 0 0 0
Map 3 .......... container SUCCEEDED 1 1 0 0 0 0
----------------------------------------------------------------------------------------------
VERTICES: 03/03 [==========================>>] 100% ELAPSED TIME: 7.83 s
----------------------------------------------------------------------------------------------
OK
844998 110102206466 6901028940689 20170905 10 TCSADMIN 60.72310558804653
820643 110102206466 6901028118187 20170905 10 TCSADMIN 9.942863654959975
938813 110102206466 6901028316866 20170905 10 TCSADMIN 10.621125508964525
44271 110102206466 6901028315012 20170905 10 TCSADMIN 10.748528216651824
486 110102206466 6901028936750 20170905 10 TCSADMIN 10.756672311271405
731530 110102206466 6901028084864 20170905 10 TCSADMIN 11.134520882990092
654557 110102206466 6901028080446 20170905 10 TCSADMIN 11.153070464110803
34264 110102206466 6901028138536 20170905 10 TCSADMIN 11.366099729791326
885429 110102206466 6901028058148 20170905 10 TCSADMIN 11.603361435468745
111977 110102206466 6901028150378 20170905 10 TCSADMIN 11.689022768057503
990265 110102206466 6901028005036 20170905 10 TCSADMIN 11.882462449560231
543829 110102206466 6901028177153 20170905 10 TCSADMIN 11.979572435677673
593878 110102206466 6901028001687 20170905 10 TCSADMIN 12.160150106222419
541634 110102206466 6901028157629 20170905 10 TCSADMIN 12.827278813265025
705441 110102206466 6901028024976 20170905 10 TCSADMIN 13.198598635991594
726156 110102206466 6901028311168 20170905 10 TCSADMIN 14.145684045570716
767149 110102206466 6901028084895 20170905 10 TCSADMIN 14.543860630556074
24331 110102206466 6901028133487 20170905 10 TCSADMIN 14.69225879590099
341835 110102206466 6901028084741 20170905 10 TCSADMIN 17.126573594234525
492184 110102206466 6901028075039 20170905 10 TCSADMIN 18.50133249391921
620450 110102206466 6901028317269 20170905 10 TCSADMIN 18.681565978743667
653944 110102206466 6901028048828 20170905 10 TCSADMIN 19.414165825698127
927122 110102206466 6901028208352 20170905 10 TCSADMIN 19.750765686500053
499760 110102206466 6901028186582 20170905 10 TCSADMIN 19.917558066949805
250920 110102206466 6901028119009 20170905 10 TCSADMIN 20.44004660243949
220688 110102206466 6901028104050 20170905 10 TCSADMIN 21.493937559219983
946574 110102206466 6901028124942 20170905 10 TCSADMIN 21.532472887028405
338996 110102206466 6901028143059 20170905 10 TCSADMIN 22.578897017969844
794241 110102206466 6901028126687 20170905 10 TCSADMIN 23.66996101158705
263745 110102206466 4711588110956 20170905 10 TCSADMIN 38.67483685224603
779277 110112103885 6901028157629 20170905 10 TCSADMIN 76.04121085665929
Time taken: 14.945 seconds, Fetched: 31 row(s)
hive> insert into table pi_cust_item_recommend8 SELECT CAST(int(rand()*1000000) as varchar(30)),CO_CUST.nation_cust_code,PLM_ITEM.pack_bar,'20170905','10','TCSADMIN',pi_cust_item_recommend2.advise_level FROM pi_cust_item_recommend2 INNER JOIN CO_CUST ON pi_cust_item_recommend2.cust_id = CO_CUST.cust_id inner join PLM_ITEM on pi_cust_item_recommend2.item_id=PLM_ITEM.item_id ;
Found ticket for [email protected] to go to krbtgt/[email protected] expiring on Fri Sep 08 22:50:06 CST 2017
Entered Krb5Context.initSecContext with state=STATE_NEW
Service ticket not found in the subject
----------------------------------------------------------------------------------------------
VERTICES MODE STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
----------------------------------------------------------------------------------------------
Map 1 .......... container SUCCEEDED 2 2 0 0 0 0
Map 2 .......... container SUCCEEDED 2 2 0 0 0 0
Map 3 .......... container SUCCEEDED 1 1 0 0 0 0
----------------------------------------------------------------------------------------------
VERTICES: 03/03 [==========================>>] 100% ELAPSED TIME: 8.69 s
----------------------------------------------------------------------------------------------
Loading data to table yxpt.pi_cust_item_recommend8
OK
Time taken: 11.178 seconds
hive> select * from pi_cust_item_recommend8 limit 3;
OK
273125 110102206466 6901028940689 20170905 10 TCSADMIN 60.723
361292 110102206466 6901028118187 20170905 10 TCSADMIN 9.943
425037 110102206466 6901028316866 20170905 10 TCSADMIN 10.621
Time taken: 0.17 seconds, Fetched: 3 row(s)
hive> select * from pi_cust_item_recommend8 limit 31;
OK
273125 110102206466 6901028940689 20170905 10 TCSADMIN 60.723
361292 110102206466 6901028118187 20170905 10 TCSADMIN 9.943
425037 110102206466 6901028316866 20170905 10 TCSADMIN 10.621
442869 110102206466 6901028315012 20170905 10 TCSADMIN 10.749
645606 110102206466 6901028936750 20170905 10 TCSADMIN 10.757
446200 110102206466 6901028084864 20170905 10 TCSADMIN 11.135
70538 110102206466 6901028080446 20170905 10 TCSADMIN 11.153
413831 110102206466 6901028138536 20170905 10 TCSADMIN 11.366
816440 110102206466 6901028058148 20170905 10 TCSADMIN 11.603
375050 110102206466 6901028150378 20170905 10 TCSADMIN 11.689
326364 110102206466 6901028005036 20170905 10 TCSADMIN 11.882
413122 110102206466 6901028177153 20170905 10 TCSADMIN 11.980
493625 110102206466 6901028001687 20170905 10 TCSADMIN 12.160
708766 110102206466 6901028157629 20170905 10 TCSADMIN 12.827
71512 110102206466 6901028024976 20170905 10 TCSADMIN 13.199
806444 110102206466 6901028311168 20170905 10 TCSADMIN 14.146
949629 110102206466 6901028084895 20170905 10 TCSADMIN 14.544
306611 110102206466 6901028133487 20170905 10 TCSADMIN 14.692
237459 110102206466 6901028084741 20170905 10 TCSADMIN 17.127
151590 110102206466 6901028075039 20170905 10 TCSADMIN 18.501
438552 110102206466 6901028317269 20170905 10 TCSADMIN 18.682
255425 110102206466 6901028048828 20170905 10 TCSADMIN 19.414
911414 110102206466 6901028208352 20170905 10 TCSADMIN 19.751
554964 110102206466 6901028186582 20170905 10 TCSADMIN 19.918
23317 110102206466 6901028119009 20170905 10 TCSADMIN 20.440
522161 110102206466 6901028104050 20170905 10 TCSADMIN 21.494
28933 110102206466 6901028124942 20170905 10 TCSADMIN 21.532
683305 110102206466 6901028143059 20170905 10 TCSADMIN 22.579
505791 110102206466 6901028126687 20170905 10 TCSADMIN 23.670
912511 110102206466 4711588110956 20170905 10 TCSADMIN 38.675
802422 110112103885 6901028157629 20170905 10 TCSADMIN 76.041
Time taken: 0.169 seconds, Fetched: 31 row(s)
hive> describe pi_cust_item_recommend8;
OK
advise_id varchar(30)
cust_code varchar(30)
cgt_code varchar(30)
crt_date varchar(8)
notice varchar(3)
user_id varchar(30)
advise_level decimal(5,3)
Time taken: 0.154 seconds, Fetched: 7 row(s)
hive>
>
> exit;
Found ticket for [email protected] to go to krbtgt/[email protected] expiring on Fri Sep 08 22:50:06 CST 2017
Entered Krb5Context.initSecContext with state=STATE_NEW
Service ticket not found in the subject
Entered Krb5Context.initSecContext with state=STATE_IN_PROCESS
>>> EType: sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType
Krb5Context setting peerSeqNumber to: 37487493
Krb5Context.unwrap: token=[05 04 01 ff 00 0c 00 00 00 00 00 00 02 3c 03 85 01 01 00 00 d1 a8 96 02 52 b4 cd f1 b8 25 bf 05 ]
Krb5Context.unwrap: data=[01 01 00 00 ]
Krb5Context.wrap: data=[01 01 00 00 ]
Krb5Context.wrap: token=[05 04 00 ff 00 0c 00 00 00 00 00 00 20 22 82 75 01 01 00 00 f7 94 f0 9b ae a6 f9 7a 3d 45 84 2a ]
You have mail in /var/spool/mail/root
[superadmin@bjycbd-agent-23 root]$
[superadmin@bjycbd-agent-23 root]$
[superadmin@bjycbd-agent-23 root]$ sqoop export --connect jdbc:oracle:thin:@10.115.32.11:1521/xsm --username ecuser --password newpass --table EC_CUST_ADVISE_NOTICE --export-dir /apps/hive/warehouse/yxpt.db/pi_cust_item_recommend8 --columns advise_id,cust_code,cgt_code,crt_date,notice,user_id,advise_level --fields-terminated-by '\001' -m 1 --input-null-string '\\N' --input-null-non-string '\\N' -m 2 --update-mode allowinsert
Warning: /usr/inspur/2.5.0.0-1245/accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
find: failed to restore initial working directory: Permission denied
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/inspur/2.5.0.0-1245/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/inspur/2.5.0.0-1245/hbase/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/inspur/2.5.0.0-1245/zookeeper/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/inspur/2.5.0.0-1245/hive/lib/log4j-slf4j-impl-2.4.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/inspur/2.5.0.0-1245/hive/lib/spark-assembly-1.6.0-hadoop2.6.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
17/09/07 23:22:42 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6
17/09/07 23:22:42 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
17/09/07 23:22:42 INFO oracle.OraOopManagerFactory: Data Connector for Oracle and Hadoop is disabled.
17/09/07 23:22:42 INFO manager.SqlManager: Using default fetchSize of 1000
17/09/07 23:22:42 INFO tool.CodeGenTool: Beginning code generation
17/09/07 23:22:43 INFO manager.OracleManager: Time zone has been set to GMT
17/09/07 23:22:43 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM EC_CUST_ADVISE_NOTICE t WHERE 1=0
17/09/07 23:22:43 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/inspur/2.5.0.0-1245/hadoop
Note: /tmp/sqoop-superadmin/compile/13f5ade1005f3971df39832eb9ae0058/EC_CUST_ADVISE_NOTICE.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
17/09/07 23:22:45 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-superadmin/compile/13f5ade1005f3971df39832eb9ae0058/EC_CUST_ADVISE_NOTICE.jar
17/09/07 23:22:45 INFO mapreduce.ExportJobBase: Beginning export of EC_CUST_ADVISE_NOTICE
17/09/07 23:22:46 INFO impl.TimelineClientImpl: Timeline service address: http://bjycbd-agent-22.bjycbd.com:8188/ws/v1/timeline/
17/09/07 23:22:46 INFO client.ConfiguredRMFailoverProxyProvider: Failing over to rm2
17/09/07 23:22:46 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token 19579 for superadmin on ha-hdfs:bjychdfs
17/09/07 23:22:46 INFO security.TokenCache: Got dt for hdfs://bjychdfs; Kind: HDFS_DELEGATION_TOKEN, Service: ha-hdfs:bjychdfs, Ident: (HDFS_DELEGATION_TOKEN token 19579 for superadmin)
17/09/07 23:22:47 INFO input.FileInputFormat: Total input paths to process : 2
17/09/07 23:22:47 INFO input.FileInputFormat: Total input paths to process : 2
17/09/07 23:22:47 INFO mapreduce.JobSubmitter: number of splits:1
17/09/07 23:22:47 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1503626611027_0234
17/09/07 23:22:47 INFO mapreduce.JobSubmitter: Kind: HDFS_DELEGATION_TOKEN, Service: ha-hdfs:bjychdfs, Ident: (HDFS_DELEGATION_TOKEN token 19579 for superadmin)
17/09/07 23:22:48 INFO impl.YarnClientImpl: Submitted application application_1503626611027_0234
17/09/07 23:22:48 INFO mapreduce.Job: The url to track the job: http://bjycbd-agent-21.bjycbd.com:8088/proxy/application_1503626611027_0234/
17/09/07 23:22:48 INFO mapreduce.Job: Running job: job_1503626611027_0234
17/09/07 23:22:55 INFO mapreduce.Job: Job job_1503626611027_0234 running in uber mode : false
17/09/07 23:22:55 INFO mapreduce.Job: map 0% reduce 0%
17/09/07 23:23:07 INFO mapreduce.Job: map 49% reduce 0%
17/09/07 23:23:10 INFO mapreduce.Job: map 85% reduce 0%
17/09/07 23:23:12 INFO mapreduce.Job: map 100% reduce 0%
17/09/07 23:23:12 INFO mapreduce.Job: Job job_1503626611027_0234 completed successfully
17/09/07 23:23:12 INFO mapreduce.Job: Counters: 30
File System Counters
FILE: Number of bytes read=0
FILE: Number of bytes written=157070
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=29826806
HDFS: Number of bytes written=0
HDFS: Number of read operations=7
HDFS: Number of large read operations=0
HDFS: Number of write operations=0
Job Counters
Launched map tasks=1
Rack-local map tasks=1
Total time spent by all maps in occupied slots (ms)=141330
Total time spent by all reduces in occupied slots (ms)=0
Total time spent by all map tasks (ms)=14133
Total vcore-milliseconds taken by all map tasks=14133
Total megabyte-milliseconds taken by all map tasks=144721920
Map-Reduce Framework
Map input records=484243
Map output records=484243
Input split bytes=250
Spilled Records=0
Failed Shuffles=0
Merged Map outputs=0
GC time elapsed (ms)=87
CPU time spent (ms)=7730
Physical memory (bytes) snapshot=795185152
Virtual memory (bytes) snapshot=10964758528
Total committed heap usage (bytes)=864026624
File Input Format Counters
Bytes Read=0
File Output Format Counters
Bytes Written=0
17/09/07 23:23:12 INFO mapreduce.ExportJobBase: Transferred 28.4451 MB in 26.4389 seconds (1.0759 MB/sec)
17/09/07 23:23:12 INFO mapreduce.ExportJobBase: Exported 484243 records.
You have mail in /var/spool/mail/root