版权声明：本文为博主原创文章，未经博主允许不得转载。 https://blog.csdn.net/qq_14959801/article/details/77753361
hive> SELECT pi_cust_item_recommend2.cust_id, pi_cust_item_recommend2.item_id, pi_cust_item_recommend2.advise_level,pi_cust_item_recommend3.cust_code FROM pi_cust_item_recommend2, pi_cust_item_recommend3 WHERE pi_cust_item_recommend2.cust_id = pi_cust_item_recommend3.cust_id limit 3;
Query ID = hive_20170831094454_1602b289-d9bc-4201-85b2-31e76a408774
Total jobs = 1
Launching Job 1 out of 1
Tez session was closed. Reopening...
Session re-established.
Status: Running (Executing on YARN cluster with App id application_1503306179155_0060)
--------------------------------------------------------------------------------
VERTICES STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
--------------------------------------------------------------------------------
Map 1 .......... SUCCEEDED 4 4 0 0 0 0
Map 2 .......... SUCCEEDED 1 1 0 0 0 0
--------------------------------------------------------------------------------
VERTICES: 02/02 [==========================>>] 100% ELAPSED TIME: 5.11 s
--------------------------------------------------------------------------------
OK
110101100985 90190202 25.33671962331747 110101100985
110101100985 34030326 23.906456902069216 110101100985
110101100985 90020726 16.010945324507635 110101100985
Time taken: 10.947 seconds, Fetched: 3 row(s)
hive> SELECT pi_cust_item_recommend2.cust_id, pi_cust_item_recommend2.item_id, pi_cust_item_recommend2.advise_level,pi_cust_item_recommend4.pack_bar FROM pi_cust_item_recommend2, pi_cust_item_recommend4 WHERE pi_cust_item_recommend2.item_id = pi_cust_item_recommend4.item_id limit 3;
Query ID = hive_20170831094723_c1694a3b-0de7-4862-b41e-8477c8ae8c67
Total jobs = 1
Launching Job 1 out of 1
Status: Running (Executing on YARN cluster with App id application_1503306179155_0060)
--------------------------------------------------------------------------------
VERTICES STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
--------------------------------------------------------------------------------
Map 1 .......... SUCCEEDED 4 4 0 0 0 0
Map 2 .......... SUCCEEDED 3 3 0 0 0 0
--------------------------------------------------------------------------------
VERTICES: 02/02 [==========================>>] 100% ELAPSED TIME: 7.13 s
--------------------------------------------------------------------------------
OK
110101100985 90190202 25.33671962331747 1689674026938
110101100985 90190202 25.33671962331747 1689674026938
110101100985 90190202 25.33671962331747 1689674026938
Time taken: 7.49 seconds, Fetched: 3 row(s)
hive> SELECT pi_cust_item_recommend2.cust_id, pi_cust_item_recommend2.item_id, pi_cust_item_recommend2.advise_level,pi_cust_item_recommend4.pack_bar FROM pi_cust_item_recommend2, pi_cust_item_recommend4 WHERE pi_cust_item_recommend2.item_id = pi_cust_item_recommend4.item_id limit 5;
Query ID = hive_20170831095034_2b4da7df-6de1-4981-bb4e-e36b95ab220b
Total jobs = 1
Launching Job 1 out of 1
Status: Running (Executing on YARN cluster with App id application_1503306179155_0060)
--------------------------------------------------------------------------------
VERTICES STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
--------------------------------------------------------------------------------
Map 1 .......... SUCCEEDED 4 4 0 0 0 0
Map 2 .......... SUCCEEDED 3 3 0 0 0 0
--------------------------------------------------------------------------------
VERTICES: 02/02 [==========================>>] 100% ELAPSED TIME: 6.03 s
--------------------------------------------------------------------------------
OK
110101100985 90190202 25.33671962331747 1689674026938
110101100985 90190202 25.33671962331747 1689674026938
110101100985 90190202 25.33671962331747 1689674026938
110101100985 90190202 25.33671962331747 1689674026938
110101100985 90190202 25.33671962331747 1689674026938
Time taken: 6.503 seconds, Fetched: 5 row(s)
hive> select cust_id,item_id ,count(1) from pi_cust_item_recommend2 group by cust_id,item_id having count(1)>1 limit 5;
Query ID = hive_20170831095623_9e2ebddb-64f3-46ad-9687-c91d903a23a2
Total jobs = 1
Launching Job 1 out of 1
Status: Running (Executing on YARN cluster with App id application_1503306179155_0060)
--------------------------------------------------------------------------------
VERTICES STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
--------------------------------------------------------------------------------
Map 1 .......... SUCCEEDED 4 4 0 0 0 0
Reducer 2 ...... SUCCEEDED 1 1 0 0 0 0
--------------------------------------------------------------------------------
VERTICES: 02/02 [==========================>>] 100% ELAPSED TIME: 7.70 s
--------------------------------------------------------------------------------
OK
Time taken: 8.233 seconds
hive> select cust_id,item_id ,count(1) from pi_cust_item_recommend2 group by cust_id,item_id having count(1)>0 limit 5;
Query ID = hive_20170831095641_05e4fcfe-0238-49c5-8d36-62b4ea004c2b
Total jobs = 1
Launching Job 1 out of 1
Status: Running (Executing on YARN cluster with App id application_1503306179155_0060)
--------------------------------------------------------------------------------
VERTICES STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
--------------------------------------------------------------------------------
Map 1 .......... SUCCEEDED 4 4 0 0 0 0
Reducer 2 ...... SUCCEEDED 1 1 0 0 0 0
--------------------------------------------------------------------------------
VERTICES: 02/02 [==========================>>] 100% ELAPSED TIME: 9.11 s
--------------------------------------------------------------------------------
OK
110101100002 11016808 1
110101100002 22240103 1
110101100002 33010104 1
110101100002 34030316 1
110101100002 34030326 1
Time taken: 9.55 seconds, Fetched: 5 row(s)
hive> select cust_id,item_id ,count(1) from pi_cust_item_recommend2 group by cust_id,item_id having count(1)>0 limit 31;
Query ID = hive_20170831095659_dcf04f30-14f5-4c45-94a2-68c7927e1911
Total jobs = 1
Launching Job 1 out of 1
Status: Running (Executing on YARN cluster with App id application_1503306179155_0060)
--------------------------------------------------------------------------------
VERTICES STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
--------------------------------------------------------------------------------
Map 1 .......... SUCCEEDED 4 4 0 0 0 0
Reducer 2 ...... SUCCEEDED 1 1 0 0 0 0
--------------------------------------------------------------------------------
VERTICES: 02/02 [==========================>>] 100% ELAPSED TIME: 3.27 s
--------------------------------------------------------------------------------
OK
110101100002 11016808 1
110101100002 22240103 1
110101100002 33010104 1
110101100002 34030316 1
110101100002 34030326 1
110101100002 35510201 1
110101100002 41020211 1
110101100002 43010101 1
110101100002 51520615 1
110101100002 51520631 1
110101100002 51520635 1
110101100002 53010210 1
110101100002 53020107 1
110101100002 53020403 1
110101100002 53020411 1
110101100002 53020413 1
110101100002 53090115 1
110101100002 90020219 1
110101100002 90020733 1
110101100002 90020744 1
110101100002 90020749 1
110101100002 90020810 1
110101100002 90020923 1
110101100002 90021008 1
110101100002 90021010 1
110101100002 90021310 1
110101100002 90120106 1
110101100002 90130117 1
110101100002 90160250 1
110101100002 90160256 1
110101100004 12010504 1
Time taken: 3.739 seconds, Fetched: 31 row(s)
hive> SELECT pi_cust_item_recommend2.cust_id, pi_cust_item_recommend2.item_id, pi_cust_item_recommend2.advise_level,pi_cust_item_recommend4.pack_bar FROM pi_cust_item_recommend2, pi_cust_item_recommend4 WHERE pi_cust_item_recommend2.item_id = pi_cust_item_recommend4.item_id limit 31;
Query ID = hive_20170831095935_97bde958-7d1b-4db8-9497-5bffe984532e
Total jobs = 1
Launching Job 1 out of 1
Status: Running (Executing on YARN cluster with App id application_1503306179155_0060)
--------------------------------------------------------------------------------
VERTICES STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
--------------------------------------------------------------------------------
Map 1 .......... SUCCEEDED 4 4 0 0 0 0
Map 2 .......... SUCCEEDED 3 3 0 0 0 0
--------------------------------------------------------------------------------
VERTICES: 02/02 [==========================>>] 100% ELAPSED TIME: 6.06 s
--------------------------------------------------------------------------------
OK
110101100985 90190202 25.33671962331747 1689674026938
110101100985 90190202 25.33671962331747 1689674026938
110101100985 90190202 25.33671962331747 1689674026938
110101100985 90190202 25.33671962331747 1689674026938
110101100985 90190202 25.33671962331747 1689674026938
110101100985 90190202 25.33671962331747 1689674026938
110101100985 90190202 25.33671962331747 1689674026938
110101100985 90190202 25.33671962331747 1689674026938
110101100985 90190202 25.33671962331747 1689674026938
110101100985 90190202 25.33671962331747 1689674026938
110101100985 90190202 25.33671962331747 1689674026938
110101100985 90190202 25.33671962331747 1689674026938
110101100985 90190202 25.33671962331747 1689674026938
110101100985 90190202 25.33671962331747 1689674026938
110101100985 90190202 25.33671962331747 1689674026938
110101100985 90190202 25.33671962331747 1689674026938
110101100985 90190202 25.33671962331747 1689674026938
110101100985 90190202 25.33671962331747 1689674026938
110101100985 90190202 25.33671962331747 1689674026938
110101100985 90190202 25.33671962331747 1689674026938
110101100985 90190202 25.33671962331747 1689674026938
110101100985 90190202 25.33671962331747 1689674026938
110101100985 90190202 25.33671962331747 1689674026938
110101100985 90190202 25.33671962331747 1689674026938
110101100985 90190202 25.33671962331747 1689674026938
110101100985 90190202 25.33671962331747 1689674026938
110101100985 90190202 25.33671962331747 1689674026938
110101100985 90190202 25.33671962331747 1689674026938
110101100985 90190202 25.33671962331747 1689674026938
110101100985 90190202 25.33671962331747 1689674026938
110101100985 90190202 25.33671962331747 1689674026938
Time taken: 6.507 seconds, Fetched: 31 row(s)
hive> SELECT distinct pi_cust_item_recommend2.cust_id, pi_cust_item_recommend2.item_id, pi_cust_item_recommend2.advise_level,pi_cust_item_recommend4.pack_bar FROM pi_cust_item_recommend2, pi_cust_item_recommend4 WHERE pi_cust_item_recommend2.item_id = pi_cust_item_recommend4.item_id limit 31;
Query ID = hive_20170831100306_03d7c0b1-1d0d-4108-af9d-3dfee575fa52
Total jobs = 1
Launching Job 1 out of 1
Status: Running (Executing on YARN cluster with App id application_1503306179155_0060)
--------------------------------------------------------------------------------
VERTICES STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
--------------------------------------------------------------------------------
Map 1 .......... SUCCEEDED 4 4 0 0 0 0
Map 3 .......... SUCCEEDED 3 3 0 0 0 0
Reducer 2 ...... SUCCEEDED 1 1 0 0 0 0
--------------------------------------------------------------------------------
VERTICES: 03/03 [==========================>>] 100% ELAPSED TIME: 2170.37 s
--------------------------------------------------------------------------------
OK
110101100002 11016808 7.978577728743111 6901028072618
110101100002 22240103 8.323954405388776 6901028100670
110101100002 33010104 8.612861933764332 6901028118187
110101100002 34030316 12.935721589125322 6901028208550
110101100002 34030326 12.036239742227387 6901028124652
110101100002 35510201 9.076054512459226 6901028143080
110101100002 41020211 9.135519878477364 6901028169257
110101100002 43010101 8.306834856945622 6901028191142
110101100002 51520615 17.701307381060147 6901028032957
110101100002 51520631 10.815725844076724 6901028034982
110101100002 51520635 11.330324620008845 6901028085724
110101100002 53010210 7.886045450320436 6901028046893
110101100002 53020107 8.295806840703609 6901028317177
110101100002 53020403 9.23849773975783 6901028316316
110101100002 53020411 10.039130843774895 6901028315012
110101100002 53020413 9.285350499728555 6901028314985
110101100002 53090115 7.9394516094605745 6901028311465
110101100002 90020219 18.28473896224463 4893225033276
110101100002 90020733 23.221826879449523 4893225390287
110101100002 90020744 26.7786732939143 4893225390331
110101100002 90020749 13.016909772649832 8500001390534
110101100002 90020810 29.348530978821387 4893225041424
110101100002 90020923 12.555340665386616 4893225020443
110101100002 90021008 41.690187993777016 4893225013414
110101100002 90021010 25.380998391729122 4893225010499
110101100002 90021310 15.959681692923539 8500001540298
110101100002 90120106 11.01113428510216 4891132018881
110101100002 90130117 11.988138769820324 4004018602436
110101100002 90160250 9.137852053707537 7623500255539
110101100002 90160256 12.087923191225055 7623500276497
110101100004 12010504 10.140608521106833 6901028075138
Time taken: 2170.84 seconds, Fetched: 31 row(s)
hive> SELECT pi_cust_item_recommend2.cust_id,pi_cust_item_recommend2.item_id,pi_cust_item_recommend2.advise_level,pi_cust_item_recommend3.cust_code FROM pi_cust_item_recommend2 INNER JOIN pi_cust_item_recommend3 ON pi_cust_item_recommend2.cust_id=pi_cust_item_recommend3.cust_id limit 3;
Query ID = hive_20170831105430_24494375-3e47-4261-aa68-5f94172eba82
Total jobs = 1
Launching Job 1 out of 1
Tez session was closed. Reopening...
Session re-established.
Status: Running (Executing on YARN cluster with App id application_1503306179155_0061)
--------------------------------------------------------------------------------
VERTICES STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
--------------------------------------------------------------------------------
Map 1 .......... SUCCEEDED 4 4 0 0 0 0
Map 2 .......... SUCCEEDED 1 1 0 0 0 0
--------------------------------------------------------------------------------
VERTICES: 02/02 [==========================>>] 100% ELAPSED TIME: 4.45 s
--------------------------------------------------------------------------------
OK
110101100985 90190202 25.33671962331747 110101100985
110101100985 34030326 23.906456902069216 110101100985
110101100985 90020726 16.010945324507635 110101100985
Time taken: 9.022 seconds, Fetched: 3 row(s)
hive> describe pi_cust_item_recommend2;
OK
cust_id string
item_id bigint
advise_level double
ymday string
# Partition Information
# col_name data_type comment
ymday string
Time taken: 0.098 seconds, Fetched: 9 row(s)
hive> describe pi_cust_item_recommend3;
OK
cust_id string
cust_code string
Time taken: 0.122 seconds, Fetched: 2 row(s)
hive> describe pi_cust_item_recommend4;
OK
item_id bigint
pack_bar string
Time taken: 0.23 seconds, Fetched: 2 row(s)
hive> select * from pi_cust_item_recommend2 limit 3;
OK
110101100985 90190202 25.33671962331747 20170830
110101100985 34030326 23.906456902069216 20170830
110101100985 90020726 16.010945324507635 20170830
Time taken: 0.123 seconds, Fetched: 3 row(s)
hive>
> create table pi_cust_item_recommend5(
> cust_id string,
> item_id bigint,
> advise_level double,
> cust_code string);
OK
Time taken: 0.176 seconds
hive> insert into table yxpt.pi_cust_item_recommend5 SELECT pi_cust_item_recommend2.cust_id,pi_cust_item_recommend2.item_id,pi_cust_item_recommend2.advise_level,pi_cust_item_recommend3.cust_code FROM pi_cust_item_recommend2 INNER JOIN pi_cust_item_recommend3 ON pi_cust_item_recommend2.cust_id=pi_cust_item_recommend3.cust_id;
Query ID = hive_20170831113008_c7f61093-ab78-453c-8bf4-04e0b7c529ef
Total jobs = 1
Launching Job 1 out of 1
Tez session was closed. Reopening...
Session re-established.
Status: Running (Executing on YARN cluster with App id application_1503306179155_0062)
--------------------------------------------------------------------------------
VERTICES STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
--------------------------------------------------------------------------------
Map 1 .......... SUCCEEDED 4 4 0 0 0 0
Map 2 .......... SUCCEEDED 1 1 0 0 0 0
--------------------------------------------------------------------------------
VERTICES: 02/02 [==========================>>] 100% ELAPSED TIME: 7.46 s
--------------------------------------------------------------------------------
Loading data to table yxpt.pi_cust_item_recommend5
Failed with exception java.lang.NullPointerException
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask
hive> select * from pi_cust_item_recommend5 limit 3;
OK
110101100985 90190202 25.33671962331747 110101100985
110101100985 34030326 23.906456902069216 110101100985
110101100985 90020726 16.010945324507635 110101100985
Time taken: 0.141 seconds, Fetched: 3 row(s)
hive> select * from pi_cust_item_recommend5 limit 31;
OK
110101100985 90190202 25.33671962331747 110101100985
110101100985 34030326 23.906456902069216 110101100985
110101100985 90020726 16.010945324507635 110101100985
110101100985 90190101 15.628847628582498 110101100985
110101100985 90020727 15.442605374580097 110101100985
110101100985 90020214 13.276358249567217 110101100985
110101100985 90160248 12.218660323168212 110101100985
110101100985 90020222 11.55113640113565 110101100985
110101100985 90160257 11.001857833289318 110101100985
110101100985 53020411 10.808402849652841 110101100985
110101100985 90160255 10.55634487493214 110101100985
110101100985 43010101 10.293531517670374 110101100985
110101100985 53020502 9.967549321098497 110101100985
110101100985 34020101 9.52875123200268 110101100985
110101100985 34030228 9.479434291239455 110101100985
110101100985 37020308 9.472155882738065 110101100985
110101100985 90020927 9.112810136181182 110101100985
110101100985 90020934 9.112810136181182 110101100985
110101100985 90021202 9.112810136181182 110101100985
110101100985 33010104 9.041770412446649 110101100985
110101100985 90021408 8.76375384049508 110101100985
110101100985 41020211 8.753379761624661 110101100985
110101100985 53020413 8.720947985984836 110101100985
110101100985 32010101 8.66504292797535 110101100985
110101100985 13070505 8.536145306189216 110101100985
110101100985 53090115 8.490782608501519 110101100985
110101100985 22240103 8.456414691761115 110101100985
110101100985 53010210 8.330416055770293 110101100985
110101100985 12010504 8.319379734896234 110101100985
110101100985 35260104 8.090941846795186 110101100985
110101101012 34030316 45.461241571600425 110101101012
Time taken: 0.066 seconds, Fetched: 31 row(s)
hive> describe pi_cust_item_recommend5;
OK
cust_id string
item_id bigint
advise_level double
cust_code string
Time taken: 0.176 seconds, Fetched: 4 row(s)
hive>
> create table pi_cust_item_recommend6(
> cust_id string,
> item_id bigint,
> advise_level double,
> cust_code string,
> pack_bar string);
OK
Time taken: 0.319 seconds
hive> describe pi_cust_item_recommend6;
OK
cust_id string
item_id bigint
advise_level double
cust_code string
pack_bar string
Time taken: 0.145 seconds, Fetched: 5 row(s)
Time taken: 0.978 seconds
hive> select * from pi_cust_item_recommend6 limit 3;
OK
Time taken: 0.76 seconds
hive> describe pi_cust_item_recommend5;
OK
cust_id string
item_id bigint
advise_level double
cust_code string
Time taken: 0.079 seconds, Fetched: 4 row(s)
hive> select * from pi_cust_item_recommend5 limit 3;
OK
110101100985 90190202 25.33671962331747 110101100985
110101100985 34030326 23.906456902069216 110101100985
110101100985 90020726 16.010945324507635 110101100985
Time taken: 1.714 seconds, Fetched: 3 row(s)
hive>
> SELECT distinct pi_cust_item_recommend2.cust_id, pi_cust_item_recommend2.item_id, pi_cust_item_recommend2.advise_level,pi_cust_item_recommend4.pack_bar FROM pi_cust_item_recommend2, pi_cust_item_recommend4 WHERE pi_cust_item_recommend2.item_id = pi_cust_item_recommend4.item_id limit 31;
Query ID = hive_20170831151421_6a6da7b2-9939-48f7-a59d-9315e5728457
Total jobs = 1
Launching Job 1 out of 1
Status: Running (Executing on YARN cluster with App id application_1503306179155_0068)
--------------------------------------------------------------------------------
VERTICES STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
--------------------------------------------------------------------------------
Map 1 .......... SUCCEEDED 4 4 0 0 0 0
Map 3 .......... SUCCEEDED 3 3 0 0 0 0
Reducer 2 ...... SUCCEEDED 1 1 0 0 0 0
--------------------------------------------------------------------------------
VERTICES: 03/03 [==========================>>] 100% ELAPSED TIME: 2040.43 s
--------------------------------------------------------------------------------
OK
110101100002 11016808 7.978577728743111 6901028072618
110101100002 22240103 8.323954405388776 6901028100670
110101100002 33010104 8.612861933764332 6901028118187
110101100002 34030316 12.935721589125322 6901028208550
110101100002 34030326 12.036239742227387 6901028124652
110101100002 35510201 9.076054512459226 6901028143080
110101100002 41020211 9.135519878477364 6901028169257
110101100002 43010101 8.306834856945622 6901028191142
110101100002 51520615 17.701307381060147 6901028032957
110101100002 51520631 10.815725844076724 6901028034982
110101100002 51520635 11.330324620008845 6901028085724
110101100002 53010210 7.886045450320436 6901028046893
110101100002 53020107 8.295806840703609 6901028317177
110101100002 53020403 9.23849773975783 6901028316316
110101100002 53020411 10.039130843774895 6901028315012
110101100002 53020413 9.285350499728555 6901028314985
110101100002 53090115 7.9394516094605745 6901028311465
110101100002 90020219 18.28473896224463 4893225033276
110101100002 90020733 23.221826879449523 4893225390287
110101100002 90020744 26.7786732939143 4893225390331
110101100002 90020749 13.016909772649832 8500001390534
110101100002 90020810 29.348530978821387 4893225041424
110101100002 90020923 12.555340665386616 4893225020443
110101100002 90021008 41.690187993777016 4893225013414
110101100002 90021010 25.380998391729122 4893225010499
110101100002 90021310 15.959681692923539 8500001540298
110101100002 90120106 11.01113428510216 4891132018881
110101100002 90130117 11.988138769820324 4004018602436
110101100002 90160250 9.137852053707537 7623500255539
110101100002 90160256 12.087923191225055 7623500276497
110101100004 12010504 10.140608521106833 6901028075138
Time taken: 2042.786 seconds, Fetched: 31 row(s)>>> hiveCtx.sql("select a2.cust_id,a1.product,rating," + end_date +" date "+ " from all_score a1,id_cid a2 " + "where a1.user=a2.id").show(5)
17/08/30 10:18:22 WARN TaskSetManager: Stage 417 contains a task of very large size (3118 KB). The maximum recommended task size is 100 KB.
+------------+--------+------------------+--------+
| cust_id| product| rating| date|
+------------+--------+------------------+--------+
|110101100985|90190202| 25.33671962331747|20170830|
|110101100985|34030326|23.906456902069216|20170830|
|110101100985|90020726|16.010945324507635|20170830|
|110101100985|90190101|15.628847628582498|20170830|
|110101100985|90020727|15.442605374580097|20170830|
+------------+--------+------------------+--------+
only showing top 5 rows
>>> hiveCtx.sql("SELECT NATION_CUST_CODE FROM yxpt.CO_CUST limit 3").show(3)
+----------------+
|NATION_CUST_CODE|
+----------------+
| 110114190495|
| 110115203224|
| 110108209145|
+----------------+
>>>
>>> hiveCtx.sql("SELECT * from id_cid limit 3").show(3) +-----+------------+
| id| cust_id|
+-----+------------+
| 4549|110105106838|
|12992|110108207746|
|30968|110228100250|
+-----+------------+
>>> hiveCtx.sql("SELECT * from id_cid limit 5").show(5)
+-----+------------+
| id| cust_id|
+-----+------------+
| 4549|110105106838|
|12992|110108207746|
|30968|110228100250|
|22213|110114100048|
|19728|110113101105|
+-----+------------+
>>> hiveCtx.sql("select B.NATION_CUST_CODE CUST_CODE "+" from yxpt.CO_CUST B,id_cid A "+" where B.CUST_ID=A.CUST_ID limit 3").show(3)
+------------+
| CUST_CODE|
+------------+
|110105106838|
|110108207746|
|110228100250|
+------------+
>>> hiveCtx.sql("select B.NATION_CUST_CODE CUST_CODE "+" from yxpt.CO_CUST B,id_cid A "+" where B.CUST_ID=A.CUST_ID limit 3 ").show(3) +------------+
| CUST_CODE|
+------------+
|110105106838|
|110108207746|
|110228100250|
+------------+
>>> hiveCtx.sql("select C.PACK_BAR "+" from yxpt.PLM_ITEM C,all_score D "+" where C.ITEM_ID=D.product limit 3").show(3)
17/08/30 15:08:24 WARN TaskSetManager: Stage 448 contains a task of very large size (3118 KB). The maximum recommended task size is 100 KB.
+-------------+
| PACK_BAR|
+-------------+
|6901028032957|
|4893225033276|
|6901028208550|
+-------------+
>>> hiveCtx.sql("select * from all_score limit 5").show(3)
17/08/30 16:09:04 WARN TaskSetManager: Stage 465 contains a task of very large size (3118 KB). The maximum recommended task size is 100 KB.
+-----+--------+------------------+
| user| product| rating|
+-----+--------+------------------+
|27456|51520615| 35.52432167919441|
|27456|90020219|29.566229211420946|
|27456|34030316| 28.08260143903327|
+-----+--------+------------------+
only showing top 3 rows
>>> hiveCtx.sql("select CO_CUST_T.NATION_CUST_CODE CUST_CODE "+" from yxpt.CO_CUST CO_CUST_T,id_cid id_cid_t "+" where CO_CUST_T.CUST_ID=id_cid_t.CUST_ID limit 3").show(3)
+------------+
| CUST_CODE|
+------------+
|110105106838|
|110108207746|
|110228100250|
+------------+
>>> hiveCtx.sql("select PLM_ITEM_T.PACK_BAR "+" from yxpt.PLM_ITEM PLM_ITEM_T,all_score all_score_t "+" where PLM_ITEM_T.ITEM_ID=all_score_t.product limit 3").show(3)
17/08/30 18:32:13 WARN TaskSetManager: Stage 481 contains a task of very large size (3118 KB). The maximum recommended task size is 100 KB.
+-------------+
| PACK_BAR|
+-------------+
|6901028032957|
|4893225033276|
|6901028208550|
+-------------+
>>> hiveCtx.sql("select * from all_score limit 5").show(3) 17/08/30 23:53:47 WARN TaskSetManager: Stage 482 contains a task of very large size (3118 KB). The maximum recommended task size is 100 KB.
+-----+--------+------------------+
| user| product| rating|
+-----+--------+------------------+
|27456|51520615| 35.52432167919441|
|27456|90020219|29.566229211420946|
|27456|34030316| 28.08260143903327|
+-----+--------+------------------+
only showing top 3 rows
>>> hiveCtx.sql("SELECT * from id_cid limit 5").show(5)
+-----+------------+
| id| cust_id|
+-----+------------+
| 4549|110105106838|
|12992|110108207746|
|30968|110228100250|
|22213|110114100048|
|19728|110113101105|
+-----+------------+
>>> hiveCtx.sql("insert into table yxpt.pi_cust_item_recommend2 PARTITION (ymday='"+end_date +"') select a2.cust_id,a1.product,rating "+ " from all_score a1,id_cid a2 " + "where a1.user=a2.id")
17/08/31 00:13:05 WARN TaskSetManager: Stage 493 contains a task of very large size (3118 KB). The maximum recommended task size is 100 KB.
17/08/31 00:13:12 ERROR KeyProviderCache: Could not find uri with key [dfs.encryption.key.provider.uri] to create a keyProvider !!
DataFrame[]
>>> len(all_rating)
33695
>>> hiveCtx.sql("select CO_CUST_T.NATION_CUST_CODE CUST_CODE "+" from yxpt.CO_CUST CO_CUST_T,id_cid id_cid_t "+" where CO_CUST_T.CUST_ID=id_cid_t.CUST_ID limit 3").show(3)
+------------+
| CUST_CODE|
+------------+
|110105106838|
|110108207746|
|110228100250|
+------------+
>>> len(all_rating)
33695
>>> hiveCtx.sql("select B.NATION_CUST_CODE CUST_CODE "+" from yxpt.CO_CUST B,id_cid A "+" where B.CUST_ID=A.CUST_ID limit 3").show(3)
+------------+
| CUST_CODE|
+------------+
|110105106838|
|110108207746|
|110228100250|
+------------+
>>> hiveCtx.sql("select A.CUST_ID,B.NATION_CUST_CODE CUST_CODE "+" from yxpt.CO_CUST B,id_cid A "+" where B.CUST_ID=A.CUST_ID limit 3").show(3)
+------------+------------+
| CUST_ID| CUST_CODE|
+------------+------------+
|110105106838|110105106838|
|110108207746|110108207746|
|110228100250|110228100250|
+------------+------------+
>>> hiveCtx.sql("select D.product,C.PACK_BAR "+" from yxpt.PLM_ITEM C,all_score D "+" where C.ITEM_ID=D.product limit 3").show(3)
17/08/31 04:02:23 WARN TaskSetManager: Stage 514 contains a task of very large size (3118 KB). The maximum recommended task size is 100 KB.
+--------+-------------+
| product| PACK_BAR|
+--------+-------------+
|51520615|6901028032957|
|90020219|4893225033276|
|34030316|6901028208550|
+--------+-------------+
>>> hiveCtx.sql("select D.product,C.PACK_BAR "+" from yxpt.PLM_ITEM C,all_score D "+" where C.ITEM_ID=D.product limit 5").show(5)
17/08/31 04:02:50 WARN TaskSetManager: Stage 516 contains a task of very large size (3118 KB). The maximum recommended task size is 100 KB.
+--------+-------------+
| product| PACK_BAR|
+--------+-------------+
|51520615|6901028032957|
|90020219|4893225033276|
|34030316|6901028208550|
|90020923|4893225020443|
|51520635|6901028085724|
+--------+-------------+
>>> hiveCtx.sql("insert into table yxpt.pi_cust_item_recommend3 select A.CUST_ID,B.NATION_CUST_CODE CUST_CODE "+" from yxpt.CO_CUST B,id_cid A "+" where B.CUST_ID=A.CUST_ID ")
DataFrame[]
>>>
>>> hiveCtx.sql("insert into table yxpt.pi_cust_item_recommend4 select D.product,C.PACK_BAR "+" from yxpt.PLM_ITEM C,all_score D "+" where C.ITEM_ID=D.product ")
17/08/31 04:31:08 WARN TaskSetManager: Stage 524 contains a task of very large size (3118 KB). The maximum recommended task size is 100 KB.
DataFrame[]hive>
> describe pi_cust_item_recommend6;
OK
cust_id string
item_id bigint
advise_level double
cust_code string
pack_bar string
Time taken: 0.062 seconds, Fetched: 5 row(s)
hive>
> select * from pi_cust_item_recommend6 limit 3;
OK
Time taken: 0.525 seconds
hive>
>
>
> SELECT pi_cust_item_recommend2.cust_id,pi_cust_item_recommend2.item_id,pi_cust_item_recommend2.advise_level,CO_CUST.nation_cust_code,PLM_ITEM.pack_bar FROM pi_cust_item_recommend2 INNER JOIN CO_CUST ON pi_cust_item_recommend2.cust_id = CO_CUST.cust_id inner join PLM_ITEM on pi_cust_item_recommend2.item_id=PLM_ITEM.item_id limit 3;
WARNING: Comparing a bigint and a string may result in a loss of precision.
Query ID = superadmin_20170901205736_f32a6214-e816-4da4-8b9b-795d55f53b4e
Total jobs = 1
Launching Job 1 out of 1
Tez session was closed. Reopening...
Session re-established.
Status: Running (Executing on YARN cluster with App id application_1504257820142_0010)
--------------------------------------------------------------------------------
VERTICES STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
--------------------------------------------------------------------------------
Map 1 .......... SUCCEEDED 4 4 0 0 0 0
Map 2 .......... SUCCEEDED 6 6 0 0 0 0
Map 3 .......... SUCCEEDED 1 1 0 0 0 0
--------------------------------------------------------------------------------
VERTICES: 03/03 [==========================>>] 100% ELAPSED TIME: 11.05 s
--------------------------------------------------------------------------------
OK
110106200362 31010403 1.9991855986707237 110106200362 6901028075022
110106200362 44510318 0.7315507811472621 110106200362 6901028003865
110106200362 42010321 0.7402456032235841 110106200362 6901028188029
Time taken: 18.479 seconds, Fetched: 3 row(s)
hive> describe pi_cust_item_recommend6;
OK
cust_id string
item_id bigint
advise_level double
cust_code string
pack_bar string
Time taken: 0.113 seconds, Fetched: 5 row(s)
hive> select * from pi_cust_item_recommend6 limit 3;
OK
Time taken: 0.133 seconds
hive> SELECT pi_cust_item_recommend2.cust_id,pi_cust_item_recommend2.item_id,pi_cust_item_recommend2.advise_level,CO_CUST.nation_cust_code,PLM_ITEM.pack_bar FROM pi_cust_item_recommend2 INNER JOIN CO_CUST ON pi_cust_item_recommend2.cust_id = CO_CUST.cust_id inner join PLM_ITEM on pi_cust_item_recommend2.item_id=PLM_ITEM.item_id limit 31;
WARNING: Comparing a bigint and a string may result in a loss of precision.
Query ID = superadmin_20170901211431_3befe5c5-bee4-41dc-a5d7-ad1a540abf76
Total jobs = 1
Launching Job 1 out of 1
Tez session was closed. Reopening...
Session re-established.
Status: Running (Executing on YARN cluster with App id application_1504257820142_0011)
--------------------------------------------------------------------------------
VERTICES STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
--------------------------------------------------------------------------------
Map 1 .......... SUCCEEDED 4 4 0 0 0 0
Map 2 .......... SUCCEEDED 6 6 0 0 0 0
Map 3 .......... SUCCEEDED 1 1 0 0 0 0
--------------------------------------------------------------------------------
VERTICES: 03/03 [==========================>>] 100% ELAPSED TIME: 10.00 s
--------------------------------------------------------------------------------
OK
110106200362 31010403 1.9991855986707237 110106200362 6901028075022
110106200362 44510318 0.7315507811472621 110106200362 6901028003865
110106200362 42010321 0.7402456032235841 110106200362 6901028188029
110106200362 90073003 0.74945149296184 110106200362 7610083310519
110106200362 31010407 0.7514422934379361 110106200362 6901028075725
110106200362 90021011 0.7516502560651163 110106200362 4893225010222
110106200362 37020314 0.800165097468575 110106200362 6901028148603
110106200362 61060101 0.8055774565277998 110106200362 6901028936781
110106200362 34030227 0.8201827330212803 110106200362 6901028208949
110106200362 32010108 0.8269939671471814 110106200362 6901028112789
110106200362 53020107 0.8317866062546577 110106200362 6901028317177
110106200362 32010112 0.8355164358601257 110106200362 6901028111270
110106200362 32030207 0.835835989620561 110106200362 6901028116190
110106200362 53020103 0.8412134030668572 110106200362 6901028316866
110106200362 31010801 0.8523856164856713 110106200362 6901028075053
110106200362 43020105 0.8745519353980199 110106200362 6901028193504
110106200362 44510302 0.9226469340889978 110106200362 6901028001687
110106200362 53010222 0.928377558398128 110106200362 6901028051989
110106200362 32030206 0.9298263229410402 110106200362 6901028114455
110106200362 52010621 0.93037959088196 110106200362 6901028221450
110106200362 90020218 0.9395628276556189 110106200362 4893225030268
110106200362 43010114 0.9473769803304297 110106200362 6901028196932
110106200362 53020118 0.9494572814437028 110106200362 6901028054447
110106200362 42010324 0.9777206880170106 110106200362 6901028181457
110106200362 33010114 1.0105876536911125 110106200362 6901028119009
110106200362 31011101 1.0201943849707453 110106200362 6901028018166
110106200362 32010110 1.0847107783324719 110106200362 6901028062329
110106200362 32010111 1.236357228925462 110106200362 6901028111546
110106200362 31010401 1.7570995391536546 110106200362 6901028075015
110106200362 90020215 1.9264300616899914 110106200362 4893225033382
110106108637 90190202 30.167922792243232 110106108637 1689674026938
Time taken: 15.028 seconds, Fetched: 31 row(s)
hive> insert into table pi_cust_item_recommend6 SELECT pi_cust_item_recommend2.cust_id,pi_cust_item_recommend2.item_id,pi_cust_item_recommend2.advise_level,CO_CUST.nation_cust_code,PLM_ITEM.pack_bar FROM pi_cust_item_recommend2 INNER JOIN CO_CUST ON pi_cust_item_recommend2.cust_id = CO_CUST.cust_id inner join PLM_ITEM on pi_cust_item_recommend2.item_id=PLM_ITEM.item_id ;
WARNING: Comparing a bigint and a string may result in a loss of precision.
Query ID = superadmin_20170901212155_3ac172ff-d523-4ef8-a99c-e9719dc06a11
Total jobs = 1
Launching Job 1 out of 1
Status: Running (Executing on YARN cluster with App id application_1504257820142_0011)
--------------------------------------------------------------------------------
VERTICES STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
--------------------------------------------------------------------------------
Map 1 .......... SUCCEEDED 4 4 0 0 0 0
Map 2 .......... SUCCEEDED 6 6 0 0 0 0
Map 3 .......... SUCCEEDED 1 1 0 0 0 0
--------------------------------------------------------------------------------
VERTICES: 03/03 [==========================>>] 100% ELAPSED TIME: 10.41 s
--------------------------------------------------------------------------------
Loading data to table yxpt.pi_cust_item_recommend6
Table yxpt.pi_cust_item_recommend6 stats: [numFiles=6, numRows=1010850, totalSize=68073639, rawDataSize=67062789]
OK
Time taken: 11.501 seconds
hive> select * from pi_cust_item_recommend6 limit 31;
OK
110106200362 31010403 1.9991855986707237 110106200362 6901028075022
110106200362 44510318 0.7315507811472621 110106200362 6901028003865
110106200362 42010321 0.7402456032235841 110106200362 6901028188029
110106200362 90073003 0.74945149296184 110106200362 7610083310519
110106200362 31010407 0.7514422934379361 110106200362 6901028075725
110106200362 90021011 0.7516502560651163 110106200362 4893225010222
110106200362 37020314 0.800165097468575 110106200362 6901028148603
110106200362 61060101 0.8055774565277998 110106200362 6901028936781
110106200362 34030227 0.8201827330212803 110106200362 6901028208949
110106200362 32010108 0.8269939671471814 110106200362 6901028112789
110106200362 53020107 0.8317866062546577 110106200362 6901028317177
110106200362 32010112 0.8355164358601257 110106200362 6901028111270
110106200362 32030207 0.835835989620561 110106200362 6901028116190
110106200362 53020103 0.8412134030668572 110106200362 6901028316866
110106200362 31010801 0.8523856164856713 110106200362 6901028075053
110106200362 43020105 0.8745519353980199 110106200362 6901028193504
110106200362 44510302 0.9226469340889978 110106200362 6901028001687
110106200362 53010222 0.928377558398128 110106200362 6901028051989
110106200362 32030206 0.9298263229410402 110106200362 6901028114455
110106200362 52010621 0.93037959088196 110106200362 6901028221450
110106200362 90020218 0.9395628276556189 110106200362 4893225030268
110106200362 43010114 0.9473769803304297 110106200362 6901028196932
110106200362 53020118 0.9494572814437028 110106200362 6901028054447
110106200362 42010324 0.9777206880170106 110106200362 6901028181457
110106200362 33010114 1.0105876536911125 110106200362 6901028119009
110106200362 31011101 1.0201943849707453 110106200362 6901028018166
110106200362 32010110 1.0847107783324719 110106200362 6901028062329
110106200362 32010111 1.236357228925462 110106200362 6901028111546
110106200362 31010401 1.7570995391536546 110106200362 6901028075015
110106200362 90020215 1.9264300616899914 110106200362 4893225033382
110106108637 90190202 30.167922792243232 110106108637 1689674026938
Time taken: 0.106 seconds, Fetched: 31 row(s)
hive> select * from pi_cust_item_recommend6 limit 61;
OK
110106200362 31010403 1.9991855986707237 110106200362 6901028075022
110106200362 44510318 0.7315507811472621 110106200362 6901028003865
110106200362 42010321 0.7402456032235841 110106200362 6901028188029
110106200362 90073003 0.74945149296184 110106200362 7610083310519
110106200362 31010407 0.7514422934379361 110106200362 6901028075725
110106200362 90021011 0.7516502560651163 110106200362 4893225010222
110106200362 37020314 0.800165097468575 110106200362 6901028148603
110106200362 61060101 0.8055774565277998 110106200362 6901028936781
110106200362 34030227 0.8201827330212803 110106200362 6901028208949
110106200362 32010108 0.8269939671471814 110106200362 6901028112789
110106200362 53020107 0.8317866062546577 110106200362 6901028317177
110106200362 32010112 0.8355164358601257 110106200362 6901028111270
110106200362 32030207 0.835835989620561 110106200362 6901028116190
110106200362 53020103 0.8412134030668572 110106200362 6901028316866
110106200362 31010801 0.8523856164856713 110106200362 6901028075053
110106200362 43020105 0.8745519353980199 110106200362 6901028193504
110106200362 44510302 0.9226469340889978 110106200362 6901028001687
110106200362 53010222 0.928377558398128 110106200362 6901028051989
110106200362 32030206 0.9298263229410402 110106200362 6901028114455
110106200362 52010621 0.93037959088196 110106200362 6901028221450
110106200362 90020218 0.9395628276556189 110106200362 4893225030268
110106200362 43010114 0.9473769803304297 110106200362 6901028196932
110106200362 53020118 0.9494572814437028 110106200362 6901028054447
110106200362 42010324 0.9777206880170106 110106200362 6901028181457
110106200362 33010114 1.0105876536911125 110106200362 6901028119009
110106200362 31011101 1.0201943849707453 110106200362 6901028018166
110106200362 32010110 1.0847107783324719 110106200362 6901028062329
110106200362 32010111 1.236357228925462 110106200362 6901028111546
110106200362 31010401 1.7570995391536546 110106200362 6901028075015
110106200362 90020215 1.9264300616899914 110106200362 4893225033382
110106108637 90190202 30.167922792243232 110106108637 1689674026938
110106108637 13070505 10.020807206623058 110106108637 6901028080446
110106108637 33010104 10.058049127786315 110106108637 6901028118187
110106108637 34020101 10.064374620662141 110106108637 6901028126014
110106108637 37020308 10.08361484204497 110106108637 6901028157650
110106108637 53020403 10.087763635581382 110106108637 6901028316316
110106108637 53090115 10.113164089985663 110106108637 6901028311465
110106108637 43010101 10.154119923238216 110106108637 6901028191142
110106108637 53020502 10.243874016468709 110106108637 6901028047128
110106108637 31010801 10.262783482332498 110106108637 6901028075053
110106108637 90020934 10.311458471348512 110106108637 4893226001854
110106108637 90021202 10.311458471348512 110106108637 4893225280113
110106108637 90020927 10.311458471348512 110106108637 4893225020610
110106108637 32010101 10.341371682910385 110106108637 6901028300063
110106108637 31010401 10.422988186451343 110106108637 6901028075015
110106108637 31010403 10.473346577622062 110106108637 6901028075022
110106108637 53020107 10.487361995694918 110106108637 6901028317177
110106108637 90160248 10.561609411466996 110106108637 7623500177718
110106108637 32010112 11.097507392141555 110106108637 6901028111270
110106108637 90021408 12.128817829579909 110106108637 4893225060043
110106108637 90160257 12.710990271805933 110106108637 7623500177695
110106108637 90020727 13.491288455248247 110106108637 8500001541226
110106108637 90020726 13.910326650737815 110106108637 4893225390263
110106108637 90160255 14.889605020678538 110106108637 7623500261363
110106108637 51520635 14.948645805670303 110106108637 6901028085724
110106108637 90020222 18.60426240446597 110106108637 4893226002301
110106108637 90020923 18.960876041888074 110106108637 4893225020443
110106108637 90020217 19.55934180163587 110106108637 4893225030190
110106108637 34030316 20.385661736337465 110106108637 6901028208550
110106108637 90190101 21.293637158931105 110106108637 1689674030782
110114203948 34030316 39.099489197710426 110114203948 6901028208550
Time taken: 0.124 seconds, Fetched: 61 row(s)
hive> select count(distinct cust_id) from pi_cust_item_recommend6;
Query ID = superadmin_20170901214037_c63d32b2-0a56-4c23-9da1-76beac517d04
Total jobs = 1
Launching Job 1 out of 1
Tez session was closed. Reopening...
Session re-established.
Status: Running (Executing on YARN cluster with App id application_1504257820142_0012)
--------------------------------------------------------------------------------
VERTICES STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
--------------------------------------------------------------------------------
Map 1 .......... SUCCEEDED 6 6 0 0 0 0
Reducer 2 ...... SUCCEEDED 1 1 0 0 0 0
Reducer 3 ...... SUCCEEDED 1 1 0 0 0 0
--------------------------------------------------------------------------------
VERTICES: 03/03 [==========================>>] 100% ELAPSED TIME: 6.51 s
--------------------------------------------------------------------------------
OK
33695
Time taken: 11.704 seconds, Fetched: 1 row(s)
hive> describe pi_cust_item_recommend6;
OK
cust_id string
item_id bigint
advise_level double
cust_code string
pack_bar string
Time taken: 0.152 seconds, Fetched: 5 row(s)
hive>
>
> describe pi_cust_item_recommend7;
OK
cust_id string
item_id bigint
advise_level double
pack_bar string
Time taken: 0.055 seconds, Fetched: 4 row(s)
hive>
> describe pi_cust_item_recommend7;
OK
cust_id string
item_id bigint
advise_level double
pack_bar string
Time taken: 0.102 seconds, Fetched: 4 row(s)
hive>
>
> drop table if exists pi_cust_item_recommend7;
OK
Time taken: 0.176 seconds
hive> create table pi_cust_item_recommend7(
> cust_id string,
> item_id bigint,
> advise_level double,
> cust_code string,
> pack_bar string,
> NOTICE varchar(3),
> Display all 491 possibilities? (y or n)
> ADVISE_ID varchar(30)
> )
> ;
OK
Time taken: 0.458 seconds
hive> describe pi_cust_item_recommend7;
OK
cust_id string
item_id bigint
advise_level double
cust_code string
pack_bar string
notice varchar(3)
advise_id varchar(30)
Time taken: 0.109 seconds, Fetched: 7 row(s)
hive>
> insert into table pi_cust_item_recommend7 SELECT pi_cust_item_recommend2.cust_id,pi_cust_item_recommend2.item_id,pi_cust_item_recommend2.advise_level,CO_CUST.nation_cust_code,PLM_ITEM.pack_bar,10,int(rand()*1000000) FROM pi_cust_item_recommend2 INNER JOIN CO_CUST ON pi_cust_item_recommend2.cust_id = CO_CUST.cust_id inner join PLM_ITEM on pi_cust_item_recommend2.item_id=PLM_ITEM.item_id ;
WARNING: Comparing a bigint and a string may result in a loss of precision.
Query ID = superadmin_20170902120131_d856801b-78f6-4d96-aa55-0ca17c7a4dac
Total jobs = 1
Launching Job 1 out of 1
Tez session was closed. Reopening...
Session re-established.
Status: Running (Executing on YARN cluster with App id application_1504257820142_0016)
--------------------------------------------------------------------------------
VERTICES STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
--------------------------------------------------------------------------------
Map 1 .......... SUCCEEDED 4 4 0 0 0 0
Map 2 .......... SUCCEEDED 6 6 0 0 0 0
Map 3 .......... SUCCEEDED 1 1 0 0 0 0
--------------------------------------------------------------------------------
VERTICES: 03/03 [==========================>>] 100% ELAPSED TIME: 12.83 s
--------------------------------------------------------------------------------
Loading data to table yxpt.pi_cust_item_recommend7
Failed with exception org.apache.hadoop.hdfs.protocol.AclException: Invalid ACL: multiple entries with same scope, type and name.
at org.apache.hadoop.hdfs.server.namenode.AclTransformation.buildAndValidateAcl(AclTransformation.java:285)
at org.apache.hadoop.hdfs.server.namenode.AclTransformation.replaceAclEntries(AclTransformation.java:230)
at org.apache.hadoop.hdfs.server.namenode.FSDirAclOp.unprotectedSetAcl(FSDirAclOp.java:206)
at org.apache.hadoop.hdfs.server.namenode.FSDirAclOp.setAcl(FSDirAclOp.java:146)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.setAcl(FSNamesystem.java:8306)
at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.setAcl(NameNodeRpcServer.java:1966)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.setAcl(ClientNamenodeProtocolServerSideTranslatorPB.java:1338)
at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:640)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2313)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2309)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2307)
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask
hive> select * from pi_cust_item_recommend7 limit 3;
OK
110106200362 31010403 1.9991855986707237 110106200362 6901028075022 10 844281
110106200362 44510318 0.7315507811472621 110106200362 6901028003865 10 784173
110106200362 42010321 0.7402456032235841 110106200362 6901028188029 10 687349
Time taken: 1.232 seconds, Fetched: 3 row(s)
hive> describe pi_cust_item_recommend7 ;
OK
cust_id string
item_id bigint
advise_level double
cust_code string
pack_bar string
notice varchar(3)
advise_id varchar(30)
Time taken: 0.107 seconds, Fetched: 7 row(s)
hive>
> select * from pi_cust_item_recommend7 limit 31;
OK
110106200362 31010403 1.9991855986707237 110106200362 6901028075022 10 844281
110106200362 44510318 0.7315507811472621 110106200362 6901028003865 10 784173
110106200362 42010321 0.7402456032235841 110106200362 6901028188029 10 687349
110106200362 90073003 0.74945149296184 110106200362 7610083310519 10 134916
110106200362 31010407 0.7514422934379361 110106200362 6901028075725 10 324736
110106200362 90021011 0.7516502560651163 110106200362 4893225010222 10 231584
110106200362 37020314 0.800165097468575 110106200362 6901028148603 10 26455
110106200362 61060101 0.8055774565277998 110106200362 6901028936781 10 886206
110106200362 34030227 0.8201827330212803 110106200362 6901028208949 10 343477
110106200362 32010108 0.8269939671471814 110106200362 6901028112789 10 435245
110106200362 53020107 0.8317866062546577 110106200362 6901028317177 10 939005
110106200362 32010112 0.8355164358601257 110106200362 6901028111270 10 249572
110106200362 32030207 0.835835989620561 110106200362 6901028116190 10 498602
110106200362 53020103 0.8412134030668572 110106200362 6901028316866 10 637790
110106200362 31010801 0.8523856164856713 110106200362 6901028075053 10 905796
110106200362 43020105 0.8745519353980199 110106200362 6901028193504 10 908966
110106200362 44510302 0.9226469340889978 110106200362 6901028001687 10 695199
110106200362 53010222 0.928377558398128 110106200362 6901028051989 10 289106
110106200362 32030206 0.9298263229410402 110106200362 6901028114455 10 722128
110106200362 52010621 0.93037959088196 110106200362 6901028221450 10 490132
110106200362 90020218 0.9395628276556189 110106200362 4893225030268 10 184845
110106200362 43010114 0.9473769803304297 110106200362 6901028196932 10 249809
110106200362 53020118 0.9494572814437028 110106200362 6901028054447 10 225160
110106200362 42010324 0.9777206880170106 110106200362 6901028181457 10 913569
110106200362 33010114 1.0105876536911125 110106200362 6901028119009 10 640293
110106200362 31011101 1.0201943849707453 110106200362 6901028018166 10 759850
110106200362 32010110 1.0847107783324719 110106200362 6901028062329 10 42010
110106200362 32010111 1.236357228925462 110106200362 6901028111546 10 3153
110106200362 31010401 1.7570995391536546 110106200362 6901028075015 10 579454
110106200362 90020215 1.9264300616899914 110106200362 4893225033382 10 295173
110106108637 90190202 30.167922792243232 110106108637 1689674026938 10 34512
Time taken: 0.176 seconds, Fetched: 31 row(s)