Encounter a problem, optimize our sqoop number of pumping to extract the mysql data source table is a partitioned table, but our field is not extracted incremental partition field, table data about 500 million, resulting in a very slow query , to find ways to improve efficiency.
(1) If you know the name of the partition , you can directly query the corresponding partition name
select * from CM.loan_shu_api_result partition(SYS_P505)
(2) If you do not know the name of the partition , but know that the partition of the primary key field value range , can be queried based on the partitioning range
select * from CM.loan_shu_api_result where default_dt>=to_date('2019-02-01','yyyy-mm-dd') and default_dt<to_date('2019-03-01','yyyy-mm-dd')
and substr(call_dt,1,4)||substr(call_dt,6,2)||substr(call_dt,9,2) ='20190220';
Description: default_dt Partition field, call_dt incremental extraction field
(3) If you do not know the name of the partition, the partition did not know the primary key field value range , you can use PARTITION FOR clause of a query, such as now only know that this is a data partition 2019-02-15
select * from CM.loan_shu_api_result partition for(to_date('2019-02-15','yyyy-mm-dd'));