Hive of parameter setting mode
1, the configuration file (global valid)
2, the command line parameters (for example promoter active hive)
3, parameter declarations (connected to the hive valid session)
(1) Profile
Hive profile include:
A. User-defined configuration file: $ HIVE_CONF_DIR / hive-site.xml
B. The default configuration file: $ HIVE_CONF_DIR / hive-default.xml
User-defined configuration will override the default configuration.
In addition, Hadoop Hive will be read into the configuration, because as a client Hadoop Hive is initiated, the configuration will overwrite the Hive Hadoop configuration.
Set profiles are valid for all native Hive process started.
(2) command line parameters
Hive startup (client or Server mode), the row may be added -hiveconf param = value in order to set the parameters, the setting of the start of the current session (server mode for the start, it is all session requests) effective .
bin/hive -hiveconf hive.root.logger=INFO,console
(3) parameter declaration
You can use the SET keyword set parameters in the HQL
Dynamic partitions (dynamic partitioning feature is turned on, nonstrict mode partitions allowing all fields can use dynamic partitioning):
set hive.exec.dynamic.partition=true; set hive.exec.dynamic.partition.mode=nonstrict;
Parallel Computing (sql the same whether a different job can be run simultaneously, while the maximum value can run job):
set hive.exec.parallel=true; set hive.exec.parallel.thread.number=8;
map join (small table to solve the problems associated with data skew caused by a large table with a map join)
map join concepts: a small table in which (a total amount of data) distributed to all MapTask for Join connecting end, thereby avoiding ReduceTask, provided that the total amount of data requires enough memory to hold
SET hive.auto.convert.join = to true ; // set MapJoin optimized automatically open SET hive.mapjoin.smalltable.filesize = 25000000 or // set small tables opened optimization does not exceed much mapjoin