Hive table partitioning

basic knowledge:

    Hadoop: File-related operations such as:

     hadoop fs -rmr 'hdfs://hdfs://192.168.8.101:8020/user/hive/warehouse';

     hadoop fs -put '/user/hive/warehouse/data.txt' 'hdfs://hdfs://192.168.8.101:8020/user/hive/warehouse/data.txt'

 

     Create a partitioned table:

   

   External table:

create external table if not exists employee(
id int,
name string,
dept string,
yoj int
)
partitioned by (
year string
)
row format delimited fields terminated by '\t'

   

  Internal table:

  

create table if not exists employee(
id int,
name string,
dept string,
yoj int
)
partitioned by (
year string
)
row format delimited fields terminated by '\t'

 

 

   Add partition and load data

    1. For external tables

       <1  alter table employee add partition (year='2013') location 'hdfs://192.168.8.101:8020/user/hive/warehouse/hivedata/data';    the data under the hivedata / folder will not be be moved. and no partition directory year=2013

      alter table  employee  drop partition ( year='2013' ); The data under hivedata / will not be deleted when the partition directory is deleted

 

  <2 load data inpath 'hdfs://192.168.8.101:8020/user/hive/warehouse/hivedata/data' overwrite into table employeepartition(year='2013');        

The data under the hivedata / folder will be moved when the data is     loaded and the partition is added  , and the partition directory logdate=2015-02-26 will be created, and the data will be moved to this directory

        alter table employee drop partition ( year=' 2013 ' ); When executing the delete partition directory, year= 2013 has been created

The partition directory will not be deleted, and the data under its folder will not be deleted;

    

    2, internal table

         alter table  employee  add partition ( year='2013' ) location ' hdfs://192.168.8.101:8020/user/hive/warehouse/hivedata/data '; data under hivedata / folder will not be moved when adding partition    . and no partition directory year=2013

       alter table employee drop partition (year='2013'); The data under hivedata / will be deleted and the hivedata / folder will also be deleted when the partition is deleted

      

       load data inpath ' hdfs://192.168.8.101:8020/user/hive/warehouse/hivedata/data ' overwrite into table eml_inn partition(year=2013);    
    When executing load data to add partition,  the data in the hivedata / folder will be Move, and create a partition directory year=2013 , and move the data to this directory

        alter table employee drop partition ( year='2013' ); When deleting the partition directory, the created year=2013   partition directory will be deleted, and the data in the folder will be deleted accordingly;