Generally, we analyze big data, you might think Spark, Storm, but only have to be JAVA and other programming languages, or can not get the data for analysis. The Hive solves this problem will only need to do Sql language mapreduce of big data analysis tasks. Today we create the actual test data analysis mapreduce with Hive.
First, the first installed Hive, Mysql environment
1, based on yesterday hdfs, install Hive, Mysql.
2, dependent on the result of Hive Mysql storing metadata dependent on the data Hdfs memory cell, configured Mysql account, the password Hive.
3. Create a database in Mysql Hive, a new Mysql database account password.
Second, create test data
1. Create a file of test data
cd /usr/local/hadoop
mkdir input
echo "hello world" >file1.txt
echo "hello hadoop" >file2.txt
2, into the hive, is introduced into the hive file data table
hive> create table docs (line string); // Create a table with only one column element
hive> load data local input '/ usr / local / hadoop / input' overwrite into table docs; // the directory input file1, file2 poured hive docs table.
Pour data confirmed success with select * from docs.
Third, create test and analysis tasks
1, analyzed and processed with a word count of sql statement.
create table word_count as select word,count(1) as count from (select explode(split(line,' ')) as word from docs) w group by word order by word;
2, check the task mapreduce
Mapreduce examination process, the above statement is started mapreduce task.
3, test results
With select * from word_count, find word count of the task completed successfully.
IV Summary
1, hive mysql deposit depends on the metadata stored in the database file depends on hdfs, a cheap data warehouse.
2, hive mapreduce create tasks using sql statement applies to non-real-time data analysis and processing.
I hope this article can help you.
More real-time updates, visit public number. 
Click here to get the highest ¥ 1888 Ali cloud offering universal vouchers