In turn enter the command ssh service is turned on, open mysql, start hdfs, start yarm, start Zookeepr
Enter the hive into the hive
Create a table to put the data: create table kafkatest (id int, name string, age int) clustered by (id) into 2 buckets stored as orc tblproperties ( 'transactional' = 'true');
Then enter exit; to exit the hive
Restart a terminal, input cd ~ / bigdata / kafka_2.11-1.0.0 kafka enter the directory.
输入:nohup bin/kafka-server-start.sh config/server.properties >~/bigdata/kafka_2.11-1.0.0/logs/server.log 2>&1 &
Background start kafka.
Input bin / kafka-topics.sh --create --zookeeper localhost: 2181 --replication-factor 1 --partitions 1 --topic kafkatest create a kafka the topic.
Then start a terminal, enter: cd ~ / bigdata / apache-flume-1.9.0-bin
Flume into the directory, enter: bin/flume-ng agent --conf conf/ --conf-file conf/kafkatoh
start flume.
Back to the terminal (second terminal opened), the input of the next directory kafka: bin / kafka-console-producer.sh --broker-list localhost: 9092 --topic kafkatest start kafka the consumer.
向启动好的kafka中输入1,a,3(1对应hive表中的id,a对应name,3对应age,可以进行适当更换)。发现flume的终端在执行任务。
最后启动一个新的终端,输入hive,输入select * from kafkatest;可以查看到刚才键入的数据。
输入exit;退出hive。