01 Let's learn big data together
Lao Liu is here! After the senior brothers and sisters in the laboratory are looking for jobs, after finishing all kinds of scientific research work, Lao Liu now has to work hard to find a job, and he wants to start reviewing and summarizing various knowledge points of big data. Lao Liu will share a summary of his knowledge. First, he hopes to get criticism and guidance from big guys, and second, he hopes to help students who want to learn big data!
As for why I shared the summary of ZooKeeper's knowledge points first, because Lao Liu only finished reading this today, he must take advantage of the hot iron, and quickly keep it in his mind! All the knowledge points, Lao Liu adopts a colloquial way.
02 Summary of knowledge points
Point 1: What is ZooKeeper?
This is very important. The first thing you need to understand when learning anything is what it is. If you don't even know what it is, that's weird!
In short, Zookeeper is a distributed coordination framework. Note that it is a coordination framework. There will be a problem in kafka0.8. About this coordination framework, we will talk about it later, let's pay attention to it. It is used to solve the problem of system consistency in a distributed system. Zookeeper can be used to achieve high availability of Hadoop, and the offset of kafka can be stored in Zookeeper, but there will be a problem here. When it comes to kafka, I will talk about this problem.
Point 2: Why use Zookeeper?
For this problem, for example, in the high availability of Hadoop, if Zookeeper is not used, then how to switch between its two NameNodes, you must write your own programs to switch, which will cause distraction, and hadoop itself is the storage Kind of, now I want my hadoop to consider the logic of high availability, it is too tired, can I let others share my pressure. So use Zookeeper to help Hadoop share the pressure and let Hadoop focus.
Point 3: Some command line operations of Zookeeper
# 启动ZooKeeper集群；在ZooKeeper集群中的每个节点执行此命令
zkServer.sh start
# 停止ZooKeeper集群（每个节点执行以下命令）
zkServer.sh stop
# 查看集群状态（每个节点执行此命令）
zkServer.sh status
# 连接ZooKeeper的服务器
zkCli.sh -server node01:2181,node02:2181,node03:2181
#查看ZooKeeper根目录/下的文件列表
ls /
#创建节点，并指定数据
create /lao liu
#获得某节点的数据
get /lao
#修改节点的数据
set /lao liu1
#删除节点
delete /lao
Point 4: Carrying out JAVA API in IDEA, this must be remembered and must not be ignored!
I won’t share this, too much. If you want, please contact Lao Liu directly and Lao Liu will give it to you.
Point 5: The composition of ZooKeeper
It has three parts, one is a directory node tree similar to a file system to store data, the other is some basic commands for operating ZooKeeper, and the third is its watcher Watcher.
What is a directory node tree similar to a file system?
So what is a listener?
Here is a brief introduction to some concepts of the listener. The listener has three parts: (1) Registration: the client registers the listener with the ZooKeeper cluster (2) Monitoring events: the listener is responsible for listening for some events (3) Callback function : When the listener listens to the event, it will call the callback function.
Let me give you an example. I saw it in some organizations. It feels more classic. I will show it to everyone.
1、一哥们去酒店办理入住，但是被告知目前无空房
2、这哥们告诉客服：帮我留意一下有没有空出的房间，如果有，就及时通知我
（相当于注册监听器，监听一些事件）
3、将近12点，有房客退房，有空闲的房间（发现要监听的事件）
4、客服发现有空房（监听到了事件）
5、及时通知这哥们
6、这哥们收到通知后，马上赶回酒店（就相当于调用了回调函数）
Point 6: The data node ZNode in ZooKeeper
ZNode is divided into 4 categories: persistent nodes, temporary nodes, non-ordered nodes, ordered nodes, and
persistent nodes are equivalent to directories in the file directory system.
The temporary node is bound to the session. When the client is connected to the ZooKeeper cluster, it is equivalent to establishing a session. When the connection is disconnected, the session becomes invalid, and the temporary node becomes invalid and deleted.
Then why are there still ordered nodes?
This is to prevent multiple clients from creating ZNodes with the same name in the same directory. If they have the same name, an error will be reported, so there is an ordered node. When the ordered node is created, ZooKeeper will automatically append these nodes. The previous integer incremented number.
Point 7: What is a session?
The client establishes a long TCP connection with a node in the ZooKeeper cluster, which is called a session. In a session, requests are executed in FIFO (first in, first out) order. For example, after the client sends a create request first, and then sends a get request, then during execution, create is executed first, and then get is executed. . However, if there are two sessions, there is no way to guarantee the FIFO between the two sessions. It can only guarantee the FIFO in one session. This old Liu doesn't know the specific reason, but just wrote it down, hahaha!
Point 8: What is a transaction?
Lao Liu hasn't understood the affairs in ZooKeeper clearly, so I won't write it here. Everyone will Baidu on their own, and Lao Liu will think of it one day, and then make up.
Point 9: What is Watcher?
Old Liu briefly talked about the Watcher example at point 5. Now let’s talk about its usefulness. First, ask a question. After the client establishes a connection with ZooKeeper, how does the client get the latest data in Zookeeper? How does it perceive data changes? That is through the watcher introduced. Liu gave an example of watcher in point 5. Old Liu thinks it is very classic. Through that example, we can understand that the watcher is the listener registered by the client on the ZooKeeper server, and it is used to monitor the data changes on the data node ZNode, and then tell the client the latest news.
03 Summary
Today is almost the end. The next part is all about the architecture principles of ZooKeeper. It is more difficult to understand. Lao Liu tries to write it in a colloquial way. I hope students who are interested in big data development will cheer up with Lao Liu! Let's go to Dachang together!
Finally, there is something, the official account: Lao Liu who works hard, contact me! It's okay, just start learning big data with Liu.