A, Zookeeper installation and set up
Server clusters and cluster registry is not the same:
In order to address the high concurrency server cluster, cluster registry
1.1, windows installation environment
Environmental requirements: must have jdk environment, this lecture using jdk1.8 ( Zookeeper is written in Java, and the structure is similar to XML )
1. Install jdk
2. Install Zookeeper. Official website http://zookeeper.apache.org/ download zookeeper. I downloaded the zookeeper-3.4.6 version.
Decompression zookeeper-3.4.6 to D: \ machine \ zookeeper-3.4.6.
In D: \ machine new data and log directory.
3.ZooKeeper installation mode is divided into three, namely: standalone mode (stand-alone), the cluster and cluster dummy pattern distribution pattern. ZooKeeper installation is relatively simple stand-alone mode, if the first contact ZooKeeper, it is recommended to install ZooKeeper dummy stand-alone mode, or a cluster distribution pattern.
Click install mode. To D: \ machine \ zookeeper-3.4.6 \ conf zoo_sample.cfg copy and paste it into the current directory, named zoo.cfg.
1.2, Zookeeper build a cluster environment (Linux)
Environmental requirements: must have jdk environment, this lecture using jdk1.8
(1) Structure
A total of three nodes (zk server cluster size not less than 3 nodes), the time between the server system requires consistent
(2) upload and unzip zk
Decompressing: tar -zxvf zookeeper-3.4.6.tar.gz
rename: mv zookeeper-3.4.6 zookeeper
(3) modify environment variables zookeeper
vi /etc/profile
export JAVA_HOME=/opt/jdk1.8.0_71
export ZOOKEEPER_HOME=/usr/local/zookeeper
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
export PATH=$JAVA_HOME/bin:$ZOOKEEPER_HOME/bin:$PATH
source /etc/profile
(4) modify zoo_sample.cfg file
cd /usr/local/zookeeper/conf
mv zoo_sample.cfg zoo.cfg
修改conf: vi zoo.cfg 修改两处
（1） dataDir=/usr/local/zookeeper/data（注意同时在zookeeper创建data目录）
（2）最后面添加
server.0=bhz:2888:3888
server.1=hadoop1:2888:3888
server.2=hadoop2:2888:3888
(5) create a server identity
Server identifies the configuration:
Create a folder: mkdir data
to create a file and fill out the contents myid 0: vi
myid (content server ID: 0)
(6) Copy the zookeeper
Zookeeper directory to copy and hadoop02 hadoop01
well / etc / profile file
to hadoop01, myid file in hadoop02 modify the value 1 and 2
paths (vi / usr / local / zookeeper / data / myid)
(7) starts zookeeper
启动zookeeper：
路径： /usr/local/zookeeper/bin
执行： zkServer.sh start
(注意这里3台机器都要进行启动)
状态： zkServer.sh
status(在三个节点上检验zk的mode,一个leader和俩个follower)
1.3, Zookeeper profile introduction
# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just
# example sakes.
dataDir=/home/myuser/zooA/data
# the port at which the clients will connect
clientPort=2181
# ZooKeeper server and its port no. # ZooKeeper ensemble should know about every other machine in the ensemble # specify server id by creating 'myid' file in the dataDir # use hostname instead of IP address for convenient maintenance
server.1=127.0.0.1:2888:3888
server.2=127.0.0.1:2988:3988
server.3=127.0.0.1:2088:3088
#
# Be sure to read the maintenance section of the
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
# autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature <br>
#autopurge.purgeInterval=1
dataLogDir=/home/myuser/zooA/log
(1) tickTime: heartbeat time, to ensure that there is connected, in milliseconds, the minimum time-out time is two heartbeat
(2) initLimit: the number of heartbeat time, and allow other initialization data server connection, if the data management ZooKeeper large, this value should be increased accordingly
(3) clientPort: listening port services
(4) dataDir: memory used to store database snapshot folder for both myid file cluster also exists in this folder (Note: A configuration file can contain only one dataDir words, even if it is commented out)
(5) dataLogDir: to set competitive transaction log directory, transaction log and general log to avoid the separation of the individual as well as snapshots
(6) syncLimit: within how many tickTime, allowing the follower synchronization, if follower backward too much, it will be discarded.
(. 7) server.A = B: C: D:
A is a number, indicates that the server is the first few numbers, B is the ip address server
C first port information used to exchange cluster members, showing that this leader server and cluster server port to exchange information
D is designed to hang in the leader election leader used
Two, Java operating Zookeeper
2.1, create a node (znode) method:
create:
provides two methods to create nodes, the nodes to create a synchronous and asynchronous way.
Synchronization:
parameter 1, node path "Name): InodeName (recursively create a node is not allowed, that is the parent node does not exist in
the case of not allowed to create a child node)
2, node content parameters: a byte array type requires (that is, does not support the serialization, if necessary to achieve sequence
column of, related sequences using java framework, such as Hessian, Kryo frame)
parameter 3, authority node: Ids.OPEN_ACL_UNSAFE open permissions to use. (This parameter is generally the right to development
is not too high demands at the scene, no need to concern)
parameter 4, node type: the type of node creation: CreateMode, offers four type as the first point
PERSISTENT persistent node
PERSISTENT_SEQUENTIAL sequence of the automatic numbering persistent nodes, such nodes are automatically added in accordance with the currently existing nodes 1
EPHEMERAL temporary node, the client session timeout such nodes will be automatically deleted
EPHEMERAL_SEQUENTIAL temporary automatic node number
2.2, Maven dependency information
<dependency>
<groupId>org.apache.zookeeper</groupId>
<artifactId>zookeeper</artifactId>
<version>3.4.6</version>
</dependency>
2.3, Zookeeper client connection
CreateMode types are divided into four kinds:
- 1.PERSISTENT-- Durable, permanently stored on disk
- 2.PERSISTENT_SEQUENTIAL-- lasting sequential type, the node ID is automatically repeated self-energizing guarantee uniqueness
- 3.EPHEMERAL-- temporary type
- 4.EPHEMERAL_SEQUENTIAL-- temporary order type
ublic class Test001 {
//ZK连接地址
private static final String ADDRES = "127.0.0.1:2181";
//session 会话,超时会话时间
private static final int SESSION_OUTTIME = 2000;
//信号量,阻塞程序执行,用户等待zookeeper连接成功,发送成功信号，
private static final CountDownLatch countDownLatch = new CountDownLatch(1);
public static void main(String[] args) throws IOException, InterruptedException, KeeperException {
//创建一个链接
ZooKeeper zk = new ZooKeeper(ADDRES, SESSION_OUTTIME, new Watcher() {
//通过Watcher监听节点是否发生变化
public void process(WatchedEvent event) {
// 获取事件状态
KeeperState keeperState = event.getState();
// 获取事件类型
EventType eventType = event.getType();
if (KeeperState.SyncConnected == keeperState) {
if (EventType.None == eventType) {
countDownLatch.countDown();
System.out.println("zk 启动连接...");
}
}
}
});
// 进行阻塞
//CreateMode有四种类型
countDownLatch.await();
String result = zk.create("/itmayeidu_Lasting", "Lasting".getBytes(), Ids.OPEN_ACL_UNSAFE,
CreateMode.PERSISTENT);
System.out.println(result);
zk.close();
}
}
2.4, create node information Zookeeper
1. 创建持久节点，并且允许任何服务器可以操作
String result = zk.create("/itmayiedu_Lasting", "Lasting".getBytes(), Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
System.out.println("result:" + result);
2. 创建临时节点
String result = zk.create("/itmayiedu_temp", "temp".getBytes(), Ids.OPEN_ACL_UNSAFE, CreateMode.EPHEMERAL);
System.out.println("result:" + result);
2.5、Watcher
(1) What is the Watcher
Watcher interface class event handler for representing a standard, which defines an event notification associated logic, comprising KeeperState (notification status) and the EventType (event type) two enumeration class also defines a method of event callback: process ( WatchedEvent event)
(2) What is the Watcher Interface
Event types represent the same meaning in different states in different notifications
Callback method process ():
The process method is a method Watcher callback interface, when sending a Watcher ZooKeeper event notifications to the client, the client will perform the corresponding process callback method, enabling handling of the incident. The process method defined as follows:
abstract public void process(WatchedEvent event);
WatchedEvent contains three basic properties of each event: notification state (keeperState), event type (the EventType) and a node of the path (path)
ZooKeeper using WatchedEvent objects to encapsulate and passed to the event server Watcher, thereby facilitating callback method of the server process to handle the event.
(3) Watcher Code
public class ZkClientWatcher implements Watcher {
// 集群连接地址
private static final String CONNECT_ADDRES = "192.168.110.159:2181,192.168.110.160:2181,192.168.110.162:2181";
// 会话超时时间
private static final int SESSIONTIME = 2000;
// 信号量,让zk在连接之前等待,连接成功后才能往下走.
private static final CountDownLatch countDownLatch = new CountDownLatch(1);
private static String LOG_MAIN = "【main】 ";
private ZooKeeper zk;
public void createConnection(String connectAddres, int sessionTimeOut) {
try {
zk = new ZooKeeper(connectAddres, sessionTimeOut, this);
System.out.println(LOG_MAIN + "zk 开始启动连接服务器....");
countDownLatch.await();
} catch (Exception e) {
e.printStackTrace();
}
}
public boolean createPath(String path, String data) {
try {
this.exists(path, true);
this.zk.create(path, data.getBytes(), Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
System.out.println(LOG_MAIN + "节点创建成功, Path:" + path + ",data:" + data);
} catch (Exception e) {
e.printStackTrace();
return false;
}
return true;
}
/**
* 判断指定节点是否存在
*
* @param path
* 节点路径
*/
public Stat exists(String path, boolean needWatch) {
try {
return this.zk.exists(path, needWatch);
} catch (Exception e) {
e.printStackTrace();
return null;
}
}
public boolean updateNode(String path,String data) throws KeeperException, InterruptedException {
exists(path, true);
this.zk.setData(path, data.getBytes(), -1);
return false;
}
public void process(WatchedEvent watchedEvent) {
// 获取事件状态
KeeperState keeperState = watchedEvent.getState();
// 获取事件类型
EventType eventType = watchedEvent.getType();
// zk 路径
String path = watchedEvent.getPath();
System.out.println("进入到 process() keeperState:" + keeperState + ", eventType:" + eventType + ", path:" + path);
// 判断是否建立连接
if (KeeperState.SyncConnected == keeperState) {
if (EventType.None == eventType) {
// 如果建立建立成功,让后程序往下走
System.out.println(LOG_MAIN + "zk 建立连接成功!");
countDownLatch.countDown();
} else if (EventType.NodeCreated == eventType) {
System.out.println(LOG_MAIN + "事件通知,新增node节点" + path);
} else if (EventType.NodeDataChanged == eventType) {
System.out.println(LOG_MAIN + "事件通知,当前node节点" + path + "被修改....");
}
else if (EventType.NodeDeleted == eventType) {
System.out.println(LOG_MAIN + "事件通知,当前node节点" + path + "被删除....");
}
}
System.out.println("--------------------------------------------------------");
}
public static void main(String[] args) throws KeeperException, InterruptedException {
ZkClientWatcher zkClientWatcher = new ZkClientWatcher();
zkClientWatcher.createConnection(CONNECT_ADDRES, SESSIONTIME);
// boolean createResult = zkClientWatcher.createPath("/p15", "pa-644064");
zkClientWatcher.updateNode("/pa2","7894561");
}
}
Third, the use Zookeeper load balancing principle (Dubbo service call load balancing)
Ideas: the server will start services to registered ZK registration center, the use of temporary node . Client to obtain the latest information from the service node ZK node, the local use of load balancing algorithm, randomly assigned server.
Create a Project:
3.1, Maven relies
<dependencies>
<dependency>
<groupId>com.101tec</groupId>
<artifactId>zkclient</artifactId>
<version>0.8</version>
</dependency>
</dependencies>
3.2 Create Server server, the service is registered on node ZK
ZkServerScoekt Service
//##ServerScoekt服务端
public class ZkServerScoekt implements Runnable {
private int port = 18080;
public static void main(String[] args) throws IOException {
int port = 18080;
ZkServerScoekt server = new ZkServerScoekt(port);
Thread thread = new Thread(server);
thread.start();
}
public ZkServerScoekt(int port) {
this.port = port;
}
public void run() {
ServerSocket serverSocket = null;
try {
serverSocket = new ServerSocket(port);
System.out.println("Server start port:" + port);
Socket socket = null;
while (true) {
socket = serverSocket.accept();
new Thread(new ServerHandler(socket)).start();
}
} catch (Exception e) {
e.printStackTrace();
} finally {
try {
if (serverSocket != null) {
serverSocket.close();
}
} catch (Exception e2) {
}
}
}
}
3.3、ServerHandler
public class ServerHandler implements Runnable {
private Socket socket;
public ServerHandler(Socket socket) {
this.socket = socket;
}
public void run() {
BufferedReader in = null;
PrintWriter out = null;
try {
in = new BufferedReader(new InputStreamReader(this.socket.getInputStream()));
out = new PrintWriter(this.socket.getOutputStream(), true);
String body = null;
while (true) {
body = in.readLine();
if (body == null)
break;
System.out.println("Receive : " + body);
out.println("Hello, " + body);
}
} catch (Exception e) {
if (in != null) {
try {
in.close();
} catch (IOException e1) {
e1.printStackTrace();
}
}
if (out != null) {
out.close();
}
if (this.socket != null) {
try {
this.socket.close();
} catch (IOException e1) {
e1.printStackTrace();
}
this.socket = null;
}
}
}
}
3.4, ZkServerClient
public class ZkServerClient {
public static List<String> listServer = new ArrayList<String>();
public static void main(String[] args) {
initServer();
ZkServerClient client= new ZkServerClient();
BufferedReader console = new BufferedReader(new InputStreamReader(System.in));
while (true) {
String name;
try {
name = console.readLine();
if ("exit".equals(name)) {
System.exit(0);
}
client.send(name);
} catch (IOException e) {
e.printStackTrace();
}
}
}
// 注册所有server
public static void initServer() {
listServer.clear();
//放在集合里边是为了集群
listServer.add("127.0.0.1:18080");
}
// 获取当前server信息
public static String getServer() {
return listServer.get(0);
}
public void send(String name) {
String server = ZkServerClient.getServer();//getServer默认取第0个
String[] cfg = server.split(":");
Socket socket = null;
BufferedReader in = null;
PrintWriter out = null;
try {
socket = new Socket(cfg[0], Integer.parseInt(cfg[1]));
in = new BufferedReader(new InputStreamReader(socket.getInputStream()));
out = new PrintWriter(socket.getOutputStream(), true);
out.println(name);
while (true) {
String resp = in.readLine();
if (resp == null)
break;
else if (resp.length() > 0) {
System.out.println("Receive : " + resp);
break;
}
}
} catch (Exception e) {
e.printStackTrace();
} finally {
if (out != null) {
out.close();
}
if (in != null) {
try {
in.close();
} catch (IOException e) {
e.printStackTrace();
}
}
if (socket != null) {
try {
socket.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
}
3.5, the transformation ZkServerScoekt
// then start a service 18081
public class ZkServerScoekt implements Runnable {
private static int port = 18081;
public static void main(String[] args) throws IOException {
ZkServerScoekt server = new ZkServerScoekt(port);
Thread thread = new Thread(server);
thread.start();
}
public ZkServerScoekt(int port) {
this.port = port;
}
public void regServer() {
// 向ZooKeeper注册当前服务器
ZkClient client = new ZkClient("127.0.0.1:2181", 60000, 1000);
String path = "/test/server" + port;
if (client.exists(path))
client.delete(path);
client.createEphemeral(path, "127.0.0.1:" + port);
}
public void run() {
ServerSocket serverSocket = null;
try {
serverSocket = new ServerSocket(port);
regServer();
System.out.println("Server start port:" + port);
Socket socket = null;
while (true) {
socket = serverSocket.accept();
new Thread(new ServerHandler(socket)).start();
}
} catch (Exception e) {
e.printStackTrace();
} finally {
try {
if (serverSocket != null) {
serverSocket.close();
}
} catch (Exception e2) {
}
}
}
}
3.6, the transformation ZkServerClient
public class ZkServerClient {
public static List<String> listServer = new ArrayList<String>();
public static String parent = "/test";
public static void main(String[] args) {
initServer();
ZkServerClient client = new ZkServerClient();
BufferedReader console = new BufferedReader(new InputStreamReader(System.in));
while (true) {
String name;
try {
name = console.readLine();
if ("exit".equals(name)) {
System.exit(0);
}
client.send(name);
} catch (IOException e) {
e.printStackTrace();
}
}
}
// 注册所有server
public static void initServer() {
// listServer.add("127.0.0.1:18080");
final ZkClient zkClient = new ZkClient("127.0.0.1:2181", 6000, 1000);
List<String> children = zkClient.getChildren(parent);
getChilds(zkClient, children);
// 监听事件
zkClient.subscribeChildChanges(parent, new IZkChildListener() {
public void handleChildChange(String parentPath, List<String> currentChilds) throws Exception {
getChilds(zkClient, currentChilds);
}
});
}
private static void getChilds(ZkClient zkClient, List<String> currentChilds) {
listServer.clear();
for (String p : currentChilds) {
String pathValue = (String) zkClient.readData(parent + "/" + p);
listServer.add(pathValue);
}
serverCount = listServer.size();
System.out.println("从zk读取到信息:" + listServer.toString());
}
// 请求次数
private static int reqestCount = 1;
// 服务数量
private static int serverCount = 0;
// 获取当前server信息
public static String getServer() {
// 实现负载均衡
String serverName = listServer.get(reqestCount % serverCount);
++reqestCount;
return serverName;
}
public void send(String name) {
String server = ZkServerClient.getServer();
String[] cfg = server.split(":");
Socket socket = null;
BufferedReader in = null;
PrintWriter out = null;
try {
socket = new Socket(cfg[0], Integer.parseInt(cfg[1]));
in = new BufferedReader(new InputStreamReader(socket.getInputStream()));
out = new PrintWriter(socket.getOutputStream(), true);
out.println(name);
while (true) {
String resp = in.readLine();
if (resp == null)
break;
else if (resp.length() > 0) {
System.out.println("Receive : " + resp);
break;
}
}
} catch (Exception e) {
e.printStackTrace();
} finally {
if (out != null) {
out.close();
}
if (in != null) {
try {
in.close();
} catch (IOException e) {
e.printStackTrace();
}
}
if (socket != null) {
try {
socket.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
}
