Implementation of unified configuration management based on ZooKeeper and zkclient (2)

The previous blog "Unified Configuration Management Implementation Based on ZooKeeper and zkclient (1)" shared the unified configuration management based on ZooKeeper's native api. This article will implement this function again by using the api encapsulated by zkclient.

The effect achieved is similar to the previous article, and will not be repeated here.

system structure

The system still consists of four components:

• ZooKeeperServer

        The cluster or stand-alone version of the ZooKeeper server is mainly used to store the configuration file information published by IConfigPublisher

• IConfigPublisher

        The publisher of the configuration file, responsible for publishing the configuration file information to ZooKeeperServer

• IConfigSubscriber

        The subscriber for configuration file changes, the client opens the subscription to the server configuration file information, and is responsible for updating the local information to the latest state when the configuration information changes

• ZkConfigChanger

        The configuration file changer, usually called manually by the user, is used to change the information of the configuration file

The method of starting the ZooKeeper cluster is the same as the previous blog, and I won't repeat it here. You can view the "Unified Configuration Management Implementation Based on ZooKeeper and zkclient (1)" asynchronously .

Among them, IConfigPublisher and IConfigSubscriber are interfaces. Let's take a look at the definitions of the two interfaces:

/**
 * Config files publisher
 * @author hwang
 *
 */
public interface IConfigPublisher {

	/**
	 * publish config files under {@link configDir} to {@link configRootNode} of {@link zkServerHost}
	 * @param zkServerHost
	 * @param configRootNode
	 * @param configDir
	 */
	public void publish(String zkServerHost,String configRootNode,String configDir);
}

The IConfigPublisher interface has only one publish method, and the main job is to publish the configuration files in the configDir directory to the configRootNode node of zkServerHost.

/**
 * Subscribe Config files change
 * @author hwang
 *
 */
public interface IConfigSubscriber {

	/**
	 * <p>Subscribe config files change event under rootNode {@link configRootNode} of {@link zkServerHost}</p>
	 * <p>include the dataChange and childrenChange </p>
	 * @param zkServerHost
	 * @param configRootNode
	 */
	public void subscribe(String zkServerHost,String configRootNode);
}

The IConfigSubscriber interface also has only one method. The main job is to subscribe to the changes of the configRootNode node in ZkServerHost.

Now let's look at the implementation of these two main core interfaces.

Configuration file publisher IConfigPublisher

The implementation class of the IConfigPublisher interface is ZkConfigPublisher. Let's see how ZkConfigPublisher implements the publish method:

public class ZkConfigPublisher implements IConfigPublisher{

	private static final Log logger = LogFactory.getLog(ZkConfigSubscriber.class);
	
	private ZkClient client;
	
	private String configRootNode;
	
	@Override
	public void publish(String zkServerHost,String configRootNode,String configDir){
		try{
			if(client==null){
				client = new ZkClient(ZkConstant.ZK_CLUSTER_HOSTS,ZkConstant.ZK_SESSION_TIMEOUT);
				client.setZkSerializer(new ZkUtils.StringSerializer(ZkConstant.CONF_CHAR_SET));
			}
			this.configRootNode = configRootNode;
			String rootNode = "/" + configRootNode;
			// 创建根节点
			ZkClientNodeUtil.createNode(client, rootNode, configDir);
			// 扫描所有配置文件
			this.scanConfigFiles(configDir,ZkConstant.ACCEPT_SUFFIX);
		}catch(Exception e){
			logger.error("",e);
		}
	}
	
	/**
     * 扫描指定目录下的所有配置文件，并将内容写入到zookeeper节点中
     * @param path	扫描的目录
     * @param acceptSuffix 接受的文件后缀
     * @throws KeeperException
     * @throws InterruptedException
     * @throws IOException 
     */
    private void scanConfigFiles(String path,String acceptSuffix) throws KeeperException, InterruptedException, IOException{
    	File dir = new File(path);
        if(dir.exists() && dir.isDirectory()){
        	File[] subFiles = dir.listFiles();
        	for(File file : subFiles){
        		String absPath = file.getAbsolutePath();
        		String fileName = file.getName();
        		if(file.isDirectory() || (null!=acceptSuffix && !fileName.endsWith(acceptSuffix))){
        			this.scanConfigFiles(absPath,acceptSuffix);
        		}else{
        			String parentDir = file.getParentFile().getAbsolutePath();
        			// 读取文件内容
        			String fileContent = FileUtils.readFileToString(file,ZkConstant.CONF_CHAR_SET);
        			// 创建目录节点
        			ZkClientNodeUtil.createDirNode(client, configRootNode, parentDir);
        			// 创建该目录下的文件节点
        			ZkClientNodeUtil.createFileNode(client, configRootNode, parentDir, fileName, fileContent);
        		}
        	}
        }
    }
    
}

The implementation method is very simple. First, a ZkClient object is created, then a root node is created, and finally all configuration files in the specified directory are scanned, and the configuration files (and directories) that meet the requirements are added to ZooKeeperServer.

 

Configuration file subscriber IConfigSubscriber

The implementation class of the IConfigSubscriber interface is ZkConfigSubscriber. Let's see how ZkConfigSubscriber implements the subscribe method:

public class ZkConfigSubscriber implements IConfigSubscriber{

	private static final Log logger = LogFactory.getLog(ZkConfigSubscriber.class);
	
	
	private ZkClient client;
	
	@Override
	public void subscribe(String zkServerHost, String configRootNode) {
		try{
			if(client==null){
				client = new ZkClient(ZkConstant.ZK_CLUSTER_HOSTS,ZkConstant.ZK_SESSION_TIMEOUT);
				client.setZkSerializer(new ZkUtils.StringSerializer(ZkConstant.CONF_CHAR_SET));
			}
			String rootNode = "/" + configRootNode;
			this.clearConfigDir(client,rootNode);
			this.subscribeRootNode(client, rootNode);
			// 等待配置信息变更
			Thread.currentThread().join();
		}catch(Exception e){
			logger.error("",e);
		}
	}

	
	
	/**
     * 清空本地的配置文件目录
     * @param client
     * @param rootNode
     * @throws IOException
     * @throws InterruptedException 
     * @throws KeeperException 
     */
	private void clearConfigDir(ZkClient client,String rootNode) throws IOException{
    	if(client.exists(rootNode)){
    		String configDir = client.readData(rootNode);
    		FileUtils.deleteDirectory(new File(configDir));
    		logger.info("Delete config dir:"+configDir);
    	}
    }
    
    
    /**
     * 订阅根节点和递归订阅所有子节点
     * @param client
     * @param rootNodePath
     * @throws KeeperException
     * @throws InterruptedException
     * @throws IOException 
     */
	private void subscribeRootNode(ZkClient client,String rootNodePath) throws IOException{
        if(client.exists(rootNodePath)){
        	logger.debug("subscribe node:"+rootNodePath);
        	ZkConfigSubscriber.subscribePath(client, rootNodePath);
        	List<String> subList = client.getChildren(rootNodePath);
        	if(null!=subList && subList.size()>0){
        		// 将节点的所有子节点保存起来
        		NodeChildrenChangedWrapper.addChildren(rootNodePath, subList);
        	}
        	for (String subNode : subList) { 
        		this.subscribeSubNode(client,rootNodePath,subNode);
        	}
        }else{
        	logger.warn("rootNode:"+rootNodePath+" does not exists!");
        }
    }
    
    /**
     * 订阅子节点
     * @param client
     * @param currentNode
     * @param subNode
     * @throws KeeperException
     * @throws InterruptedException
     * @throws IOException 
     */
	private void subscribeSubNode(ZkClient client,String currentNode,String subNode) throws IOException{
    	String nodePath = currentNode+"/"+subNode;
    	if(nodePath.startsWith("/")){
    		// 订阅子节点
    		if(client.exists(nodePath)){
    			// sync content to client
            	String content = client.readData(nodePath);
            	OnetimeConfigSyncer.syncToClient(content);
            	
    			logger.debug("subscribe node:"+nodePath);
    			ZkConfigSubscriber.subscribePath(client, nodePath);
            	
            	List<String> subList = client.getChildren(nodePath); 
            	if(null!=subList && subList.size()>0){
            		// 将节点的所有子节点保存起来
            		NodeChildrenChangedWrapper.addChildren(nodePath, subList);
            	}
            	for (String _subNode : subList) {
            		this.subscribeSubNode(client,nodePath,_subNode);
            	}
            }else{
            	logger.warn("subNode:"+nodePath+" does not exists!");
            }
    	}
    }

}

The specific implementation of the subscribe method is also very simple. First, clear the local configuration file directory, then subscribe to the root node and recursively subscribe to all child nodes. When subscribing, the situation of each node and the child nodes of the node will be saved to the Map. The specific reasons have been explained in the previous blog, and this will not be repeated. The specific method to perform subscription is the subscribePath() method in the ZkConfigSubscriber class. Let's take a look at the content of this method:

    /**
	 * Store the paths that already subscribed
	 */
	private static Set<String> subscribedPathSet = new CopyOnWriteArraySet<String>();
	
	public static void subscribePath(ZkClient client,String path){
		if(!subscribedPathSet.contains(path)){
			subscribedPathSet.add(path);
			// Subscribe ChildChange and DataChange event at path
        	client.subscribeChildChanges(path, new ChildrenChangeListener(client));
        	client.subscribeDataChanges(path, new DataChangeListener(client));
        	logger.info("Subscribe ChildChange and DataChange event at path:"+path);
		}
	}
	
	public static void unsubscribePath(ZkClient client,String path){
		if(subscribedPathSet.contains(path)){
			subscribedPathSet.remove(path);
			// Unsubscribe ChildChange and DataChange event at path
			client.unsubscribeChildChanges(path, new ChildrenChangeListener(client));
			client.unsubscribeDataChanges(path, new DataChangeListener(client));
			logger.info("Unsubscribe ChildChange and DataChange event at path:"+path);
		}
	}
	

Mainly use a CopyOnWriteArraySet to store the paths of all subscribed nodes to prevent repeated subscriptions.

Two Listener classes are used when subscribing, namely ChildrenChangeListener and DataChangeListener.

ChildrenChangeListener

Let's first look at the implementation of ChildrenChangeListener:

    /**
	 * ChildrenChangeListener 
	 * @author hwang
	 *
	 */
	public static class ChildrenChangeListener implements IZkChildListener{

		private static final Log logger = LogFactory.getLog(ChildrenChangeListener.class);
		
		private ZkClient client;
		
		public ChildrenChangeListener(ZkClient client){
			this.client = client;
		}
		
		@Override
		public void handleChildChange(String parentPath, List<String> currentChilds) throws Exception {
			if(currentChilds==null || currentChilds.isEmpty()){
				logger.warn("No currentChilds get form parentPath:"+parentPath);
				return;
			}
			ChildrenChangeResult changeResult = NodeChildrenChangedWrapper.diff(parentPath, currentChilds);
	    	ChildrenChangeType changeType = changeResult.getChangeType();
	    	List<String> changePath = changeResult.getChangePath();
	    	if(changePath==null || changePath.isEmpty()){
	    		logger.warn("No children changePath get form parentPath:"+parentPath);
	    		return;
	    	}
	    	switch(changeType){
	        	case add:{
	        		for(String subPath : changePath){
	        			logger.info("Add children node,path:"+parentPath+"/"+subPath);
	        			String path = parentPath+"/"+subPath;
	        			RealtimeConfigSyncer.syncToClient(client,path);
	        		}
	        	}break;
	        	case delete:{
	        		for(String subPath : changePath){
	        			ZkConfigSubscriber.unsubscribePath(client, subPath);
	        			String filePath = subPath.replaceAll(ZkConstant.SEPRATOR, "/");
	    	        	FileUtils.deleteQuietly(new File(filePath));
	    	        	logger.info("Delete children node,file:"+filePath);
	        		}
	        	}break;
	        	case update:{
	        		logger.info("Update children node,will do nothing");
	        	}break;
	        	default:{
	        		logger.info("Default children node operate,will do nothing");
	        	}break;
	    	}
		}
	}
	

ZkClient will actively trigger the handleChildChange method of the IZkChildListener interface when the NodeChildChanged event occurs. So we only need to implement the handleChildChange method of the IZkChildListener interface, and the same path only needs to be subscribed once, and zkclient will automatically renew the path for us.

DataChangeListener

There is also the IZkDataListener interface. We only need to implement the handleDataChange and handleDataDeleted methods of the IZkDataListener interface. The following is the specific implementation of the interface:

    /**
	 * DataChangeListener 
	 * @author hwang
	 */
	public static class DataChangeListener implements IZkDataListener{

		private static final Log logger = LogFactory.getLog(DataChangeListener.class);
		
		private ZkClient client;
		
		public DataChangeListener(ZkClient client){
			this.client = client;
		}
		@Override
		public void handleDataChange(String dataPath, Object data) throws Exception {
			logger.info("handleDataChange event,dataPath:"+dataPath);
			RealtimeConfigSyncer.syncToClient(client,dataPath);
		}

		@Override
		public void handleDataDeleted(String dataPath) throws Exception {
			logger.info("handleDataDeleted event,dataPath:"+dataPath);
			ZkConfigSubscriber.unsubscribePath(client, dataPath);
			String filePath = dataPath.substring(dataPath.indexOf(ZkConstant.SEPRATOR)).replaceAll(ZkConstant.SEPRATOR, "/");
	    	FileUtils.deleteQuietly(new File(filePath));
		}
	}
	

It should be noted that when a new or modified event occurs, you only need to synchronize the content of the latest configuration file to the local, but when a deletion event occurs, in addition to deleting the local related configuration files, you also need to subscribe to the local configuration file. The event is cancelled, that is, the ZkConfigSubscriber.unsubscribePath() method needs to be executed.

 

Configuration file changer ZkConfigChanger

After implementing the publisher and subscriber, the last one is the configuration file mutator. The main job of the mutator is to modify the content of the configuration file on the ZooKeeperServer side. The specific implementation is as follows:

/**
 * 服务端配置文件更改器
 * @author hwang
 *
 */
public class ZkConfigChanger {

	private static final Log logger = LogFactory.getLog(ZkConfigChanger.class);
	
	private static ZkClient client;
	
	/**
	 * 初始化zkclient
	 */
    public static void init(){
    	if(client==null){
    		try {
    			client = new ZkClient(ZkConstant.ZK_CLUSTER_HOSTS,ZkConstant.ZK_SESSION_TIMEOUT);
    			client.setZkSerializer(new ZkUtils.StringSerializer(ZkConstant.CONF_CHAR_SET));
    		} catch (Exception e) {
    			logger.error("",e);
    		}	
    	}
    }
    
    
    /**
     * 新增目录节点
     * @param configRootNode
     * @param dirAbsolutePath 目录的绝对路径，该目录必须是/config/开头的目录
     * @throws KeeperException
     * @throws InterruptedException
     * @throws UnsupportedEncodingException
     */
    public static boolean addConfigDir(String configRootNode,String dirAbsolutePath) throws KeeperException, InterruptedException, UnsupportedEncodingException{
    	if(null==client){
    		logger.warn("Not connected to ZooKeeper,will return");
    		return false;
    	}
    	if(StringUtils.isEmpty(dirAbsolutePath)){
    		logger.error("dirAbsolutePath can't be empty");
    		return false;
    	}
    	return ZkClientNodeUtil.createDirNode(client, configRootNode, dirAbsolutePath);
    }
  
    /**
     * 删除目录节点
     * @param configRootNode
     * @param dirAbsolutePath
     * @throws InterruptedException
     * @throws KeeperException
     */
    public static boolean deleteConfigDir(String configRootNode,String dirAbsolutePath) throws InterruptedException, KeeperException{
    	if(null==client){
    		logger.warn("Not connected to ZooKeeper,will return");
    		return false;
    	}
    	if(StringUtils.isEmpty(dirAbsolutePath)){
    		logger.error("dirAbsolutePath can't be empty");
    		return false;
    	}
    	return ZkClientNodeUtil.deleteDirNode(client, configRootNode,  dirAbsolutePath);
    }
    

    
    /**
     * 新增文件节点
     * @param configRootNode
     * @param fileAbsolutePath 文件的绝对路径，不包括文件名
     * @param fileName 文件名
     * @param fileContent 文件内容
     * @throws KeeperException
     * @throws InterruptedException
     * @throws UnsupportedEncodingException
     */
    public static boolean addConfigFile(String configRootNode,String fileAbsolutePath,String fileName,String fileContent) throws KeeperException, InterruptedException, UnsupportedEncodingException{
    	if(null==client){
    		logger.warn("Not connected to ZooKeeper,will return");
    		return false;
    	}
    	if(StringUtils.isEmpty(fileAbsolutePath) || StringUtils.isEmpty(fileName) || StringUtils.isEmpty(fileContent)){
    		logger.error("fileAbsolutePath,fileName,fileContent can't be empty");
    		return false;
    	}
    	return ZkClientNodeUtil.createFileNode(client, configRootNode,  fileAbsolutePath, fileName, fileContent);
    }
    
    /**
     * 删除文件节点
     * @param configRootNode
     * @param fileAbsolutePath 文件的绝对路径，不包括文件名
     * @param fileName 文件名
     * @throws InterruptedException
     * @throws KeeperException
     */
    public static boolean deleteConfigFile(String configRootNode,String fileAbsolutePath,String fileName) throws InterruptedException, KeeperException{
    	if(null==client){
    		logger.warn("Not connected to ZooKeeper,will return");
    		return false;
    	}
    	if(StringUtils.isEmpty(fileAbsolutePath) || StringUtils.isEmpty(fileName)){
    		logger.error("fileAbsolutePath,fileName can't be empty");
    		return false;
    	}
    	return ZkClientNodeUtil.deleteFileNode(client, configRootNode,  fileAbsolutePath, fileName);
    }
    

    /**
     * 更新配置文件内容
     * @param configRootNode
     * @param fileAbsolutePath
     * @param fileName
     * @param fileContent
     * @throws InterruptedException
     * @throws KeeperException
     * @throws UnsupportedEncodingException
     */
    public static boolean updateConfigFile(String configRootNode,String fileAbsolutePath,String fileName,String fileContent) throws InterruptedException, KeeperException, UnsupportedEncodingException{
    	if(null==client){
    		logger.warn("Not connected to ZooKeeper,will return");
    		return false;
    	}
    	if(StringUtils.isEmpty(fileAbsolutePath)  || StringUtils.isEmpty(fileName)  || StringUtils.isEmpty(fileContent)){
    		logger.error("fileAbsolutePath,fileName,fileContent can't be empty");
    		return false;
    	}
		return ZkClientNodeUtil.updateFileNode(client, configRootNode,  fileAbsolutePath, fileName, fileContent);
    }
    
    
    
}

So far, the unified configuration management framework refactored by ZkClient is completed.

After actual testing, zkclient can perfectly solve the unsolved problems in the previous blog, thanks to zkclient's extensive and correct use of the retryUntilConnected method.

 

{{o.name}}

{{m.name}}