How can Java implement file change monitoring

How can Java implement file change monitoring

If logback is used as the log output component in the application, most of them will configure logback.xmlthis file, and in the production environment, directly modify the log level in the logback.xml file, and it will take effect without restarting the application.

So, how is this function achieved?

I. Problem description and analysis

In response to the above problem, first throw an actual case. In my personal website Z+ , all gadgets are dynamically added and hidden through configuration files. Because there is only one server, the configuration files are simplified. directly placed in a directory on the server

The current problem is that when the content of this file changes, the application can perceive the change, reload the content of the file, and update the internal cache of the application

One of the easiest ways to think of is to poll to determine whether the file has been modified. If it is modified, it will be reloaded and the memory will be refreshed. Therefore, the main issues to be concerned about are as follows:

• How to poll?
• How can I tell if a file has been modified?
• If the configuration is abnormal, will the service be unavailable? (That is, fault tolerance, this is not related to this topic, but it is more important...)

II. Design and Implementation

After the problem is abstracted, the corresponding solution becomes clearer

• How to poll? --" Timer, ScheduledExecutorService can be implemented
• How to judge file modification? --"According to the last modification time of the java.io.File#lastModifiedobtained file, you can compare

Then a very simple implementation is easier:

public class FileUpTest {

    private long lastTime;

    @Test
    public void testFileUpdate() {
        File file = new File("/tmp/alarmConfig");

        // 首先文件的最近一次修改时间戳
        lastTime = file.lastModified();

        // 定时任务，每秒来判断一下文件是否发生变动，即判断lastModified是否改变
        ScheduledExecutorService scheduledExecutorService = Executors.newScheduledThreadPool(1);
        scheduledExecutorService.scheduleAtFixedRate(new Runnable() {
            @Override
            public void run() {
                if (file.lastModified() > lastTime) {
                    System.out.println("file update! time : " + file.lastModified());
                    lastTime = file.lastModified();
                }
            }
        },0, 1, TimeUnit.SECONDS);


        try {
            Thread.sleep(1000 * 60);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
    }
}

The above is a very simple and basic implementation, and it can basically meet our needs, so what is the problem with this implementation?

What happens if an exception occurs during the execution of the scheduled task?

A slight modification to the code above

public class FileUpTest {

    private long lastTime;

    private void ttt() {
        throw new NullPointerException();
    }

    @Test
    public void testFileUpdate() {
        File file = new File("/tmp/alarmConfig");

        lastTime = file.lastModified();

        ScheduledExecutorService scheduledExecutorService = Executors.newScheduledThreadPool(1);
        scheduledExecutorService.scheduleAtFixedRate(new Runnable() {
            @Override
            public void run() {
                if (file.lastModified() > lastTime) {
                    System.out.println("file update! time : " + file.lastModified());
                    lastTime = file.lastModified();
                    ttt();
                }
            }
        }, 0, 1, TimeUnit.SECONDS);


        try {
            Thread.sleep(1000 * 60 * 10);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
    }
}

In the actual test, it is found that the above code is triggered only when it is modified for the first time, but it has no effect if it is modified again, that is, when an exception is thrown, the scheduled task will no longer be executed. The main reason for this problem is because ScheduledExecutorServiceof

View the source code comment description of ScheduledExecutorService directly

If any execution of the task encounters an exception, subsequent executions are suppressed. Otherwise, the task will only terminate via cancellation or termination of the executor. That is, if an exception occurs during the execution of the scheduled task, the subsequent tasks will no longer be executed .

So, when using this posture, make sure that your task does not throw exceptions, otherwise you will not be able to play later.

The corresponding solution is also relatively simple, just catch the whole

III. Advanced Edition

The front is a basic implementation version. Of course, in the Java circle, basically many common needs can be found with corresponding open source tools to use. Of course, this is no exception, and it should be the apache series that everyone compares.

First maven dependency

<dependency>
    <groupId>commons-io</groupId>
    <artifactId>commons-io</artifactId>
    <version>2.6</version>
</dependency>

FileAlterationObserverThe three classes FileAlterationListenerin this tool are mainly FileAlterationMonitorused to realize the relevant demand scenarios. Of course, the use is very simple, so that it is not clear how to explain it. Look directly at one of my open source projects below. Code copied from quick-alarm

public class PropertiesConfListenerHelper {

    public static boolean registerConfChangeListener(File file, Function<File, Map<String, AlarmConfig>> func) {
        try {
            // 轮询间隔 5 秒
            long interval = TimeUnit.SECONDS.toMillis(5);


            // 因为监听是以目录为单位进行的，所以这里直接获取文件的根目录
            File dir = file.getParentFile();

            // 创建一个文件观察器用于过滤
            FileAlterationObserver observer = new FileAlterationObserver(dir,
                    FileFilterUtils.and(FileFilterUtils.fileFileFilter(),
                            FileFilterUtils.nameFileFilter(file.getName())));

            //设置文件变化监听器
            observer.addListener(new MyFileListener(func));
            FileAlterationMonitor monitor = new FileAlterationMonitor(interval, observer);
            monitor.start();

            return true;
        } catch (Exception e) {
            log.error("register properties change listener error! e:{}", e);
            return false;
        }
    }


    static final class MyFileListener extends FileAlterationListenerAdaptor {

        private Function<File, Map<String, AlarmConfig>> func;

        public MyFileListener(Function<File, Map<String, AlarmConfig>> func) {
            this.func = func;
        }

        @Override
        public void onFileChange(File file) {
            Map<String, AlarmConfig> ans = func.apply(file); // 如果加载失败，打印一条日志
            log.warn("PropertiesConfig changed! reload ans: {}", ans);
        }
    }
}

For the above implementation, briefly explain a few points:

• This file monitoring is rooted in the directory, and then filters can be set to monitor the corresponding file changes.
• As in the above registerConfChangeListenermethod, the incoming file is a specific configuration file, so when building the parameters, the directory and the file name are taken out as a filter.
• The second parameter is the jdk8 syntax, which is the specific read configuration file content and maps to the corresponding entity object

One question, what happens if an exception is thrown when the func method is executed?

The actual test results are the same as above. After throwing an exception, you still kneel, so you still have to pay attention not to run exceptions.

So simply take a look at the above implementation logic and directly deduct the core module

public void run() {
    while(true) {
        if(this.running) {
            Iterator var1 = this.observers.iterator();

            while(var1.hasNext()) {
                FileAlterationObserver observer = (FileAlterationObserver)var1.next();
                observer.checkAndNotify();
            }

            if(this.running) {
                try {
                    Thread.sleep(this.interval);
                } catch (InterruptedException var3) {
                    ;
                }
                continue;
            }
        }

        return;
    }
}

From the above, it is basically clear at a glance that the entire implementation logic is different from our first method of timing tasks. Here, threads are used directly, infinite loops, and sleep is used internally to suspend. Therefore, when an exception occurs, it is equivalent to Throw it out directly, this thread will kneel

---

Supplemental JDK version

jdk1.7, provides one WatchService, which can also be used to monitor file changes. I have never touched it before, so I know there is such a thing, and then I searched for the related use, and found that it is quite simple. I saw a blog post stating that it is based on Event-driven, more efficient, a simple example demo is also given below

@Test
public void testFileUpWather() throws IOException {
    // 说明，这里的监听也必须是目录
    Path path = Paths.get("/tmp");
    WatchService watcher = FileSystems.getDefault().newWatchService();
    path.register(watcher, ENTRY_MODIFY);

    new Thread(() -> {
        try {
            while (true) {
                WatchKey key = watcher.take();
                for (WatchEvent<?> event : key.pollEvents()) {
                    if (event.kind() == OVERFLOW) {
                        //事件可能lost or discarded 
                        continue;
                    }
                    Path fileName = (Path) event.context();
                    System.out.println("文件更新: " + fileName);
                }
                if (!key.reset()) { // 重设WatchKey
                    break;
                }
            }
        } catch (Exception e) {
            e.printStackTrace();
        }
    }).start();


    try {
        Thread.sleep(1000 * 60 * 10);
    } catch (InterruptedException e) {
        e.printStackTrace();
    }
}

IV. Summary

Using Java to monitor configuration file changes mainly involves two points

• How to poll: Timer (Timer, ScheduledExecutorService), thread infinite loop + sleep
• File Modification: File#lastModified

Overall, this implementation is relatively simple. Whether it is a custom implementation or relying on commos-io to do it, there is not much technical cost, but one thing to note is:

• Never throw an exception in the callback method of a scheduled task or file change! ! !

In order to avoid the above situation, a possible implementation is to use the asynchronous message notification of EventBus. When the file changes, send a message, and then add an @Subscribeannotation . This not only achieves decoupling, but also avoids service exceptions caused by exceptions (if you are interested in this implementation, you can comment and explain)

V. Other

Reference project

• Project: quick-alarm
• Test class: FileUpTest.java

statement

It is not as good as a letter of faith. The content has already been posted, which is purely the opinion of the family. Because of my average ability, my opinions are incomplete. If you have any questions, please criticize and correct me.

Scan attention, java share