WatchService not polling correctly

Michael :

I would like to poll a directory every 10 seconds to see if any files have been added or modified. If their are any changes within the 10 seconds I would like to have a set of all file paths that I can then pass to another method.

Problem

When a file is added it is instantly recognized and the addedFiles method is called. Instead I would be expecting it to wait 10 seconds and to call the addedFiles method with multiple files that have been found.

Example
I've created a complete example that watches a directory. A thread then waits 5 seconds and copies 2000 files into the watched directory.
The expected behavior is for the WatchService to check for changes every 10 seconds. Instead it seems to be picking up changes instantly.

Code

import java.io.IOException;
import java.io.PrintWriter;
import java.nio.file.FileSystems;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.nio.file.StandardWatchEventKinds;
import java.nio.file.WatchEvent;
import java.nio.file.WatchKey;
import java.nio.file.WatchService;
import java.util.Collection;
import java.util.HashSet;
import java.util.concurrent.TimeUnit;

public class DirectoryWatcherExample 
{
    private static final int POLLING_TIME = 10;

    public static void main(final String args[]) throws InterruptedException, IOException
    {
        final Path directory = Paths.get("directory/to/be/watched");

        /**
         * Start a thread that will create 2000 files to the selected directory
         * This will occur after waiting 5 seconds.
         */
        new Thread(new Runnable()
        {
            @Override
            public void run() 
            {
                try 
                {
                    Thread.sleep(5000);         
                    System.out.println("Copying 2000 files to directory: " + directory);
                    for(int i = 0; i < 2000; i++)
                    {
                        final PrintWriter writer = new PrintWriter(directory.resolve("test_file_" + i + ".txt").toFile(), "UTF-8");
                        writer.println("The first line");
                        writer.println("The second line");
                        writer.close();
                    }
                    System.out.println("Finished copying files to directory: " + directory);
                } 
                catch (final Exception e) 
                {
                    e.printStackTrace();
                } 
            }
        }).start();

        /**
         * Start the watch service polling every 10 seconds
         */
        new DirectoryWatcherExample().startWatchService(directory);
    }

    public void startWatchService(final Path directory) throws InterruptedException, IOException
    {
        final WatchService watchService = FileSystems.getDefault().newWatchService();
        directory.register(watchService, StandardWatchEventKinds.ENTRY_CREATE, StandardWatchEventKinds.ENTRY_MODIFY);

        while(true)
        {
            System.out.println("Start polling");
            final WatchKey key = watchService.poll(POLLING_TIME, TimeUnit.SECONDS);
            System.out.println("Finished polling and retrieved key");

            if(key != null)
            {
                final Collection<Path> paths = new HashSet<>();
                for (final WatchEvent<?> watchEvent : key.pollEvents())
                {
                    final Path path = ((Path) key.watchable()).resolve((Path) watchEvent.context());
                    paths.add(path);
                    System.out.println("Path added: " + path);
                }

                // Do something with the paths
                addedFiles(paths);

                if (!key.reset())
                {
                    break;
                }   
            }

        }
    }

    // Unimplemented
    public void addedFiles(final Collection<Path> paths)
    {

    }
}

What could be causing this?

Saptarshi Basu :

There are two options:

1. You need to invoke poll on the watchService after a certain interval by sleeping in between. As others pointed out, the timeout in the poll method is for scenarios where no event is available in the buffer. Also, since you are not handling the events immediately, some of the events may overflow the operating system buffer and eventually lost. Therefore, you need to handle the overflow scenario as well.

2. Alternatively, you may want to use Apache Commons IO File Monitoring library. It polls the file system as you want. You can even set the polling interval.

Refer to the following three class/interface here:

• FileAlterationMonitor - It's basically a thread (a Runnable implementation) which sleeps for the polling interval and after every interval invokes FileAlterationObserver
• FileAlterationObserver - It lists the files in the directory, compares the current list with the previous list, identifies the file changes and invokes the appropriate method in the FileAlterationListener implementation
• FileAlterationListener - The interface that you need to implement and write your logic

What you may consider doing for your use case is, keep adding all the file details in a list as and when they are added or modified. Finally, when the onStop() method is invoked, you call your addedFiles method with the complete list, clear the list and start again.