Article Directory
Outline
At the outset, watchdog of this article, not in the single-chip watchdog, nor the linux watchdog, but the python world to monitor file system changes in a third-party modules. In python file monitoring has two main libraries, one pyinotify , it is a Watchdog . pyinotify mechanism relies on inotify linux platform, can only be applied on the linux platform. watchdog the event of different platforms have the package, not only can monitor windows file system, you can also monitor linux file system.
File system event category
File system event base class is defined as follows:
watchdog.events.FileSystemEvent(event_type, src_path, is_directory=False)
# event.event_type - 事件类型，为 moved / deleted / created / modified 其中之一
# event.src_path - 触发该事件的文件或目录路径
# event.is_directory - 该事件是否由一个目录触发
Watchdog.events.FileSystemEvent derived from the base class subclasses as follows:
watchdog.events.FileDeletedEvent()
# 文件被删除时触发该事件
watchdog.events.DirDeletedEvent()
# 目录被删除时触发该事件
watchdog.events.FileCreatedEvent()
# 文件被创建时触发该事件
watchdog.events.DirCreatedEvent()
# 目录被创建时触发该事件
watchdog.events.FileModifiedEvent()
# 文件被修改时触发该事件(修改文件内容、修改文件inode信息如权限和访问时间，都会触发该事件)
watchdog.events.DirModifiedEvent()
# 目录被修改时触发该事件
watchdog.events.FileMovedEvent()
# 文件被移动或重命名时触发该事件，因为涉及文件移动，所以除了event.src_path表示原路径，还有event.dest_path表示目的路径
watchdog.events.DirMovedEvent()
# 目录被移动或重命名时触发该事件，因为涉及文件移动，所以除了event.src_path表示原路径，还有event.dest_path表示目的路径
File system event handler class
watchdog.events.FileSystemEventHandler is the base class event processor for processing an event, the user must inherit the class, and a corresponding method override in subclasses. The method requires the user to rewrite are:
self.on_any_event(event)
# 任何事件发生都会首先执行该方法，该方法默认为空，dispatch()方法会先执行该方法，然后再把 event 分派给其他方法处理
self.on_moved(event)
# 处理 DirMovedEvent 和 FileMovedEvent 事件，默认为空
self.on_created(event)
# 处理 DirCreatedEvent 和 FileCreatedEvent 事件，默认为空
self.on_deleted(event)
# 处理 DirDeletedEvent 和 FileDeletedEvent 事件，默认为空
self.on_modified(event)
# 处理 DirModifiedEvent 和 FileModifiedEvent 事件，默认为空
In the above method, event has several properties are available:
- event.is_directory - whether the trigger event folder
- event.src_path - source path
- event.dest_path - destination path
The simplest application examples
The following example shows how to monitor the D: \ XufiveGit \ PEC \ moved client folder of all files / deleted / created / modified. Please note that renaming is considered moved (mobile).
#-*- coding: utf-8 -*-
from watchdog.observers import Observer
from watchdog.events import *
class FileEventHandler(FileSystemEventHandler):
def on_any_event(self, event):
pass
def on_moved(self, event):
if event.is_directory:
print("directory moved from {0} to {1}".format(event.src_path,event.dest_path))
else:
print("file moved from {0} to {1}".format(event.src_path,event.dest_path))
def on_created(self, event):
if event.is_directory:
print("directory created:{0}".format(event.src_path))
else:
print("file created:{0}".format(event.src_path))
def on_deleted(self, event):
if event.is_directory:
print("directory deleted:{0}".format(event.src_path))
else:
print("file deleted:{0}".format(event.src_path))
def on_modified(self, event):
if event.is_directory:
print("directory modified:{0}".format(event.src_path))
else:
print("file modified:{0}".format(event.src_path))
if __name__ == "__main__":
import time
observer = Observer()
event_handler = FileEventHandler()
observer.schedule(event_handler, r"D:\XufiveGit\PEC\client", True)
observer.start()
try:
while True:
time.sleep(1)
except KeyboardInterrupt:
observer.stop()
Problems
After truly tested, you will find that the above example is almost no practical value, because the file operations triggered by an event a lot more than we think, and it is difficult to make specific treatment in the event function. For example, add a file, will inevitably lead created event, but also lead to the folder where the modified event, if the file is a directory rather deep, also trigger modified event multilayer parent folder.
If you think this is not a problem, then the windows platform and each time the trigger twice modified file modification event, the issue is not considered a headache it? How to behave on the linux platform, I have not tested, but the windows platform, due to the watchdog package is FileSystemWatcher Events windows system files during the processing performed in a number of file system operations, can not be avoided trigger a number of events .
improve proposals
If you do not ask for real-time monitoring file, too lazy to deal with a lot of events, then the file before and after the event folder compare snapshots is worth trying to improve the program. Realize this idea, there are three prerequisites:
- Quick access to folders snapshot. Fortunately, watchdog module provides capabilities to our DirectorySnapshot
- 200 milliseconds is acceptable. Event triggered a lot of file operations will be concentrated in a short period of time, under normal circumstances, operate 200 ms after the file folder to obtain a snapshot, is a good interval
- Quickly compare file folder snapshots. This is not a problem, watchdog module DirectorySnapshotDiff sub-module
Improvement idea is this: Set a timer, a snapshot after 200 ms, and compared with the previous snapshot. Whenever an event occurs, check whether the timer has started. If it does not, directly start the timer; otherwise, indicating that the event is less than 200 milliseconds from the last event, be regarded as the same set of events, while the termination timer, reboot again. Specific code as follows:
#-*- coding: utf-8 -*-
import os, threading
from watchdog.observers import Observer
from watchdog.events import *
from watchdog.utils.dirsnapshot import DirectorySnapshot, DirectorySnapshotDiff
class FileEventHandler(FileSystemEventHandler):
def __init__(self, aim_path):
FileSystemEventHandler.__init__(self)
self.aim_path = aim_path
self.timer = None
self.snapshot = DirectorySnapshot(self.core.proj_path)
def on_any_event(self, event):
if self.timer:
self.timer.cancel()
self.timer = threading.Timer(0.2, self.checkSnapshot)
self.timer.start()
def checkSnapshot(self):
snapshot = DirectorySnapshot(self.aim_path)
diff = DirectorySnapshotDiff(self.snapshot, snapshot)
self.snapshot = snapshot
self.timer = None
print("files_created:", diff.files_created)
print("files_deleted:", diff.files_deleted)
print("files_modified:", diff.files_modified)
print("files_moved:", diff.files_moved)
print("dirs_modified:", diff.dirs_modified)
print("dirs_moved:", diff.dirs_moved)
print("dirs_deleted:", diff.dirs_deleted)
print("dirs_created:", diff.dirs_created)
# 接下来就是你想干的啥就干点啥，或者该干点啥就干点啥
pass
class DirMonitor(object):
"""文件夹监视类"""
def __init__(self, aim_path):
"""构造函数"""
self.aim_path= aim_path
self.observer = Observer()
def start(self):
"""启动"""
event_handler = FileEventHandler(self.aim_path)
self.observer.schedule(event_handler, self.aim_path, True)
self.observer.start()
def stop(self):
"""停止"""
self.observer.stop()
if __name__ == "__main__":
monitor = DirMonitor(r"D:\XufiveGit\PEC\client")
monitor.start()