iNotify

iNotify

Let us consider that we want to trigger a script based on the filesystem events like file creation or deletion or etc. How we can approach this? We can start a script that constantly checks the last modified time of a directory / file. When the check results in a positive situation, we can call a function and then we may go to sleep for a while before doing another check or the script can exit. If the check is negative, the script may go for a sleep before making another check and this process may continue until the positive condition is met or for certain time period.

In one of my previous projects, we had a script like this that monitors the arrival of files in N number of directories from various other systems through ftp and updates the database with info like filename, size, mtime (last modified time or timestamp of the file). Some of those files are big with size of around 2GB. Because of the size, the modified time changes continuously until the file is written completely. The script had to take care of this as well. The script was written in Perl with 2000 lines of code and it was running for years with little or no change at all. Because there arisen no requirement, and the script was doing the functionality very well, no one touched it at all. Suddenly, may be after 10 years of the original script creation date time, there came a requirement just to rewrite the script for easier maintenance but without any change in the functionality. It was rewritten then with similar approach but instead of directly updating the database, the script spawned N number of child processes that wrote to a text file and an external table was used to read the file.

Is there any better approach? Well, Linux kernel has iNotify. Mac has fsevents and I am not sure about Windows. In GUI applications, we ask the OS to execute a code block when an event like button-click occurs. Similar to this, we can ask the OS to execute a code block when a filesystem event occurs. In Linux, this is achieved by the iNotify API.

For Python, we have pyinotify bindings and for the Perl we have Linux::iNotify2 bindings.
The code fragments given below are taken from https://github.com/seb-m/pyinotify/blob/master/python2/examples/tutorial_notifier.py and https://metacpan.org/pod/Linux::Inotify2 but with minor changes.

Either of the scripts notify whenever a new file is created (but not when the file is overwritten) or when a file is deleted. As soon as it notifies the delete operation, the script exits.

import pyinotify

wm = pyinotify.WatchManager()  # Watch Manager
mask = pyinotify.IN_DELETE | pyinotify.IN_CREATE  # watched events

class EventHandler(pyinotify.ProcessEvent):
    stopOK = False
    def process_IN_CREATE(self, event):
        print "Creating:", event.pathname

    def process_IN_DELETE(self, event):
        print "Removing:", event.pathname
        EventHandler.stopOK = True    # to make notifier to stop processing

handler = EventHandler()
notifier = pyinotify.Notifier(wm, handler)
wdd = wm.add_watch('/tmp', mask, rec=True)

notifier.loop(callback = lambda x: EventHandler.stopOK)
use Linux::Inotify2;

# create a new object
my $inotify = new Linux::Inotify2
   or die "unable to create new inotify object: $!";

# add watchers
$inotify->watch ("/tmp", IN_CREATE | IN_DELETE, sub {
   my $e = shift;
   my $name = $e->fullname;
   print "$name\n";

   # cancel this watcher: remove no further events
   $e->w->cancel if $e->IN_DELETE;
});

# manual event loop
1 while $inotify->poll;
Advertisements