Python Decorators

If you are beginner to this, please read them in the following links.
http://simeonfranklin.com/blog/2012/jul/1/python-decorators-in-12-steps/
Bruce Eckel’s articles:
http://www.artima.com/weblogs/viewpost.jsp?thread=240808
http://www.artima.com/weblogs/viewpost.jsp?thread=240845

This blog just tries to find the difference between creating decorators using functions and classes.

Lets say that we have a class CLASS1. The object to which can be created like obj1 = CLASS1(). What if we try to call the object itself as in obj1()? This will call the magic method __call__(self). Let us see this in the python interpreter.

$ python
Python 2.7.5+ (default, Feb 27 2014, 19:39:55) 
[GCC 4.8.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> class CLASS1(object):
...     def __init__(self):
...             print "Object initialization"
...     def __call__(self):
...             print "__call__() method"
... 
>>> obj1 = CLASS1()
Object initialization
>>> obj1()
__call__() method
>>> 

Now, with this understanding, we can easily compare and contrast the decorators created by functions and classes as shown below. The argument to the decorators can be manipulated at different levels and in the snippet below, they are identified by the print statements.

def fFunProcessor(fun):
    def __funWrapper(*args, **kwargs):
        print fun.__name__, "starts"
        val = fun(*args, **kwargs)
        print fun.__name__, "ends"
        return val
    return __funWrapper

def fDecoArgProcessor(*decoargs, **decokwargs):
    print "<< 1 >>", decoargs
    def __funProcessor(fun):
        print "<< 2 >>", decokwargs
        def __funWrapper(*args, **kwargs):
            print "<< 3 >>", decoargs, decokwargs
            print fun.__name__, "starts"
            val = fun(*args, **kwargs)
            print fun.__name__, "ends"
            return val
        return __funWrapper
    return __funProcessor

class cFunProcessor(object):
    def __init__(self, fun):
        self.fun = fun
        return
    def __call__(self, *args, **kwargs):
        print self.fun.__name__, "starts"
        val = self.fun(*args, **kwargs)
        print self.fun.__name__, "ends"
        return val

class cDecoArgProcessor(object):
    def __init__(self, *decoargs, **decokwargs):
        self.decoargs = decoargs
        self.decokwargs = decokwargs
        print ">> 1 <<", self.decoargs
        return
    def __call__(self, fun):
        print ">> 2 <<", self.decokwargs
        def __funWrapper(*args, **kwargs):
            print ">> 3 <<", self.decoargs, self.decokwargs
            print fun.__name__, "starts"
            val = fun(*args, **kwargs)
            print fun.__name__, "ends"
            return val
        return __funWrapper

@fFunProcessor
def fPrintLower(s):
    print s.lower()

@fDecoArgProcessor(1, 2, 3, foo = 'bar')
def fPrintUpper(s):
    print s.upper()

@cFunProcessor
def cPrintLower(s):
    print s.lower()

@cDecoArgProcessor(1, 2, 3, foo = 'bar')
def cPrintUpper(s):
    print s.upper()

fPrintLower('Hello World!')
fPrintUpper('Hello World!')
cPrintLower('Hello World!')
cPrintUpper('Hello World!')

From the name of the functions/classes itself, we can understand their purpose like what the function is expecting as input and what it can process. The wrapper function is required to process any arguments to the decorated function. On concluding, I do not see much difference and its just a matter of style or preference.

Happy Programming!

Advertisements

iNotify

iNotify

Let us consider that we want to trigger a script based on the filesystem events like file creation or deletion or etc. How we can approach this? We can start a script that constantly checks the last modified time of a directory / file. When the check results in a positive situation, we can call a function and then we may go to sleep for a while before doing another check or the script can exit. If the check is negative, the script may go for a sleep before making another check and this process may continue until the positive condition is met or for certain time period.

In one of my previous projects, we had a script like this that monitors the arrival of files in N number of directories from various other systems through ftp and updates the database with info like filename, size, mtime (last modified time or timestamp of the file). Some of those files are big with size of around 2GB. Because of the size, the modified time changes continuously until the file is written completely. The script had to take care of this as well. The script was written in Perl with 2000 lines of code and it was running for years with little or no change at all. Because there arisen no requirement, and the script was doing the functionality very well, no one touched it at all. Suddenly, may be after 10 years of the original script creation date time, there came a requirement just to rewrite the script for easier maintenance but without any change in the functionality. It was rewritten then with similar approach but instead of directly updating the database, the script spawned N number of child processes that wrote to a text file and an external table was used to read the file.

Is there any better approach? Well, Linux kernel has iNotify. Mac has fsevents and I am not sure about Windows. In GUI applications, we ask the OS to execute a code block when an event like button-click occurs. Similar to this, we can ask the OS to execute a code block when a filesystem event occurs. In Linux, this is achieved by the iNotify API.

For Python, we have pyinotify bindings and for the Perl we have Linux::iNotify2 bindings.
The code fragments given below are taken from https://github.com/seb-m/pyinotify/blob/master/python2/examples/tutorial_notifier.py and https://metacpan.org/pod/Linux::Inotify2 but with minor changes.

Either of the scripts notify whenever a new file is created (but not when the file is overwritten) or when a file is deleted. As soon as it notifies the delete operation, the script exits.

import pyinotify

wm = pyinotify.WatchManager()  # Watch Manager
mask = pyinotify.IN_DELETE | pyinotify.IN_CREATE  # watched events

class EventHandler(pyinotify.ProcessEvent):
    stopOK = False
    def process_IN_CREATE(self, event):
        print "Creating:", event.pathname

    def process_IN_DELETE(self, event):
        print "Removing:", event.pathname
        EventHandler.stopOK = True    # to make notifier to stop processing

handler = EventHandler()
notifier = pyinotify.Notifier(wm, handler)
wdd = wm.add_watch('/tmp', mask, rec=True)

notifier.loop(callback = lambda x: EventHandler.stopOK)
use Linux::Inotify2;

# create a new object
my $inotify = new Linux::Inotify2
   or die "unable to create new inotify object: $!";

# add watchers
$inotify->watch ("/tmp", IN_CREATE | IN_DELETE, sub {
   my $e = shift;
   my $name = $e->fullname;
   print "$name\n";

   # cancel this watcher: remove no further events
   $e->w->cancel if $e->IN_DELETE;
});

# manual event loop
1 while $inotify->poll;

Variable Declarations in Python and Perl

Consider the following program in a file named var.pl

if (0) { print $abc }
if (1) { print $abc }

When we execute this program as shown below, the program executes fine but we do not get any output (actually, it prints an empty string). Perl does not complain that the variable $abc is not defined.

$ perl var.pl

But it does complain when we execute as shown below.

$ perl -Mstrict var.pl
Global symbol "$abc" requires explicit package name at no_strict.pl line 1.
Global symbol "$abc" requires explicit package name at no_strict.pl line 2.

When the pragma strict is enabled, the compiler mandates the variable declaration before its usage.

Let us see how it works in Python.

$ python
Python 2.7.5+ (default, Sep 19 2013, 13:49:51) 
[GCC 4.8.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> if 0: print abc
... 
>>> if 1: print abc
... 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'abc' is not defined
>>> 

Python complains only for the 2nd if statement but not the 1st one.

Let me try to put what is happening in my own words …

Perl: When the pragma is used, we get additional help from the compiler or it just tries to execute whatever is given (Undeclared variables can act as the number zero or an empty string, etc. and it depends on the context. Context in Perl can be easily understood but difficult to master)

Python: I haven’t come across any such pragma (but tools like py_checker or pep8 would help, not tried). If you think that the code will get executed, make sure that the variables used are written (assigned some value) at least once before reading from them.

Learning: If you want a quick dirty command line or a neat perfect program, Perl offers you both (with just a pragma). Python couldn’t be relied or used the same way. Even a typo error would go unnoticed easily and it may crash at anytime (Well, this is my opinion before using py_checker or pep8 tools).

Happy Weird Programming!

P.S: I wanted to write at least one blog this month and at the very last moment, I am posting it.

Python – Readability

Python is NOT a “free-format” language.
It was designed to be a highly readable language.
It uses whitespace to delimit program blocks, following the off-side rule.

From these points, I don’t think that the off-side rule is good enough to make the program highly readable. Because I have come across Python scripts that are hardly readable.

Example 1:

if row==0:
  empFirstName='Green'
  empLastName=  'Red'
  empAge=35
  empDateOfBirth='14/04/2012'
  empSalary =  5000
  process_emp(empFirstName,empLastName,empAge,empDateOfBirth,empSalary)

This could be written in a better way.

if row == 0:
    empFirstName   = 'Green'
    empLastName    = 'Red'
    empAge         = 35
    empDateOfBirth = '14/04/2012'
    empSalary      = 5000
    
    process_emp(empFirstName, empLastName, empAge,
                empDateOfBirth, empSalary)

Python tries to be readable by enforcing off-side rule.
But still, it has to recommend PEP 8 and PEP 20 to make the programs really highly readable.

So, its not just Python; even C++ or Perl can be highly readable by adhering to the style guides and with a little common sense. Programming can be an art when accompanied by Aesthetics

Happy Programming!

Eight Queens and Processing.js

In the earlier post about the 8-queens problem, we arrived at the final solution.
We couldn’t get a chance on deeper insights on the intermediate steps involved in solving.
The GIF image found at the wikipedia entry says it all.
I just tried emulating the GIF animation with html5/canvas and processing.js

The animation is achieved with the help of alert() function.
There are better alternate ways to do it.

This actually solves not just 8 queens problem but the generalized N Queens problem.
It is controlled by the dim variable (please see the code below).
You can choose the larger number at your own risk.

Fortunately, all the measurements like
1) the cell size (70), 2) font-size(48) and 3) painting location of the queen (X)
worked together perfectly (well, almost).
So, if you want the board to be bigger or smaller, please also take care of other dependent parameters.

I have written three files.
1) .html file 2) .pde (processing.js) file 3) .js file (that contains the core logic of solving the problem. This uses the backtracking algorithm)
Let us see the code of these files in the same order

my_chess_board1.html

<!DOCTYPE html>
<html>
  <head>
    <title>Eight Queens</title>
    <meta charset="UTF-8">
    <script type="text/javascript" src="8q.js"></script>
    <script type="text/javascript" src="processing-js-1.4.1/processing-1.4.1.js"></script>

  </head>

  <body>
    <canvas data-processing-sources="my_chess_board1.pde" style="border: 1px solid black;"></canvas>
  </body>
</html>

my_chess_board1.pde


int dim = 6;
int siz = 70;
String queenStr = "X";
Color white = #FFFFFF;
Color black = #000000;
Color grey  = #DDDDDD;

void setup() {
  size(siz * dim, siz * dim);
  textFont(createFont("Arial", 48));
  noLoop();
}

void draw() {
  drawBoard(dim, siz);
  MAIN(dim, 1, placeQueen, deleteQueen);
}

void drawQueen(row, col, clr) {
  fill(clr);
  float tWidth = textWidth(queenStr);
  text(queenStr, col * siz + tWidth / 2, row * siz + siz * 0.75);
  alert(' ');
}

void placeQueen(row, col) { drawQueen(row, col, black); }

void deleteQueen(row, col) {
  if((row % 2 == 0 && col % 2 == 0) || (row % 2 == 1 && col % 2 == 1))
    drawQueen(row, col, white);
  else
    drawQueen(row, col, grey);
}

void drawBoard(boardSize, cellSize) {
  for(var r = 0; r < boardSize; ++r) {
    for(var c = 0; c < boardSize; ++c) {
      if((r % 2 == 0 && c % 2 == 0) || (r % 2 == 1 && c % 2 == 1))
        fill(white);
      else
        fill(grey);
      rect(r * cellSize, c * cellSize, cellSize, cellSize);
    }
  }
}

8q.js


function colPresence(c, qpos) {
	for(var i = 0; i < qpos.length; ++i)
		if(qpos[i] == c)
			return true;
}

function diagPresence(r, c, qpos) {
	return diagChk(r - 1, c - 1, -1, qpos) || diagChk(r - 1, c + 1,  1, qpos);
}

function diagChk(r, c, d, qpos) {
	return r < 0        ? false :
	       qpos[r] == c ? true  :
	       diagChk(r - 1, c + d, d, qpos);
}

function tryPos(boardSize, r, c, qpos, placeQueenFun, deleteQueenFun) {
	var nextr, nextc;
	if (r == boardSize)
		return;

	if (c == boardSize) {
		if (r == 0) {
			console.log("Solution not possible");
			return;
		}
		nextr = r - 1;
		nextc = qpos.pop() + 1;
		deleteQueenFun(nextr, nextc - 1);
	} else if (colPresence(c, qpos) || diagPresence(r, c, qpos)) {
		nextr = r;
		nextc = c + 1;
	} else {
		qpos.push(c);
		placeQueenFun(r, c);
		nextr = r + 1;
		nextc = 0;
	}

	tryPos(boardSize, nextr, nextc, qpos, placeQueenFun, deleteQueenFun);
}

function MAIN(boardSize, noOfColsToTry, placeQueenFun, deleteQueenFun) {
	for(var i = 0; i < noOfColsToTry; ++i) {
		console.log('Trying for '+ boardSize +', '+ i);
		tryPos(boardSize, 0, i, [], placeQueenFun, deleteQueenFun);
	}
}