Writing a TCE scheduler pass in Python

The TCE schefuler front end has support for passes written in Python. If the name of the file in a scheduler configuration file ends with .py, the scheduler front end runs the pass using a Python interprepter embedded in the scheduler.

Ideally, the passes written in Python should be completely isolated from each other. Unfortunately this does not seem to be possible with current Python and current Boost.Python (i.e. Python 2.5 and Boost 1.33.1). The Python C API has a promising function Py_Finalize(), which, according to its documentation, releases all resources held by the Python interpreter. This is not quite true, though, and repeated sequences of Py_Initialize() ... Py_Finalize() leak memory. Boost.Python also does not work nicely with Py_Initialize().

In the current design, the Python interpreter is initialized exactly once when it is first needed, and Py_Finalize() is not called at all. Thus, the resources held by the Python interpreter are not released while the scheduler is running.

The main effect this has on writing scheduler passes in Python is that all the passes share the same name space, and all the global state left behind by one pass is there when the next one starts. This should not really matter unless perhaps if you do serious metaprogramming in Python.

Interface

The scheduler frontend communicates with a pass written in Python using the following interface.

The name of the Python class implementing the pass is derived from the name of the Python source file. If the pass is in file MyCleverPass.py, the frontend attempts to instantiate an object of class MyCleverPass.

The class implementing a scheduler pass should implement the following methods.

__init__(self)
Constructor, should do nothing.
needsTarget(self)
Return true/false depending on whether the pass needs the target or not.
needsProgram(self)
Return true/false depending on whether the pass needs the program (POM) or not.
needsProgramRepresentation(self)
Return true/false depending on whether the pass needs the program in another representation. This is redundant for now, as the scheduler front end only passes the program in POM.
setTarget(self, target)
This method is called by the scheduler front end if needsTarget() returned true. The parameter target is a TTAMachine::Machine object wrapped so that it can be used from Python. The method should store it somewhere for later use, for example assign it to self.target.
setProgram(self, program)
This method is called by the scheduler front end if needsProgram() returned true. The parameter program is a TTAProgram::Program object wrapped so that it can be used from Python. The method should store it somewhere for later use, for example assign it to self.program.
start(self)
This method is called by the scheduler front end to start the actual pass.

Example

For example, if you want to run the pass NoOpPass, written in Python, the scheduler configuration file would contain the following:

<scheduler-configuration>
	<pass>
		<module>
			<name>NoOpPass</name>
			<file>NoOpPass.py</file>
		</module>
	</pass>
</scheduler-configuration>

The scheduler front end would then load NoOpPass.py, and create an instance of the Python class NoOpPass. An example implementation of the class is given below.

#
# A scheduler pass that does nothing.
#

import TTAMachine
import TTAProgram

#
# This class will be instantiated by the scheduler frontend.
#

class NoOpPass:
    def __init__(self):
        print "NoOpPass instantiated"

    def needsTarget(self):
        print "needsTarget() called"
        return True

    def needsProgram(self):
        print "needsProgram() called"
        return True

    def needsProgramRepresentation(self):
        print "needsProgramRepresentation() called"
        return False

    def setTarget(self, target):
        print "setTarget() called with", target
        self.target = target

    def setProgram(self, program):
        print "setProgram() called with", program
        self.program = program

    def start(self):
        print "Running NoOpPass.py"

Pertti Kellomäki, firstname.lastname@tut.fi