Pondering Python Path Programming Problems

[glyf]

Most Python programmers are at least vaguely aware of sys.path, PYTHONPATH, and the effect they have on importing modules.  However, there's a lot of confusion about how to use them properly, and how powerful these concepts can be if you know how to apply them.  Twisted - and in particular the plugin system - make very nuanced use of the python

Comments

Re: setuptools

[glyf]

I don't really understand your question, because setuptools does a lot, much of which is more or less irrelevant to the path manipulation stuff that I'm talking about here. Also, although I mentioned the plugin stuff in passing, and didn't use any plugin APIs, just discussed the way it did path discovery using structured path objects. I think I might understand the underlying question though: I assume you're really asking about the approach to pluggability in twisted.plugin vs. the approach in setuptools.

Twisted's plugin system is a very simple system for loading code based on the type of plugin to be provided. It's very simple, very focused, and built out of a small collection of structured objects. It is more or less agnostic to issues of installation, except for the optimizations that can be applied at deployment time to cache and speed up plugin lookup.

In effect, it's a way to adapt your runtime environment to a particular interface.

setuptools, by contrast, is very much focused on installation and deployment. "development mode" is a specialized form of installation, invoked by running a 'setup.py' command. It has lots of mini-languages for describing various pieces of metadata.

I obviously prefer Twisted's approach, because it seems to me like setuptools introduces at least a portion of a edit/compile/run cycle into development. With setuptools you write external metadata in ini files and setup.py files. With Twisted you just put python files in your existing Python path.

In the interests of fairness, though, Twisted's plugin system is somewhat lower level. For example, you would have to implement resource_string like this:

def resource_string(module_name, resource_name):
module = getModule(module_name)
return module.filePath.parent().child(resource_name).getContent()Similar to setuptools, this does actually work with zip files, and could easily be made to work with other import systems. It's operating at a lower level, because it is a virtual filesystem paired with modules, rather than an abstract notion of related resources. You have to be familiar with the twisted.python.modules 'module' interface, 'getModule', and as-yet-ad-hoc 'FilePath' interface.

Of course, given that I like Twisted better, I'd also mention that this additional level makes it easier to operate on directory collections of resources as well :).

Is this the kind of thing that you meant?