Namespaces are one honking great idea -- let's do more of those!
-- The Zen of Python (PEP 20) by Tim Peters
In the quest for order and heirarchy in our code, namespaces are our chief weapon (along with fear and surprise). Namespaces come in 2 forms in python: Implicitly inside classes and functions, and explicitly as modules. Subtleties aside, modules are just namespaces with an odd usage: They must be files, and only other modules can parent them.
Of the implicit namespaces, each have their own "quirks":
- Classes - these namespaces have two modes - "normal" and instanciated, and both have odd behaviours.
- Functions - their internal space cannot be accessed from the outside; they only provide locality.
All this babble was meant to demonstrate that these are not enough. You can't always keep your code organized if your choice is only between file seperation, twisted behaviour or no-public policy.
What else could I possibly want?
Well, for example, dividing responsibilities inside a class. Classes infamously have several responsibilities, and while they all must reside inside the same class, there's no reason why they should be blended inside each other.
Consider all of the GUI libraries you ever used. Now consider the following hypothetical GUI window:
class YetAnotherGuiWindow(whatever):
...
namespace spatial:
location = property(_get_loc, _set_loc)
size = property(_get_size, _set_size)
def move(self, where_to):
...
namespace visual:
color = property(_get_color, _set_color)
...
...
#Somewhere Else
guiwindow.spatial.move( guiwindow.spatial.pos + (5,5) )
guiwindow.visual.color = Colors.Black
...
Think how wonderful it would be to perform dir(guiwindow.visual) and see only visual possiblities, without all the other 500 attributes. Want to find an event? dir(guiwindow.events), it's a rather short list. It would also be a lot clearer to the reader that when you try to access "background_process" you mean "threads.background_process" and not "cool_visual_filter_effects.background_process".
There are more examples I could think of.
At this point I can foresee a possible response, that I wish to address in advance. People might propose that I rename methods to fake a namespace, and that the dir listing can work by filtering the names for that fake namespace. This would vaguely emulate what I'm hoping for, but I can't help comparing it to writing C++ using C structs and function pointers.
I like your idea. Is there anything I can do with what python currently offers?
I have given this some thought. The short answer is: No. Python's namespaces are crippled, and this cannot be solved. The long answer is: Maybe, depends on what you want. You could use a class to provide a namespace, in which all methods (perhaps through a decorator for each) would recieve the parent class' instance as self, rather than their own. This is a bit tedious, as it requires addition to every function, so I tried to see if I can use python's dynamic and introspective nature to my advantage:
def make_namespace(self, ns_cls):
"This code iterates all functions in ns_cls and binds them to self"
for attr_name in dir(ns_cls):
attr = getattr(ns_cls,attr_name)
if type(attr) == types.MethodType:
setattr(ns_cls, attr_name, attr.__get__(self) )
Usage:
class YetAnotherGuiWindow:
def __init__(self, pos, size, ...):
...
self._size = size
make_namespace(self, self.spatial)
class spatial:
...
def get_size(self):
return self._size
def set_size(self, size):
self._size = size
...
#Somewhere Else
guiwindow.spatial.set_size( (100, 100) )
This seems to work fine for methods, but not for properties.
In fact, I couldn't manage to write any code that would work for properties. Readers are welcome to see this as a challenge to produce such code. Honorable mentions promised to solvers.
24 Comments
Heya
My first comment on your blog. Mazal Tov on the blog!
My thoughts on your post:
1. It happened to me before when I needed some namespace for a few functions. I usually use some singleton class for that (potentially with staticmethod-s)
2. I remember reading some time ago a post in Python-Ideas about changing the syntax, so that you could do something like
class MyClass: .... MyClass Foo: ....Alas, I can't find the original post so I won't elaborate on the semantics, but your post reminded me of this idea. Given this idea, it would be quite simple to implement your idea.
3. Another related issue is enums. From time to time I need a few constants, and I usually either keep them in an enum, or create an ad-hoc class for them. However, the official recommendation from GvR is not to do that.
4. One thing that annoys me with too much namespaces is that it's sometimes too much work to locate a function. I think that given your proposal, a regular dir() call should return a kind of a dictionary, like so
{"spatial": ["location","size"],"visual":["color"] }
What do you think?
Thanks!
1. It's relatively easy to solve with a custom decorator for each function. But properties still require special treatment.
2. Thanks, it's an interesting idea (but I can settle for less).
3. I think a namespace would be a bit better suited for enums than a class (though this is of small relevance)
4. Interesting, I like it. It might make automatic searching a bit awkward, but not overly, as a function can be written for it.
Python 3 will have an __dir__ special method that you might find useful.
"In fact, I couldn’t manage to write any code that would work for properties."
That's because you're not using new-style classes. You have to inherit from object unless you're using 3.0.
Thanks, but I couldn't get it to work using new-style classes.
The problem is that properties are sort of a magic in Python, and I can't find a way to add (or change) them in run-time..
If I understand you correctly, you're after something as described here:
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/409366
I have used this recipie to create mini-embedded statemachines within a class.
No, this is not exactly what I'm looking for.
If I understand the code correctly, doing something like: inst.namespace.some_var might return inst.some_var if it doens't exist inside 'namespace', which kinda ruins the point of having namespaces.
This probably could've been made to work, if I could know if the caller is inside the class or the outside of it. However, I'm pretty sure it's not possible to do in Python.
Nice Blog. Good Look
Carl, isn't it __methods__ and __members__?
I feel I must be missing something, or else I have a simple solution. Here's how I think your example could be written.
class YetAnotherGuiWindow(whatever): def __init__(self, pos, size, ...): self.spatial = spatial() self.spatial._size = size ... class spatial: def get_size(self): return self._size def set_size(self, size): self._size = size size = property(get_size, set_size) def move(self, where_to): ... #Somewhere Else guiwindow.spatial.move( guiwindow.spatial.pos + (5,5) ) ...You get the idea.
Indentation got messed up. How do I get my code pretty like yours?
Put it between [ sourcecode language="Python" ] and [ /sourcecode ]
In your solution it's not possible to access the parent instance, or other "namespaces", which is a likely requirement in a real scenario.
I assumed that if they are in different namespaces, then they are separate enough to make that rarely needed. In the rare cases it is needed, you can use the following workaround. I suppose it's too tedious to use all the time though.
class YetAnotherGuiWindow(whatever): def __init__(self, pos, size, ...): self.spatial = spatial() self.spatial.guiwindow = self self.spatial._size = size ... class spatial: def get_size(self): return self._size def set_size(self, size): self._size = size size = property(get_size, set_size) def move(self, where_to): self.guiwindow.visual._draw_here(whereto) ... #Somewhere Else guiwindow.spatial.move( guiwindow.spatial.pos + (5,5) ) ...What do you say? Too tedious?
Well, I provided a solution in my post. What advantage does your solution have over mine?
You complained that yours does not work for properties. With mine, properties are not special, and work just like everything else.
Oh. I already solved that: http://blog.erezsh.com/namespaces-in-python-revisited/
I forget we're in the older post.
Ok, yeah, but that took serious work. This solution is simpler, and with a few drops of syntactic sugar is just as convenient as that solution. Also, it will work for all "special" attributes of a class, whether they be methods, properties, some Python feature that hasn't been invented yet, or simply some user-defined type that overides __get__. In short, this way, you get Python to do the work for you.
Well, you're right, both our solutions are functionally equivalent. Your solution is definitely easier to implement, and mine is more elegant to use imho.
Either way, it's just a minor point. My main point to take from this post is that when a class has many attributes and methods, and it can't be split into smaller classes, then it's a good idea to categorize them into namespaces. Your code certainly fits in with that idea.
Yours is more elegant to use because you don't have to keep saying .guiwindow, right?
Yes, that's my reasoning.
Hm, maybe this is fixable, although it is starting to approach the complexity of your solution.
Now if all namespaces inherit from namespace, we've mostly fixed the problem. I say mostly because although we can transparently read things from the outer namespace, we can not write to it. However, presumably namespaces are the only things at the outer level, and we don't want to overwrite those.