[FIXED] Organizing methods in python classes with hierarchical names

Issue

Python officially recognizes namespaces as a "honking great idea" that we should "do more of". One nice thing about namespaces is their hierarchical presentation that organizes code into related parts. Is there an elegant way to organize python class methods into related parts, much as hierarchical namespaces are organized — especially for the purposes of tab-completion?

Some of my python classes cannot be split up into smaller classes, but have many methods attached to them (easily over a hundred). I also find (and my code's users tell me) that the easiest way to find useful methods is to use tab-completion. But with so many methods, this becomes unwieldy, as an enormous list of options is presented — and usually organized alphabetically, which means that closely related methods may be located in completely different parts of this massive list.

Typically, there are very distinct groups of closely related methods. For example, I have one class in which almost all of the methods fall into one of four groups:

• io
• statistics
• transformations
• symmetries

And the io group might have read and write subgroups, where there are different options for the file type to read or write, and then some additional methods involved in looking at the metadata for example. To a small extent, I can address this problem using underscores in my method names. For example, I might have methods like

myobject.io_read_from_csv
myobject.io_write_to_csv

This helps with the classification, but is ugly and still leads to unwieldy tab-completion lists. I would prefer it if the first tab-completion list just had the four options listed above, then when one of those options is selected, additional options would be presented with the next tab.

For a slightly more concrete example, here's a partial list of the hierarchy that I have in mind for my class:

myobject.io
    myobject.io.read
        myobject.io.read.csv
        myobject.io.read.h5
        myobject.io.read.npy
    myobject.io.write
        myobject.io.write.csv
        myobject.io.write.h5
        myobject.io.write.npy
    myobject.io.parameters
        myobject.io.parameters.from_csv_header
        myobject.io.parameters.from_h5_attributes
        ...
    ...
myobject.statistics
    myobject.statistics.max
    myobject.statistics.max_time
    myobject.statistics.norm
    ...
myobject.transformations
    myobject.transformations.rotation
    myobject.transformations.boost
    myobject.transformations.spatial_translation
    myobject.transformations.time_translation
    myobject.transformations.supertranslation
    ...
myobject.symmetries
    myobject.symmetries.parity
        myobject.symmetries.parity.conjugate
        myobject.symmetries.parity.symmetric_part
        myobject.symmetries.parity.antisymmetric_part
        myobject.symmetries.parity.violation
        myobject.symmetries.parity.violation_normalized
    myobject.symmetries.xreflection
        myobject.symmetries.xreflection.conjugate
        myobject.symmetries.xreflection.symmetric_part
        ...
    ...
...

One way I can imagine solving this problem is to create classes like IO, Statistics, etc., within my main MyClass class whose sole purpose is to store a reference to myobject and provide the methods that it needs. The main class would then have @property methods that just return the instances of those lower-lever classes, for which tab-completion should then work. Does this make sense? Would it work at all to provide tab-completion in ipython, for example? Would this lead to circular-reference problems? Is there a better way?

Solution

It looks like my naive suggestion of defining classes within the class does indeed work with ipython's tab-completion and without any circularity problems.

Here's the proof-of-concept code:

class A(object):
    class _B(object):
        def __init__(self, a):
            self._owner = a
        def calculate(self, y):
            return y * self._owner.x

    def __init__(self, x):
        self.x = x
        self._b = _B(self)

    @property
    def b(self):
        return self._b

(In fact, it would be even simpler if I used self.b = _B(self), and I could skip the property, but I like this because it impedes overwriting b from outside the class. Plus this shows that this more complicated case still works.)

So if I create a = A(1.2), for example, I can hit a.<TAB> and get b as the completion, then a.b.<TAB> suggests calculate as the completion. I haven't run into any problems with this structure in my brief tests so far, and the changes to my code aren't very big — just adding ._owner into a lot of the methods code.

Answered By - Mike