objtools/docs/registry.rst

Class registry

Background

I initially developed the class registry for stdqs, a Python package aiming to add Django QuerySet-like features to any Python iterable. Django has a notion of Transforms and Lookups, classes which allow to transform data (using for example an SQL function like LOWER()) perform boolean operations on them, respectively. One may create custom transforms and lookups on a given field using the @TheField.register_lookup decorator. See the Django docs to learn more.

My issue with this system is that it requires registering into a Field, which is a notion that does not exist inside stdqs as I do not need to handle persistence or really know much about the type of an attribute myself; dealing with types is left to the users as most of the standard lookups and transforms in stdqs will simply apply standard Python operators and methods.

Simply using a dict linking classes to lookup/transform names would require using global variables, which is a pattern I really wanted to avoid, so using a module-level decorator alone would not be enough. After some thinking, I had build a stdqs.lookups.LookupRegistry class, which included a register method which could be used as a decorator on classes. I wanted to cleanly separate lookups and transforms, so I also made a stdqs.transforms.TransformRegistry, which was way too similar to the first registry to not make them use a common class, stdqs.helpers.ClassRegistry. Both registries included ways to perform various checks on keys and values of the dictionary, for example to ensure that a lookup has a valid name so it can be used inside of stdqs QuerySets.

Since both lookups and transforms in my project had abstract base classes, I thought it would be possible to use a custom metaclass to allow for automatic registration simply by subclassing. I had done something similar in a very complex way using a metaclass in PseudoScience for the unit system a while ago.

This, and other common patterns explained elsewhere, led me to create this Python library.

Usage

The ClassRegistry <objtools.registry.ClassRegistry> provides a generic answer to the problem of automatic registration of classes with names in a way that allows easy extension by third parties: merely importing a module which uses the registry's metaclass will auto-register everything in this module. The registry itself subclasses dict, therefore has all of its usual methods.

The most basic usage of the registry is as follows:

>>> from objtools.registry import ClassRegistry
>>> registry = ClassRegistry()
>>> class RegisteredClass(metaclass=registry.metaclass): ...
>>> dict(registry)
{'RegisteredClass': __main__.RegisteredClass}

The metaclass <objtools.registry.ClassRegistry.metaclass> property provides a metaclass which performs auto-registration on the current registry instance. Two keyword arguments may optionally be sent to this metaclass:

register (bool)

When True, will perform auto-registration. Defaults to True.

key (str or None)

If defined, will override the default name when registering. If left unspecified, will use the class' __name__.

>>> class NotRegisteredClass(
...     metaclass=registry.metaclass,
...     register=False): ...
>>> class RenamedClass(metaclass=registry.metaclass, key='foo'): ...
>>> dict(registry)
{'RegisteredClass': __main__.RegisteredClass, 'foo': __main__.RenamedClass}

Manual registration and unregistration can also be performed using the register <objtools.registry.ClassRegistry.register> and unregister <objtools.registry.ClassRegistry.unregister> methods. These are aliases to the standard __setitem__ and __delitem__ dict methods.

Custom checks

The default implementation allows registering any class with any name, but provides overridable methods to perform custom checks upon registration: check_key <objtools.registry.ClassRegistry.check_key> and check_value <objtools.registry.ClassRegistry.check_value>. Any exception raised in those methods will be turned into KeyError or ValueError, respectively. This allows, for example, to check for specific attributes, or subclassing an abstract class

>>> class MyClass(object): ...
>>> class CustomRegistry(ClassRegistry):
...     def check_key(self, key):
...         assert key.lower(), 'Key must be lowercase'
...
...     def check_value(self, value):
...         assert issubclass(value, MyClass), 'Must subclass MyClass'
...
>>> registry = CustomRegistry()
>>> class UppercasedKeyClass(metaclass=registry.metaclass, key='FOO'): ...
Traceback (most recent call last):
...
KeyError: Key must be lowercase
>>> class NotMyClass(metaclass=registry.metaclass): ...
Traceback (most recent call last):
...
ValueError: Must subclass MyClass

Cleaner code

I suggest using the following pattern to make using auto-registration cleaner in other parts of your code:

from objtools.registry import ClassRegistry

class CustomRegistry(ClassRegistry):
    pass

registry = CustomRegistry()
register = registry.register
unregister = registry.unregister

class BaseClass(metaclass=registry.metaclass, register=False):
    pass

This will then allow other modules to import yourpkg.yourmodule.BaseClass and use it in a perfectly normal way, making users who do not have to use the registry directly not have to deal with anything relating to it. Note that you may need to explain in a documentation of some sort the register and key keyword arguments.

Working with ABCs or another metaclass

Python's abc module uses the abc.ABCMeta metaclass to perform checks on subclasses and prevent creating instances of subclasses which do not implement all abstract methods or properties, and has an abc.ABC class which does not do much more than having ABCMeta as its metaclass.

Using the above suggested pattern, a class which uses the metaclass but also needs to be an abstract base class, combined with a custom check on the registry itself to ensure all registered classes subclass it, will help ensure all registered classes provde a common set of methods and properties.

from abc import ABC
from objtools.registry import ClassRegistry

class CustomRegistry(ClassRegistry):
    # ... custom checks etc. ...
    pass

registry = CustomRegistry()

class BaseClass(ABC, metaclass=registry.metaclass, register=False):
    pass

However, since ABC uses a custom metaclass itself, creating such an abstract class will result in a cryptic exception:

TypeError: metaclass conflict: the metaclass of a derived class must be
a (non-strict) subclass of the metaclasses of all its bases

The root cause of this issue is that ClassRegistry.metaclass <objtools.registry.ClassRegistry.metaclass> does not inherit from abc.ABCMeta. One could override this property to provide a custom metaclass which inherits from it, but having to rewrite the whole metaclass property — which is probably the strangest code of the ClassRegistry <objtools.registry.ClassRegistry> — is not easy or future-proof. Instead, creating a class which subclasses both metaclasses solves this issue easily:

from abc import ABCMeta

# Registry code here

class BaseClassMetaclass(registry.metaclass, ABCMeta):
    # Note that we do not use metaclass=... here
    pass

class BaseClass(metaclass=BaseClassMetaclass, register=False):
    pass

Reference

objtools.registry