Path to cache backend not always works

Issue #68 new
Victor Varvaryuk
created an issue

dogpile.cache / dogpile / cache / util.py:

    def register(self, name, modulepath, objname):
        def load():
            mod = __import__(modulepath)
            for token in modulepath.split(".")[1:]:
                mod = getattr(mod, token)
            return getattr(mod, objname)
        self.impls[name] = load

This way of getting the object doesn't work if the module on modulepath is not fully imported yet and there is no the corresponding module attribute at top level module.

This should work for all cases:

    def register(self, name, modulepath, objname):
        def load():
            mod = __import__(modulepath, fromlist=[objname])
            return getattr(mod, objname)
        self.impls[name] = load

Comments (5)

  1. Victor Varvaryuk reporter

    mylib/__init__.py:

    from . import tools
    

    mylib/tools/__init__.py:

    from dogpile.cache import CacheRegion, register_backend
    from dogpile.cache.api import CacheBackend
    
    
    class UwsgiCacheBackend(CacheBackend):
        pass
    
    
    register_backend('mylib.cache.uwsgi', __name__, UwsgiCacheBackend.__name__)
    
    cache_region = CacheRegion().configure(
        'mylib.cache.uwsgi',
    )
    

    Error:

    <ipython-input-1-27ccd4826f08> in <module>()
    ----> 1 import mylib
    
    /home/vic/projects/mylib/mylib/__init__.py in <module>()
    ----> 1 from . import tools
    
    /home/vic/projects/mylib/mylib/tools/__init__.py in <module>()
         10 
         11 cache_region = CacheRegion().configure(
    ---> 12     'mylib.cache.uwsgi',
         13 )
    
    /home/vic/projects/venv/mylib/local/lib/python2.7/site-packages/dogpile/cache/region.py in configure(self, backend, expiration_time, arguments, _config_argument_dict, _config_prefix, wrap)
        228                     "configured with backend: %s"
        229                     % self.backend)
    --> 230         backend_cls = _backend_loader.load(backend)
        231         if _config_argument_dict:
        232             self.backend = backend_cls.from_config_dict(
    
    /home/vic/projects/venv/mylib/local/lib/python2.7/site-packages/dogpile/cache/util.py in load(self, name)
         33     def load(self, name):
         34         if name in self.impls:
    ---> 35             return self.impls[name]()
         36         else:  # pragma NO COVERAGE
         37             import pkg_resources
    
    /home/vic/projects/venv/mylib/local/lib/python2.7/site-packages/dogpile/cache/util.py in load()
         50             mod = __import__(modulepath)
         51             for token in modulepath.split(".")[1:]:
    ---> 52                 mod = getattr(mod, token)
         53             return getattr(mod, objname)
         54         self.impls[name] = load
    
    AttributeError: 'module' object has no attribute 'tools'
    
  2. Michael Bayer repo owner

    OK....you are using a module path in register_backend() that is purposely not where the actual backend implementation is? is that right? what's the reason for that? how is "mylib.cache.uwsgi.tools" going to exist here? Guess I'm still not following. (edit for those watching: the author is forgetting how his own code works)

  3. Victor Varvaryuk reporter

    register_backend('mylib.cache.uwsgi', __name__, UwsgiCacheBackend.__name__) says to register a backend with name 'mylib.cache.uwsgi' on module path mylib.tools with class name UwsgiCacheBackend. myilb.tools module exists (in sys.modules), it just hasn't finished importing yet, so tools attribute doesn't exist yet in mylib module.

  4. Michael Bayer repo owner

    OK achievement unlocked for @Victor Varvaryuk ! reminded a project's maintainer on how his own code works since he had no idea :) I think the __name__ thing there was throwing me off and I've not used fromlist before. the fix should be good if you want to send me a PR or I will get around to it, thanks!

  5. Log in to comment