@memoize() decorator steps on function's __doc__ string

Issue #2 resolved
Former user created an issue

When using the @memoize() decorator, the original function's doc string has been stepped on/replaced by the Memorizer classes' docstring.

In [1]: from remember.memoize import memoize

In [2]: @memoize()
   ...: def test(a, b):
   ...:     """example docstring"""
   ...:     pass

In [3]: help(test)

In [4]: test.__doc__
Out[4]: "Memoize a callable's values.\n\n    ``Memoizer`` is mainly meant for internal use. To memoize a callable,\n    use the :func:`memoize` function.\n    \n    "

python has a way to deal with this in functools; it has a magic method called wraps which can help with the passthrough of the original function name and docstring.

Comments (5)

  1. Ryan T. Dean

    It looks like it might be enough to simply add functools.update_wrapper(self, f) to the last line of Memoizer's init method.

    diff -r af922eac912c remember/memoize.py
    --- a/remember/memoize.py       Wed May 19 00:12:00 2010 -0500
    +++ b/remember/memoize.py       Tue Aug 08 16:49:02 2017 -0700
    @@ -23,6 +23,7 @@
             self._allow_unhashable = allow_unhashable
             self._f = f
    +        functools.update_wrapper(self, f)
         def __call__(self, *args, **kwargs):
             key = args, dicts.FrozenDict.supply_dict(kwargs)
  2. Ryan T. Dean

    Alternatively, if you were willing to bring in an outside dependency on decorator, you could make it both docstring and signature preserving by replacing memoize with something like this:

    import decorator
    def memoize(method=None, cache_size=None, allow_unhashable=True):
        if method is None:
            return functools.partial(memoize, cache_size=cache_size,
        return decorator.FunctionMaker.create(
            method, 'return decfunc(%(signature)s)',
                'decfunc': Memoizer(
                '__wrapped__': method

    This also has the added benefit of allowing you to decorate with just @memoize if you don't need to change the default arguments, and still works if you decorate via @memoize()

  3. Mike Graham repo owner

    Thanks for the feedback.

    I pushed dfc2a665226e28c4a2081e95e473bb5c67ee2aba which adds a functools.update_wrapper call as you first described.

    I ended up not going with your second method because I didn't want to dive in enough to understand precisely what it's up to ;)

    FWIW, I didn't provide the convenience of a decorator(factory) that can be used like @foo or @foo(...) for philosophical reasons about having it work one way. Perhaps I was being overly pure...in any event, I left it for now.

  4. Ryan T. Dean

    If you do get a chance, take a look at decorator - https://decorator.readthedocs.io. From it's introduction:

    The decorator module is over ten years old, but still alive and kicking. It is used by several frameworks (IPython, scipy, authkit, pylons, pycuda, sugar, ...) and has been stable for a long time. It is your best option if you want to preserve the signature of decorated functions in a consistent way across Python releases.

    For context, I stumbled across the issue when generating documentation using sphinx and autodoc. I had a familiarly named function, with no signature, and remember.memoize.Memoizer's docstring.

    My second proposal was simply a slightly modified version of decorator's dealing with third party decorators suggestion... which is to say, wrapping the existing decorator (remember.memoize.Memoizer) in a new decorator that supports signature and docstring presentation.

    I'll admit that when I was looking at their docs, I didn't see a clearcut way to rewrite Memoizer to cleanly use decorator directly. But then, I only looked at it for 20 minutes or so. shrug

    With the functools.update_wrapper call, at least docstrings are being preserved now, even if the signature isn't (well, it is on python3.5+, I think, but not before then) - which solves the larger of the two issues, at least from a sphinx/autodoc standpoint.

    Thanks for the quick turnaround.

  5. Log in to comment