Presence of __index__ method in class hides some of its right-handed (__r) operators

Issue #1861 resolved
mcesar created an issue

Hi all, this code:

class oops:
  def __init__(self, val):
    self.val = val

  def __rmul__(self, other):
    if isinstance(other, str):
      other = int(other)
    return self.val * other

a = oops(3)
print((2 * a, '2' * a))

when run returns: (6, 6)

But if the _ index _ method is added to the class:

class oops:
  def __init__(self, val):
    self.val = val

  def __rmul__(self, other):
    if isinstance(other, str):
      other = int(other)
    return self.val * other

  def __index__(self):
    return int(self.val)

a = oops(3)
print((2 * a, '2' * a))

the output is: (6, '222')

Seems that adding the _ index _ method to the class forces pypy to catch the * operator as the _ mul _ operator of the string, instead of the _ rmul _ of the oops.

This happens in both 2.X and 3.X flavours of pypy.

CPython returns (6, 6) in both cases (and also in 3.4 and 2.7 versions).

Comments (9)

  1. Philip Jenvey

    Sigh, this is due to CPython's sq_repeat vs nb_multiply slots.

    CPython's strings only have a sq_repeat, not an nb_multiply. In the case of multiplication CPython only checks for a left hand side nb_multiply first before deferring to a right hand side's potential implementation (the __rmul__)

    Only then does it check for the LHS sq_repeat if there was no RHS nb_multiply found.

    This is non-sensical behavior that if anything is wrong on CPython. python-dev's conceded this on similar issues in the past. Here's a previous discussion of a related issue (note that this older issue was initially mischaracterized and was able to be resolved later, but the example shown here seems legitimately bogus):

    https://mail.python.org/pipermail/python-dev/2011-March/109131.html

  2. mcesar reporter

    Thanks, from your links I was able to find http://bugs.python.org/issue11477 , Apparently this is a bug not in PyPy, but in CPython, so eventually both them will behave in the PyPy way sometime in the future.

    In any case, I was surprised that in this correct behavior, instances of a class not inherited from any numeric type (but having an _ index _ method) are treated as numbers. Such behavior precludes overloading _ r _ methods for such class.

  3. Philip Jenvey

    Yup, and that's by design, see PEP 357.

    You can really see how broken the behavior is here:

    Python 2.7.8 (v2.7.8:ee879c0ffa11, Sep  5 2014, 15:30:14) 
    [GCC 4.2.1 Compatible Apple LLVM 5.1 (clang-503.0.40)] on darwin
    Type "help", "copyright", "credits" or "license" for more information.
    >>> from issue1861 import oops
    (6, 6)
    >>> a = oops(3)
    >>> '2' * a
    6
    >>> '2'.__mul__(a)
    '222'
    
  4. Armin Rigo

    Note that we can try harder to reproduce the behavior. It's just buggy in CPython (for almost any definition of bug), but bug-for-bug compatibility is one of the goals of PyPy. It's just that it seems very rare to see people relying on this bug.

    It's nevertheless possible that we find a fix. I'm thinking about something that generalizes the "string + unicode subclass" special case added here: https://mail.python.org/pipermail/pypy-commit/2008-May/027851.html

    We could try to add some internal flag, set on some built-in types, that says "for CPython bug compatiblity consider me as a sequence". Then the operators + and * would alter their behavior if __add__ or __mul__ is not overridden on the left but __radd__ or __rmul__ is overridden on the right. The details are not clear so far...

  5. Log in to comment