OrdDict problem seems to make rcall unusable

Issue #383 resolved
Sebastian Ertel created an issue

I'm trying to use rcall following the example in the docs:

import rpy2.rlike.container as rpc
args = rpc.OrdDict()
args['x'] = rinterface.IntSexpVector([1,2,3])
args[None] = rinterface.IntSexpVector([4,5])
args['y'] = rinterface.IntSexpVector([6, ])
rlist = rinterface.baseenv['list']
rl = rlist.rcall(args.items(), rinterface.globalenv)

My code looks as follows:

    reshape2 = importr("reshape2")
    rcall_args = rpc.OrdDict()
    rcall_args[None] = r_data # a data frame
    rcall_args['measure.vars'] = ri.StrSexpVector(("start", "end"))
    gantt_frame = reshape2.melt(rcall_args.items(), ri.globalenv)

The first problem that I encountered is in the last line which uses the items() function of OrdDict. When running this, I get the following error:

NotImplementedError: Conversion 'py2ri' not defined for objects of type '<type 'listiterator'>'

Now I looked into OrdDict and found that it defines the items() function twice:

...
    def items(self):
        """ Return an ordered list of all key/value pairs """
        res = [self.byindex(i) for i in range(len(self.__l))]
        return tuple(res)
...
    def items(self):
        """ OD.items() -> an iterator over the (key, value) items of D """
        return iter(self.__l)
...

So the first one is of course overridden by the second one.

I think this is a bug, no? The function should be defined only once, right? (Not sure how issue #259 is connected to this as it also suggests to create a tuple.)

Then I changed the class a bit to get access to the first version of the function and I get the following error:

NotImplementedError: Conversion 'py2ri' not defined for objects of type '<type 'tuple'>'

So it seems that neither is supported in the conversion. So how would rcall with an OrdDict ever work?

I'm running python 2.7.10 and rpy2 version 2.8.4

Comments (5)

  1. Sebastian Ertel reporter

    Hi Laurent,

    I can’t see that reply anymore on bitbucket. I guess you already realized that this also does not work as I already stated in the description of the problem. Neither, tuple nor listiterator are supported by the conversion. That’s why I was finally saying: When this conversion never worked for these basic types then how could this rcall ever have worked.

    Thanks for all the effort that you put into this project and your help!

    Cheers, Sebastian.

  2. Laurent Gautier

    Hi @Sebastian Ertel , I deleted my reply almost immediately as I realized that I was wrong (I was answering in a hurry).

    The class rpc.OrdDict seems to need a bit of cleanup. As you noted it the method items() has 2 definitions, the later overriding the earlier definition. This is fixed with revision 8c91509b5bdc.

    Beside that the documentation is not correct as the methods items() returns an iterator (to be consistent with what the method of the same name available for lists and tuples is returning with Python 3 - Python 3 is the preferred Python flavor for rpy2)

    I'll fix the doc, and the changes above will be made available form rpy2-2.8.5 (I can push a bugfix release quickly if this is a critical blocker on your end). In the meantime, the following code appears to be running

    # result of the first definition of `items()`
    l  = tuple(args.byindex(i) for i in range(3))
    fix = iter(l)
    
    # rcall requires a sequence rather than an iterator
    rl = rlist.rcall(tuple(fix), rinterface.globalenv)
    
  3. Sebastian Ertel reporter

    I tried what you suggested but it does not work. My code now looks like this:

    rcall_args = rpc.OrdDict()
    rcall_args[None] = r_data
    rcall_args['measure.vars'] = ri.StrSexpVector(("start", "end"))
    
    # result of the first definition of `items()`
    l  = tuple(rcall_args.byindex(i) for i in range(len(rcall_args)))
    fix = iter(l)
    
    # rcall requires a sequence rather than an iterator
    gantt_frame = reshape2.melt(tuple(fix), ri.globalenv)
    

    But I could see that without even trying because you once again pass in a tuple. So the conversion again says:

    NotImplementedError: Conversion 'py2ri' not defined for objects of type '<type 'tuple'>'

    Just as I stated in my initial description.

    To be honest, I could not understand why it would work now: you are converting a tuple to an iterator and then back to a tuple. I didn't get that. Is there some python magic that makes the second tuple representation different from from the first?

    However, I tried a bit more and noticed that my example does not call rcall. So I transformed my code to this:

    rcall_args = rpc.OrdDict()
    rcall_args[None] = r_data
    rcall_args['measure.vars'] = ri.StrSexpVector(("start", "end"))
    
    # result of the first definition of `items()`
    l  = tuple(rcall_args.byindex(i) for i in range(len(rcall_args)))
    
    # rcall requires a sequence rather than an iterator
    gantt_frame = reshape2.melt.rcall(l, ri.globalenv)
    

    This seems to work now :)

    Thanks for your help, Sebastian.

  4. Laurent Gautier

    I am understanding it as the latest statement ("this seems to work") is overriding the earlier ("it does not work"). The fix already in the repos will make it work with:

    gantt_frame = reshape2.melt.rcall(tuple(rcall_args.items()), ri.globalenv)
    

    The Docker image can be a quick way to check your code:

    1. Pull the latest build
    docker pull rpy2/rpy2:2.8.x
    
    1. Run ipython in a container
    docker run -it --rm rpy2/rpy2:8.x ipython
    

    Beside this, if you do not need rcall_args to be an R vector, you could just use a Python tuple:

    rcall_args = ((None, r_data), 
                  ("measure.vars", ri.StrSexpVector(("start", "end"))))
    gantt_frame = reshape2.melt.rcall(rcall_args, ri.globalenv)
    

    ```

  5. Log in to comment