No automatic conversion to python objects anymore with IPython magic

Issue #271 resolved
Joris Van den Bossche created an issue

Updated the title, as it isn't an issue specific to windows.

I know Windows is not officially supported, but I thought to report it anyway (in any case, rpy2 is easily installable on windows using the gholke wheels).

In the docs of the rmagic, you can see the following snippet:

In [18]: _ = %R x = c(3,4,6.7); y = c(4,6,7); z = c('a',3,4)

In [19]: %Rpull x  y z

In [20]: x
Out[20]: array([ 3. ,  4. ,  6.7])

However, when I do this on Windows, I get for the last line:

In [8]: x
<FloatVector - Python:0x000000000FC3FF88 / R:0x00000000130A0258>
[3.000000, 4.000000, 6.700000]

So the rpy2 robject is apparently not converted automatically to a python object (in this case numpy array as I have numpy installed).

This is the same for the other rmagics (%R -o, %Rget).

If I do the conversion explicitely, this works:

In [10]: rpy2.robjects.conversion.ri2py(x)
Out[10]: array([ 3. ,  4. ,  6.7])

Comments (19)

  1. Joris Van den Bossche reporter

    Apparantly this is the same at ubuntu (@freeasinfreedom commented this, I got a mail notification for this, but for some reason the comments don't up here)

  2. J Heo

    I left two comments mentioning that it is the same at ubuntu and that rpy2 2.4.4 works, but they do not show up here. I don't know why.

  3. Joris Van den Bossche reporter

    I updated the title, as it isn't an issue specific to windows. So it seems that the ipython magic is a bit broken (I tested it on linux with rpy2 2.5.6)

  4. Laurent Gautier

    I had a quick look at it and the default converter for ipython is not doing much.

    The good news is that customization can now be carried out very easily:

    from rpy2.ipython import rmagic
    from rpy2.robjects import conversion
    from rpy2.rinterface import Sexp
    # Conversion from rpy2 low-level (by default covers also high-level)
    # to ipython. We simply call the conversion to python.
    def my_ri2ipy(obj):
        return conversion.ri2py(obj)
    # register the conversion for all rpy2 objects representing R objects
    rmagic.ri2ipython.register(Sexp, my_ri2py)

    Now try again:

    %Rpull x  y z
  5. Laurent Gautier

    @davclark @takluyver : I am considering an early release of rpy2-2.6.0 because of the increasing traffic around conflicts when translating R names with importr() (see issue 274).

    The conversion mechanism is already much easier to customize than it was, and users can cook there own, but I was wondering whether you'd like to introduce changes in the default conversion for the /R magic/.

  6. Dav Clark

    What's the timeline you're thinking about? My schedule is clearing up a tiny bit and I would like to see this addressed (and it seems it is not much work at this point).

    I like the idea of making the ipython magic more convenient by default, while making scripting do less by default. In particular, it seems good if something like %R -o somevar does auto-conversion on somevar. However, if the rpy2.ipython magic is active, should we change the return value of rpy2.robjects.r.somevar?

    Note - I want to avoid getting into how here - just what should it do?

  7. Dav Clark

    This gets into technical details that would allow for certain answers to the questions I just posed ;)

    But before we get into that, can we figure out what we want the interface to be? I'm trying not to steer the boat too much, but I'd lean towards only the rmagics doing aggressive conversion, and that semantics for rmagic conversion and regular ri2py being separate. Here, I'm mostly thinking about conversion to scripts, which likely won't have the rmagic activated in them.

    It's fine if you don't know the codebase, and you only need to comment on functions you want (or don't want). You can leave the rest (e.g., python wrappers that avoid %%R) for those that use it!

  8. Joris Van den Bossche reporter

    OK, the interface: for me, I just expect that the %R magic does the conversion for me. Simply as that (and this is also how it is documented in the docstrings).
    The question then is, do you automatically activate pandas2ri/numpy2ri when using the rmagic, or do you use the default conversion.ri2py and do you let the user activate pandas2ri/numpy2ri themselves. Based on the codebase, I think the first was the intention, as there is a try: from rpy2.robjects import pandas2ri as py2ri and this gets activated in the load_ipython_extension

    I don't think the return value of rpy2.robjects.r.somevar should be changed (this seems a bit another discussion for me, at least from a users perspective?)

  9. Laurent Gautier

    I'd be fine with, and favorable to, keeping the conversions associated to the input and output of R cells distinct from the conversions in Python/rpy2.robjects.

    The rationale is that the "aggressiveness" in converting for added convenience can coexist with something milder.

  10. Laurent Gautier

    Should I understand that the default conversion for ipython-rmagic be equivalent to the ones defined in pandas2ri (and that all agree to that) ?

  11. Laurent Gautier

    Update: I am looking that a proposed fix for the issue. I stumbling on the intent behind a unit test at the moment, and suspect that I have reached issue #179 (and could potentially solve both of them an once).

  12. Joris Van den Bossche reporter

    Answering on your previous comment:

    Yes, I think we agree that the default conversion wihtin the ipython magic may be more aggressive than the default conversion of rpy2 (while now it is less aggressive, as it is not converting anything at all).
    Although I would also be fine if you need to have to do pandas2ri.activate() or numpy2ri.activate() to activate the pandas/numpy specific conversions, but I think it would be nice if the rmagic uses these by default if pandas/numpy are installed.

  13. Laurent Gautier

    The pull request #45 is now in: loading the rmagic extension is now activating the automatic conversion (pandas2ri leve).

    The conversion can be customized further, but I have also added utility function to swtich off (and on again) the conversion to pandas DataFrame, as the copying can quickly turn into a performance issue if one is handling large data tables.

    # load R magic
    %load_ext rpy2.ipython
    # Automatic conversion to pandas DataFrame
    mtcars = %R mtcars
    from rpy2.ipython import rmagic
    # turn the conversion to pandas.DataFrame off
    # (now we have rpy2.robjects.vectors.DataFrame)
    mtcars = %R mtcars
    # turn the conversion to pandas.DataFrame back on
    mtcars = %R mtcars

    (see revision c5e83b91030a)

  14. Log in to comment