Line magic not working in Jupyter Notebook (on Windows)

Issue #385 new
Alexander Whatley
created an issue

I am running into a weird issue which seems like a bug. The following code is giving me an error message in the Jupyter Notebook:

%R library(edgeR)
%R y <- matrix( rpois(1000, lambda=5), nrow=200 )
%R calcNormFactors(y, method = "upperquartile")

The error message is:

Error in match.arg(method) : object 'upperquartile' not found
---------------------------------------------------------------------------
RRuntimeError                             Traceback (most recent call last)
C:\Users\Alexander\Anaconda3\lib\site-packages\rpy2\ipython\rmagic.py in eval(self, code)
    241             # Need the newline in case the last line in code is a comment
--> 242             value, visible = ro.r("withVisible({%s\n})" % code)
    243         except (ri.RRuntimeError, ValueError) as exception:

C:\Users\Alexander\Anaconda3\lib\site-packages\rpy2\robjects\__init__.py in __call__(self, string)
    358         p = _rparse(text=StrSexpVector((string,)))
--> 359         res = self.eval(p)
    360         return conversion.ri2py(res)

C:\Users\Alexander\Anaconda3\lib\site-packages\rpy2\robjects\functions.py in __call__(self, *args, **kwargs)
    177                 kwargs[r_k] = v
--> 178         return super(SignatureTranslatedFunction, self).__call__(*args, **kwargs)
    179 

C:\Users\Alexander\Anaconda3\lib\site-packages\rpy2\robjects\functions.py in __call__(self, *args, **kwargs)
    105             new_kwargs[k] = conversion.py2ri(v)
--> 106         res = super(Function, self).__call__(*new_args, **new_kwargs)
    107         res = conversion.ri2ro(res)

RRuntimeError: Error in match.arg(method) : object 'upperquartile' not found


During handling of the above exception, another exception occurred:

RInterpreterError                         Traceback (most recent call last)
C:\Users\Alexander\Anaconda3\lib\site-packages\rpy2\ipython\rmagic.py in R(self, line, cell, local_ns)
    670                 for line in code.split(';'):
--> 671                     text_result, result, visible = self.eval(line)
    672                     text_output += text_result

C:\Users\Alexander\Anaconda3\lib\site-packages\rpy2\ipython\rmagic.py in eval(self, code)
    244             warning_or_other_msg = self.flush() # otherwise next return seems to have copy of error
--> 245             raise RInterpreterError(code, str_to_unicode(str(exception)), warning_or_other_msg)
    246         text_output = self.flush()

RInterpreterError: Failed to parse and evaluate line 'calcNormFactors(y, method = upperquartile)'.
R error message: "Error in match.arg(method) : object 'upperquartile' not found"

During handling of the above exception, another exception occurred:

PermissionError                           Traceback (most recent call last)
<ipython-input-19-b44092050f2b> in <module>()
      1 get_ipython().magic('R library(edgeR)')
      2 get_ipython().magic('R y <- matrix( rpois(1000, lambda=5), nrow=200 )')
----> 3 get_ipython().magic('R calcNormFactors(y, method = "upperquartile")')

C:\Users\Alexander\Anaconda3\lib\site-packages\IPython\core\interactiveshell.py in magic(self, arg_s)
   2156         magic_name, _, magic_arg_s = arg_s.partition(' ')
   2157         magic_name = magic_name.lstrip(prefilter.ESC_MAGIC)
-> 2158         return self.run_line_magic(magic_name, magic_arg_s)
   2159 
   2160     #-------------------------------------------------------------------------

C:\Users\Alexander\Anaconda3\lib\site-packages\IPython\core\interactiveshell.py in run_line_magic(self, magic_name, line)
   2077                 kwargs['local_ns'] = sys._getframe(stack_depth).f_locals
   2078             with self.builtin_trap:
-> 2079                 result = fn(*args,**kwargs)
   2080             return result
   2081 

<decorator-gen-127> in R(self, line, cell, local_ns)

C:\Users\Alexander\Anaconda3\lib\site-packages\IPython\core\magic.py in <lambda>(f, *a, **k)
    186     # but it's overkill for just that one bit of state.
    187     def magic_deco(arg):
--> 188         call = lambda f, *a, **k: f(*a, **k)
    189 
    190         if callable(arg):

C:\Users\Alexander\Anaconda3\lib\site-packages\rpy2\ipython\rmagic.py in R(self, line, cell, local_ns)
    689             if not e.stdout.endswith(e.err):
    690                 print(e.err)
--> 691             if tmpd: rmtree(tmpd)
    692             return
    693         finally:

C:\Users\Alexander\Anaconda3\lib\shutil.py in rmtree(path, ignore_errors, onerror)
    486             os.close(fd)
    487     else:
--> 488         return _rmtree_unsafe(path, onerror)
    489 
    490 # Allow introspection of whether or not the hardening against symlink

C:\Users\Alexander\Anaconda3\lib\shutil.py in _rmtree_unsafe(path, onerror)
    381                 os.unlink(fullname)
    382             except OSError:
--> 383                 onerror(os.unlink, fullname, sys.exc_info())
    384     try:
    385         os.rmdir(path)

C:\Users\Alexander\Anaconda3\lib\shutil.py in _rmtree_unsafe(path, onerror)
    379         else:
    380             try:
--> 381                 os.unlink(fullname)
    382             except OSError:
    383                 onerror(os.unlink, fullname, sys.exc_info())

PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\ALEXAN~1\\AppData\\Local\\Temp\\tmp6qk60d_y\\Rplots001.png'

However, when I place the code into an R code block using the %%R magic, it works fine:

%%R
library(edgeR)
y <- matrix( rpois(1000, lambda=5), nrow=200 )
calcNormFactors(y, method = "upperquartile")

I am using rpy2 version 2.8.2, jupyter 4.2.0 on Windows 10. Thanks for your help.

Comments (11)

  1. Laurent Gautier

    If the issue is with double-quoted strings in one-line calls to R, I cannot reproduce it (with latest ryp2 release and ipython 5.1.0 or jupyter 4.2.3):

    In [1]: %load_ext rpy2.ipython
    
    In [2]: %R formatC(c(3.24, 2.3e-6), format = "f")
    Out[2]: 
    array(['3.2400', '0.0000'], 
          dtype='<U6')
    

    May be windows-specific or only with older versions of ipython/jupyter or rpy2 ?

  2. Alexander Whatley reporter

    Yeah, the code is working on my mac as well. It is probably an issue with the windows version of rpy2 then. I'll also post about this error on the Anaconda github page as well.

  3. Laurent Gautier

    Feels like windows-specific: I think that only one file handle per file is allowed, and deletion of a file is prohibited if there is an open handle pointing to it.

    After a quick look it seems that the R magic code is creating a directory, let R open files for graphical output (open a graphical device), try to delete the directory, then ask R to close the device. Fixing this would make it better for all OSes, I believe. @Dav Clark will probably know that part of the code better than I do. I am curious to hear what he thinks.

  4. Dav Clark

    Sorry - I had really extensive notes, then tried to use a mac keybinding on Windows 7. Oops.

    I can replicate on Windows 7 USING THE COMMAND-LINE with @Laurent Gautier's simpler problem code. The issue is indeed with double-quotes and the relevant "true" error message is:

    RInterpreterError: Failed to parse and evaluate line 'formatC(c(3.24, 2.3e-6), format = f)'.
    R error message: "Error in formatC(c(3.24, 2.3e-06), format = f) : object 'f' not found"
    

    NB - it's been a long-standing improvement goal of mine to move the RInterpreterError messages to the top of the output so we see things more clearly!

    I assume the issue arises while parsing the code in the line-magic call, and parse_arguments is imported here.

    Now what's super-weird is that @Laurent Gautier and I get different behavior on Win vs. whatever he runs (some Unix?). Especially since IPython 5.1.0 (which is what we are both running) is using parse_args in a subclass of argparse.Argumentparser.

    Even though I used ALL-CAPS once, I'll repeat that I replicate on the command line on windows, this has nothing to do with the web or graphics systems.

    There are issues around on the internets about quotes and argparse on windows, but it seems to relate to the windows shell, which shouldn't be an issue here, right? I'm a bit confused as to where we should start!

  5. Laurent Gautier

    @Dav Clark : I am running Linux. I believe that I tried the example with rpy2's very own docker image:

    docker run --rm -it rpy2/rpy2:devel ipython
    

    Speaking of which, I am pointing out Windows users to the Docker image whenever issues...

  6. Dav Clark

    Since there's chatter about this on the Anaconda issue tracker again, if you don't want to use the docker solutiion per the above, you can simply avoid double-quotes in your code. Single quotes work fine and are semantically equivalent in R code (except of course in the treatment of internal quotes - but you can address that easily too).

  7. Log in to comment