1. raineszm
  2. rubypython
  3. Issues
Issue #5 open

RubyPython seems to call wrong new() method when creating instance of class

Anonymous created an issue

When I create an instance of a Python class for the first time, it works like expected. But when I try to do the same thing again, RubyPython seems to call a different new() method.

{{{

$ irb irb(main):001:0> require 'rubypython' => true irb(main):002:0> RubyPython.start(:python_exe => "python2.6") => true irb(main):003:0> sys = RubyPython.import "sys" => <module 'sys' (built-in)> irb(main):004:0> sys.argv = [""] => [""] irb(main):005:0> pyDrQueue = RubyPython.import("DrQueue") => <module 'DrQueue' from '/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/DrQueue/init.py'> irb(main):006:0> foo = pyDrQueue.Client.new => <DrQueue.client.Client instance at 0x10124cef0> irb(main):007:0> RubyPython.stop => true

irb(main):008:0> RubyPython.start(:python_exe => "python2.6") => true irb(main):009:0> sys = RubyPython.import "sys" => <module 'sys' (built-in)> irb(main):010:0> sys.argv = [""] => [""] irb(main):011:0> pyDrQueue = RubyPython.import("DrQueue") => <module 'DrQueue' from '/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/DrQueue/init.py'> irb(main):012:0> foo = pyDrQueue.Client.new RubyPython::PythonError: TypeError: an integer is required from /opt/local/lib/ruby/gems/1.8/gems/rubypython-0.5.1/lib/rubypython/rubypyproxy.rb:311:in `new' from (irb):12 irb(main):013:0>

}}}

Comments (11)

  1. raineszm repo owner

    I've started looking into this and RubyPython is in fact calling the correct method both times. However, something is going wrong in the Client class's init method within the IPython code. I'm assuming that you are using the DrQueueIPython module. Below is the Python traceback:

    File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/DrQueue/client.py", line 23, in __init__
        self.ip_client = IPClient()
      File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/IPython/parallel/client/client.py", line 244, in __init__
        context = zmq.Context.instance()
      File "context.pyx", line 72, in zmq.core.context.Context.instance (zmq/core/context.c:957)
    
    

    which then culminates in the error that you posted. I'm investigating this further, but for now you may want to try avoiding restarting the interpreter. That is, structure your application as

    #application initialization
    RubyPython.start
    ...
    #main code logic
    ...
    #application cleanup
    RubyPython.stop
    
  2. Thomas Kluyver

    For information from the Python side: zmq.Context.instance has a default argument, which should be an integer. You get this error if you call, e.g. zmq.Context.instance(None) in pure Python.

    To further complicate matters, the Context class is defined in a Cython file - i.e. it isn't loaded directly as a Python module, but automatically translated into C code and compiled.

  3. Min RK

    This is either a problem with RubyPython or Cython.

    If I create the simple Cython module containing only:

    cdef class C:
    
        @classmethod
        def foo(cls, a=5):
            print cls, a
    

    and installed it as 'c', I can reproduce the error with:

    require "rubygems"
    require "rubypython"
    
    RubyPython.start
    cmod = RubyPython.import('c')
    cmod.C.foo
    RubyPython.stop
    RubyPython.start
    cmod = RubyPython.import('c')
    cmod.C.foo
    RubyPython.stop
    

    Which has the output:

    <type 'c.C'> 5
    <type 'c.C'> <type 'c.C'>
    

    So at the second invocation of the interpreter, classmethods on Cython classes are wrong. I don't know if it's RubyPython or Cython that is doing something incorrect, but that's the actual issue here.

  4. raineszm repo owner

    Thanks for the work testing. I figured that was the cause but I wasn't completely sure. This is probably related to the crash that occurs when the python interpreter version is changed within one ruby thread. RubyPython is unable to unload Python extensions when the interpreter stops. Therefore the module loaded in the first interpreter invocation may still be present in the second and that might be causing the issues. Frankly I'm not sure what would cause the default argument to be stored incorrectly as I'm not sure how such things are handled in Cython.

  5. raineszm repo owner

    Thanks for the work testing. I figured that was the cause but I wasn't completely sure. This is probably related to the crash that occurs when the python interpreter version is changed within one ruby thread. RubyPython is unable to unload Python extensions when the interpreter stops. Therefore the module loaded in the first interpreter invocation may still be present in the second and that might be causing the issues. Frankly I'm not sure what would cause the default argument to be stored incorrectly as I'm not sure how such things are handled in Cython.

  6. Anonymous

    Is there a workaround for this problem? Would it help to unload all loaded modules before stopping RubyPython? I need to use RubyPython for a RubyOnRails application and would like to use it in different helper methods.

    Regards, kaazoo

  7. raineszm repo owner

    I've been unable to find a solution to this problem except for:

    • Starting the RubyPython interpreter only once
    • Or forking a new process each time the interpreter starts

    I unfortunately do not know much about Cython so I don't really know the source of this issue. Additionally, I have been unable to find anyway to truly unload compiled Python extensions from the Python interpreter. So at the moment moment I sadly have no workaround.

  8. Austin Ziegler

    Should we consider changing the behaviour of RubyPython for 1.0 to prevent starting the interpreter more than once without forking? I know it's useful to be able to do if all you're doing is pure Python, but unless we can figure out a way to detect the use of a C extension for Python, I don't see a way to make this work.

  9. Austin Ziegler

    I'm starting work on RubyPython again and have a 0.7 release bookmark. I've added an spec/issues/cython to help track this issue. I don't necessarily see a way around it, but will probably get involved in the Cython community to understand this, or build a non-Cython extension (that does the same thing) to see if this is a problem outside of Cython.

    We know it will be a problem for some compiled modules because they hold on to references when they shouldn’t (and there may be something involving atexit as a possible solution for those modules).

  10. Log in to comment