PicklingError: Can't pickle <type 'function'>, Apache, Windows 7

Issue #314 open
RogerHaase
created an issue

On my Windows 7 test system there are multiple WSGI applications running under Apache on Windows 7. This includes Moin2, Moin 1.9.6, a werkzeug app serving images, and a old Webware for Python app.

If the Apache server is restarted and any application other than Moin2 is accessed first, then Moin2 is accessed to modify and save an item, then the result is a Pickling error within env\Lib\site-packages\whoosh\filedb\fileindex.py. This failure happens 100% of the time.

 mod_wsgi (pid=3936): Exception occurred processing WSGI script 'C:/home/web/m2-fixedleft/moin2.wsgi'., referer: http://antelope.digitalrockart.org/moin2/%2Bmodify/FrontPage
Traceback (most recent call last):, referer: http://antelope.digitalrockart.org/moin2/%2Bmodify/FrontPage
  File "C:\\home\\web\\m2-fixedleft\\env\\Lib\\site-packages\\flask\\app.py", line 1701, in __call__, referer: http://antelope.digitalrockart.org/moin2/%2Bmodify/FrontPage
    return self.wsgi_app(environ, start_response), referer: http://antelope.digitalrockart.org/moin2/%2Bmodify/FrontPage
  File "C:\\home\\web\\m2-fixedleft\\env\\Lib\\site-packages\\flask\\app.py", line 1689, in wsgi_app, referer: http://antelope.digitalrockart.org/moin2/%2Bmodify/FrontPage
    response = self.make_response(self.handle_exception(e)), referer: http://antelope.digitalrockart.org/moin2/%2Bmodify/FrontPage
  File "C:\\home\\web\\m2-fixedleft\\env\\Lib\\site-packages\\flask\\app.py", line 1687, in wsgi_app, referer: http://antelope.digitalrockart.org/moin2/%2Bmodify/FrontPage
    response = self.full_dispatch_request(), referer: http://antelope.digitalrockart.org/moin2/%2Bmodify/FrontPage
  File "C:\\home\\web\\m2-fixedleft\\env\\Lib\\site-packages\\flask\\app.py", line 1360, in full_dispatch_request, referer: http://antelope.digitalrockart.org/moin2/%2Bmodify/FrontPage
    rv = self.handle_user_exception(e), referer: http://antelope.digitalrockart.org/moin2/%2Bmodify/FrontPage
  File "C:\\home\\web\\m2-fixedleft\\env\\Lib\\site-packages\\flask\\app.py", line 1358, in full_dispatch_request, referer: http://antelope.digitalrockart.org/moin2/%2Bmodify/FrontPage
    rv = self.dispatch_request(), referer: http://antelope.digitalrockart.org/moin2/%2Bmodify/FrontPage
  File "C:\\home\\web\\m2-fixedleft\\env\\Lib\\site-packages\\flask\\app.py", line 1344, in dispatch_request, referer: http://antelope.digitalrockart.org/moin2/%2Bmodify/FrontPage
    return self.view_functions[rule.endpoint](**req.view_args), referer: http://antelope.digitalrockart.org/moin2/%2Bmodify/FrontPage
  File "C:\\home\\web\\m2-fixedleft\\MoinMoin\\apps\\frontend\\views.py", line 493, in modify_item, referer: http://antelope.digitalrockart.org/moin2/%2Bmodify/FrontPage
    return item.do_modify(), referer: http://antelope.digitalrockart.org/moin2/%2Bmodify/FrontPage
  File "C:\\home\\web\\m2-fixedleft\\MoinMoin\\items\\__init__.py", line 727, in do_modify, referer: http://antelope.digitalrockart.org/moin2/%2Bmodify/FrontPage
    self.modify(meta, data, comment, contenttype_guessed, contenttype_qs), referer: http://antelope.digitalrockart.org/moin2/%2Bmodify/FrontPage
  File "C:\\home\\web\\m2-fixedleft\\MoinMoin\\items\\__init__.py", line 395, in modify, referer: http://antelope.digitalrockart.org/moin2/%2Bmodify/FrontPage
    return self._save(meta, data, contenttype_guessed=contenttype_guessed, comment=comment), referer: http://antelope.digitalrockart.org/moin2/%2Bmodify/FrontPage
  File "C:\\home\\web\\m2-fixedleft\\MoinMoin\\items\\__init__.py", line 511, in _save, referer: http://antelope.digitalrockart.org/moin2/%2Bmodify/FrontPage
    return_rev=True,, referer: http://antelope.digitalrockart.org/moin2/%2Bmodify/FrontPage
  File "C:\\home\\web\\m2-fixedleft\\MoinMoin\\storage\\middleware\\protecting.py", line 300, in store_revision, referer: http://antelope.digitalrockart.org/moin2/%2Bmodify/FrontPage
    rev = self.item.store_revision(meta, data, overwrite=overwrite, return_rev=return_rev, **kw), referer: http://antelope.digitalrockart.org/moin2/%2Bmodify/FrontPage
  File "C:\\home\\web\\m2-fixedleft\\MoinMoin\\storage\\middleware\\indexing.py", line 1077, in store_revision, referer: http://antelope.digitalrockart.org/moin2/%2Bmodify/FrontPage
    self.indexer.index_revision(meta, content, backend_name), referer: http://antelope.digitalrockart.org/moin2/%2Bmodify/FrontPage
  File "C:\\home\\web\\m2-fixedleft\\MoinMoin\\storage\\middleware\\indexing.py", line 458, in index_revision, referer: http://antelope.digitalrockart.org/moin2/%2Bmodify/FrontPage
    writer.update_document(**doc)  # update, because store_revision() may give us an existing revid, referer: http://antelope.digitalrockart.org/moin2/%2Bmodify/FrontPage
  File "C:\\home\\web\\m2-fixedleft\\env\\Lib\\site-packages\\whoosh\\writing.py", line 80, in __exit__, referer: http://antelope.digitalrockart.org/moin2/%2Bmodify/FrontPage
    self.commit(), referer: http://antelope.digitalrockart.org/moin2/%2Bmodify/FrontPage
  File "C:\\home\\web\\m2-fixedleft\\env\\Lib\\site-packages\\whoosh\\filedb\\filewriting.py", line 518, in commit, referer: http://antelope.digitalrockart.org/moin2/%2Bmodify/FrontPage
    self._commit_toc(finalsegments), referer: http://antelope.digitalrockart.org/moin2/%2Bmodify/FrontPage
  File "C:\\home\\web\\m2-fixedleft\\env\\Lib\\site-packages\\whoosh\\filedb\\filewriting.py", line 454, in _commit_toc, referer: http://antelope.digitalrockart.org/moin2/%2Bmodify/FrontPage
    segments, self.generation), referer: http://antelope.digitalrockart.org/moin2/%2Bmodify/FrontPage
  File "C:\\home\\web\\m2-fixedleft\\env\\Lib\\site-packages\\whoosh\\codec\\whoosh2.py", line 123, in commit_toc, referer: http://antelope.digitalrockart.org/moin2/%2Bmodify/FrontPage
    toc.write(storage, indexname), referer: http://antelope.digitalrockart.org/moin2/%2Bmodify/FrontPage
  File "C:\\home\\web\\m2-fixedleft\\env\\Lib\\site-packages\\whoosh\\filedb\\fileindex.py", line 161, in write, referer: http://antelope.digitalrockart.org/moin2/%2Bmodify/FrontPage
    stream.write_string(pickle.dumps(schema, -1)), referer: http://antelope.digitalrockart.org/moin2/%2Bmodify/FrontPage
PicklingError: Can't pickle <type 'function'>: attribute lookup __builtin__.function failed, referer: http://antelope.digitalrockart.org/moin2/%2Bmodify/FrontPage

This same failure was happening on CentOS, but the problem went away after adding a WSGIDaemonProcess for each app. Similar to:

WSGIDaemonProcess moin2_wsgi processes=1 threads=3 stack-size=524288
WSGIScriptAlias /moin2/ /home/rockart/webapps/web/moin-2.0/moin2.wsgi/
<Location /moin2>
    WSGIProcessGroup moin2_wsgi
</Location>

Unfortunately, Windows Apache does not support WSGIDaemonProcess.

One easy workaround is to always access Moin2 first after every Apache restart. It is not necessary to modify and save an item.

Another workaround that was discovered is to modify whoosh/filedb/fileindex.py near line 32 as follows:

    #~ from whoosh.compat import pickle, string_type, xrange
    from whoosh.compat import string_type, xrange
    import pickle  # workaround PicklingError

... which replaces the cPickle module imported in whoosh.comat with the pickle module. (importing cPickle as pickle here fails to prevent the error).

Comments (12)

  1. Wim Lewis

    I've been hitting this problem, or one similar to it, with Whoosh 2.5.6 in a Django+WSGI application (MacOSX, python 2.7.5):

     File ".../python2.7/site-packages/django/contrib/admin/options.py", line 1084, in response_action
       response = func(self, request, queryset)
    
     File ".../admin.py", line 69, in re_index
       writer.commit()
    
     File ".../python2.7/site-packages/whoosh/writing.py", line 935, in commit
       self._commit_toc(finalsegments)
    
     File ".../python2.7/site-packages/whoosh/writing.py", line 882, in _commit_toc
       toc.write(self.storage, self.indexname)
    
     File ".../python2.7/site-packages/whoosh/index.py", line 693, in write
       pickle.dumps(field)
    
     File ".../python2.7/copy_reg.py", line 70, in _reduce_ex
       raise TypeError, "can't pickle %s objects" % base.__name__
    
    TypeError: can't pickle function objects
    

    (I've trimmed the paths a bit for readability's sake.)

  2. Wim Lewis

    This is the schema:

        freetext_analyzer = StemmingAnalyzer() | CharsetFilter(accent_map)
    
        schema = Schema(path = ID(stored=True, unique=True),
                        file_mtime = DATETIME(stored=True),
                        name = TEXT(stored=False, field_boost=2.0),
                        description = TEXT(stored=False, field_boost=1.5, analyzer=freetext_analyzer),
                        content = TEXT(analyzer=freetext_analyzer))
    

    I've added TypeError and ValueError to that except block in whoosh/index.py, so next time it happens I'll at least know which field it is. (Like Roger Haase, I usually don't have the problem, just sometimes. I haven't found an easy way to force it to happen.)

  3. Manel Clos

    I have the same problem on Linux, Python 2.7.3 and Whoosh 2.6.0. Simple Schema will fail:

    schema = Schema(name=TEXT())

    can't pickle function objects, here:

    #virtualenv_path/.../site-packages/whoosh/index.py in write, line 693
            try:
                stream.write_string(pickle.dumps(schema, -1))
            except pickle.PicklingError:
                # Try to narrow down the error to a single field
                for fieldname, field in schema.items():
                    try:
    **                    pickle.dumps(field)**
                    except pickle.PicklingError:
                        e = sys.exc_info()[1]
                        raise pickle.PicklingError("%s %s=%r" % (e, fieldname, field))
                # Otherwise, re-raise the original exception
                raise
    
  4. Matt Chaput repo owner

    Hi, sorry to get back to this so late! I'm guessing this had something to do with the stemming function in the stemming analyzer, but it may have been fixed in the meantime. This test passes:

    def test_pickle_schema():
        from whoosh import analysis
        from whoosh.support.charset import accent_map
        from whoosh.compat import dumps
    
        freetext_analyzer = (
            analysis.StemmingAnalyzer() |
            analysis.CharsetFilter(accent_map)
        )
    
        schema = fields.Schema(
            path=fields.ID(stored=True, unique=True),
            file_mtime=fields.DATETIME(stored=True),
            name=fields.TEXT(stored=False, field_boost=2.0),
            description=fields.TEXT(stored=False, field_boost=1.5,
                                    analyzer=freetext_analyzer),
            content=fields.TEXT(analyzer=freetext_analyzer)
        )
    
        # Try to make some sentences that will require stemming
        docs = [
            u"The rain in spain falls mainly in the plain",
            u"Plainly sitting on the plain",
            u"Imagine a greatly improved sentence here"
        ]
    
        with TempIndex(schema) as ix:
            with ix.writer() as w:
                for doc in docs:
                    w.add_document(description=doc, content=doc)
    
            assert dumps(schema, 2)
    
            with ix.reader() as r:
                assert dumps(r.schema, 2)
    

    I'm going to close this but please feel free to open again if this is still an issue.

  5. RogerHaase reporter
    • changed status to open

    On Windows 10 with Whoosh 2.7.2 installed by Moin2, the error still occurs 100% of the time by restarting Apache, accessing any application other than Moin2, and then doing an edit and save of any Moin2 item.

    As per the initial issue description, there is an easy workaround by just accessing Moin2 before any other application. As this only occurs on my test system this issue is of very low priority to me.

    2016-02-23 09:02:23,730 ERROR MoinMoin:1423 Exception on /+modify/moin [POST]
    Traceback (most recent call last):
      File "D:\\Bitbucket\\moin2-venv-python\\Lib\\site-packages\\flask\\app.py", line 1817, in wsgi_app
        response = self.full_dispatch_request()
      File "D:\\Bitbucket\\moin2-venv-python\\Lib\\site-packages\\flask\\app.py", line 1477, in full_dispatch_request
        rv = self.handle_user_exception(e)
      File "D:\\Bitbucket\\moin2-venv-python\\Lib\\site-packages\\flask\\app.py", line 1381, in handle_user_exception
        reraise(exc_type, exc_value, tb)
      File "D:\\Bitbucket\\moin2-venv-python\\Lib\\site-packages\\flask\\app.py", line 1475, in full_dispatch_request
        rv = self.dispatch_request()
      File "D:\\Bitbucket\\moin2-venv-python\\Lib\\site-packages\\flask\\app.py", line 1461, in dispatch_request
        return self.view_functions[rule.endpoint](**req.view_args)
      File "d:\\bitbucket\\moin2\\MoinMoin\\apps\\frontend\\views.py", line 637, in modify_item
        return item.do_modify()
      File "d:\\bitbucket\\moin2\\MoinMoin\\items\\__init__.py", line 882, in do_modify
        self.modify(meta, data, comment, contenttype_guessed, **{CONTENTTYPE: contenttype_qs})
      File "d:\\bitbucket\\moin2\\MoinMoin\\items\\__init__.py", line 496, in modify
        return self._save(meta, data, contenttype_guessed=contenttype_guessed, comment=comment)
      File "d:\\bitbucket\\moin2\\MoinMoin\\items\\__init__.py", line 638, in _save
        return_rev=True,
      File "d:\\bitbucket\\moin2\\MoinMoin\\storage\\middleware\\protecting.py", line 315, in store_revision
        rev = self.item.store_revision(meta, data, overwrite=overwrite, return_rev=return_rev, fqname=fqname, **kw)
      File "d:\\bitbucket\\moin2\\MoinMoin\\storage\\middleware\\indexing.py", line 1135, in store_revision
        self.indexer.index_revision(meta, content, backend_name)
      File "d:\\bitbucket\\moin2\\MoinMoin\\storage\\middleware\\indexing.py", line 500, in index_revision
        writer.update_document(**doc)  # update, because store_revision() may give us an existing revid
      File "D:\\Bitbucket\\moin2-venv-python\\Lib\\site-packages\\whoosh\\writing.py", line 210, in __exit__
        self.commit()
      File "D:\\Bitbucket\\moin2-venv-python\\Lib\\site-packages\\whoosh\\writing.py", line 932, in commit
        self._commit_toc(finalsegments)
      File "D:\\Bitbucket\\moin2-venv-python\\Lib\\site-packages\\whoosh\\writing.py", line 879, in _commit_toc
        toc.write(self.storage, self.indexname)
      File "D:\\Bitbucket\\moin2-venv-python\\Lib\\site-packages\\whoosh\\index.py", line 693, in write
        pickle.dumps(field)
      File "/home/web/envBASELINE\\lib\\copy_reg.py", line 70, in _reduce_ex
        raise TypeError, "can't pickle %s objects" % base.__name__
    TypeError: can't pickle function objects
    
  6. Matt Chaput repo owner

    Is the code above still used to create the schema?

    There's code there that's supposed to tell you something useful, but it isn't catching this because it's a TypeError instead of a PicklingError. As a first step to diagnosing, can you add the following lines after line 696 in whoosh/index.py:

                    except TypeError:
                        e = sys.exc_info()[1]
                        raise TypeError("%s %s=%r" % (e, fieldname, field))
    

    That should at least reveal which field is causing the problem, then we can look for what object in the field is holding on to a function.

    Thanks!

  7. RogerHaase reporter

    After finding that both the comment and content fields were failing, I added the following code before the try block:

        for fieldname, field in schema.items():
            print '@@ fieldname = %s, field = %s' % (fieldname, field)
            if fieldname in ['comment', 'content']:
                print '== test pickle fieldname = %s' % fieldname
                for d in dir(field):
                    print '=== test pickle d = %s' % d
                    try:
                        pickle.dumps(d, 2)
                    except pickle.PicklingError:
                        e = sys.exc_info()[1]
                        raise pickle.PicklingError("%s %s=%r , d= %s" % (e, fieldname, field, d))
                    except TypeError:
                        e = sys.exc_info()[1]
                        raise TypeError("%s %s=%r ==type=%s , d = %s" % (e, fieldname, field, type(field), d))
    

    The results from the log file are as follows:

    [error] @@ fieldname = action, field = ID(format=Existence(boost=1.0), scorable=None, stored=True, unique=False)
    [error] @@ fieldname = address, field = ID(format=Existence(boost=1.0), scorable=None, stored=True, unique=False)
    [error] @@ fieldname = backendname, field = ID(format=Existence(boost=1.0), scorable=None, stored=True, unique=False)
    [error] @@ fieldname = comment, field = TEXT(format=Positions(boost=1.0), scorable=True, stored=True, unique=None)
    [error] == test pickle fieldname = comment
    [error] === test pickle d = __class__
    [error] === test pickle d = __delattr__
    [error] === test pickle d = __dict__
    [error] === test pickle d = __doc__
    [error] === test pickle d = __eq__
    [error] === test pickle d = __format__
    [error] === test pickle d = __getattribute__
    [error] === test pickle d = __hash__
    [error] === test pickle d = __init__
    [error] === test pickle d = __module__
    [error] === test pickle d = __ne__
    [error] === test pickle d = __new__
    [error] === test pickle d = __reduce__
    [error] === test pickle d = __reduce_ex__
    [error] === test pickle d = __repr__
    [error] === test pickle d = __setattr__
    [error] === test pickle d = __sizeof__
    [error] === test pickle d = __str__
    [error] === test pickle d = __subclasshook__
    [error] === test pickle d = __weakref__
    [error] === test pickle d = analyzer
    [error] === test pickle d = clean
    [error] === test pickle d = column_type
    [error] === test pickle d = default_column
    [error] === test pickle d = format
    [error] === test pickle d = from_bytes
    [error] === test pickle d = from_column_value
    [error] === test pickle d = index
    [error] === test pickle d = indexed
    [error] === test pickle d = multitoken_query
    [error] === test pickle d = on_add
    [error] === test pickle d = on_remove
    [error] === test pickle d = parse_query
    [error] === test pickle d = parse_range
    [error] === test pickle d = process_text
    [error] === test pickle d = scorable
    [error] === test pickle d = self_parsing
    [error] === test pickle d = separate_spelling
    [error] === test pickle d = set_sortable
    [error] === test pickle d = sortable_terms
    [error] === test pickle d = sortable_typecode
    [error] === test pickle d = spellable_words
    [error] === test pickle d = spelling
    [error] === test pickle d = spelling_fieldname
    [error] === test pickle d = spelling_prefix
    [error] === test pickle d = stored
    [error] === test pickle d = subfields
    [error] === test pickle d = supports
    [error] === test pickle d = to_bytes
    [error] === test pickle d = to_column_value
    [error] === test pickle d = tokenize
    [error] === test pickle d = unique
    [error] === test pickle d = vector
    [error] @@ fieldname = content, field = TEXT(format=Positions(boost=1.0), scorable=True, stored=True, unique=None)
    [error] == test pickle fieldname = content
    [error] === test pickle d = __class__
    [error] === test pickle d = __delattr__
    [error] === test pickle d = __dict__
    [error] === test pickle d = __doc__
    [error] === test pickle d = __eq__
    [error] === test pickle d = __format__
    [error] === test pickle d = __getattribute__
    [error] === test pickle d = __hash__
    [error] === test pickle d = __init__
    [error] === test pickle d = __module__
    [error] === test pickle d = __ne__
    [error] === test pickle d = __new__
    [error] === test pickle d = __reduce__
    [error] === test pickle d = __reduce_ex__
    [error] === test pickle d = __repr__
    [error] === test pickle d = __setattr__
    [error] === test pickle d = __sizeof__
    [error] === test pickle d = __str__
    [error] === test pickle d = __subclasshook__
    [error] === test pickle d = __weakref__
    [error] === test pickle d = analyzer
    [error] === test pickle d = clean
    [error] === test pickle d = column_type
    [error] === test pickle d = default_column
    [error] === test pickle d = format
    [error] === test pickle d = from_bytes
    [error] === test pickle d = from_column_value
    [error] === test pickle d = index
    [error] === test pickle d = indexed
    [error] === test pickle d = multitoken_query
    [error] === test pickle d = on_add
    [error] === test pickle d = on_remove
    [error] === test pickle d = parse_query
    [error] === test pickle d = parse_range
    [error] === test pickle d = process_text
    [error] === test pickle d = scorable
    [error] === test pickle d = self_parsing
    [error] === test pickle d = separate_spelling
    [error] === test pickle d = set_sortable
    [error] === test pickle d = sortable_terms
    [error] === test pickle d = sortable_typecode
    [error] === test pickle d = spellable_words
    [error] === test pickle d = spelling
    [error] === test pickle d = spelling_fieldname
    [error] === test pickle d = spelling_prefix
    [error] === test pickle d = stored
    [error] === test pickle d = subfields
    [error] === test pickle d = supports
    [error] === test pickle d = to_bytes
    [error] === test pickle d = to_column_value
    [error] === test pickle d = tokenize
    [error] === test pickle d = unique
    [error] === test pickle d = vector
    [error] @@ fieldname = contenttype, field = TEXT(format=Positions(boost=1.0), scorable=True, stored=True, unique=None)
    [error] @@ fieldname = dataid, field = ID(format=Existence(boost=1.0), scorable=None, stored=True, unique=False)
    [error] @@ fieldname = hostname, field = ID(format=Existence(boost=1.0), scorable=None, stored=True, unique=False)
    [error] @@ fieldname = itemid, field = ID(format=Existence(boost=1.0), scorable=None, stored=True, unique=False)
    [error] @@ fieldname = itemtype, field = ID(format=Existence(boost=1.0), scorable=None, stored=True, unique=False)
    [error] @@ fieldname = language, field = ID(format=Existence(boost=1.0), scorable=None, stored=True, unique=False)
    [error] @@ fieldname = mtime, field = DATETIME(format=Existence(boost=1.0), scorable=None, stored=True, unique=False)
    [error] @@ fieldname = name, field = TEXT(format=Positions(boost=2.0), scorable=True, stored=True, unique=None)
    [error] @@ fieldname = name_exact, field = ID(format=Existence(boost=3.0), scorable=None, stored=False, unique=False)
    [error] @@ fieldname = namespace, field = ID(format=Existence(boost=1.0), scorable=None, stored=True, unique=False)
    [error] @@ fieldname = parentid, field = ID(format=Existence(boost=1.0), scorable=None, stored=True, unique=False)
    [error] @@ fieldname = ptime, field = DATETIME(format=Existence(boost=1.0), scorable=None, stored=True, unique=False)
    [error] @@ fieldname = revid, field = ID(format=Existence(boost=1.0), scorable=None, stored=True, unique=True)
    [error] @@ fieldname = size, field = NUMERIC(format=Existence(boost=1.0), scorable=None, stored=True, unique=False)
    [error] @@ fieldname = summary, field = TEXT(format=Positions(boost=1.0), scorable=True, stored=True, unique=None)
    [error] @@ fieldname = tags, field = ID(format=Existence(boost=1.0), scorable=None, stored=True, unique=False)
    [error] @@ fieldname = trash, field = BOOLEAN(format=Existence(boost=1.0), scorable=None, stored=True, unique=None)
    [error] @@ fieldname = userid, field = ID(format=Existence(boost=1.0), scorable=None, stored=True, unique=False)
    [error] @@ fieldname = wikiname, field = ID(format=Existence(boost=1.0), scorable=None, stored=True, unique=False)
    [error] 2016-02-28 10:33:45,489 ERROR MoinMoin.signalling.log:30 can't pickle function objects comment=TEXT(format=Positions(boost=1.0), scorable=True, stored=True, unique=None) ==type=<class 'whoosh.fields.TEXT'> 
    [error] Traceback (most recent call last):
    [error]   File "D:\\Bitbucket\\moin2-venv-python\\Lib\\site-packages\\flask\\app.py", line 1817, in wsgi_app
    [error]     response = self.full_dispatch_request()
    [error]   File "D:\\Bitbucket\\moin2-venv-python\\Lib\\site-packages\\flask\\app.py", line 1477, in full_dispatch_request
    [error]     rv = self.handle_user_exception(e)
    [error]   File "D:\\Bitbucket\\moin2-venv-python\\Lib\\site-packages\\flask\\app.py", line 1381, in handle_user_exception
    [error]     reraise(exc_type, exc_value, tb)
    [error]   File "D:\\Bitbucket\\moin2-venv-python\\Lib\\site-packages\\flask\\app.py", line 1475, in full_dispatch_request
    [error]     rv = self.dispatch_request()
    [error]   File "D:\\Bitbucket\\moin2-venv-python\\Lib\\site-packages\\flask\\app.py", line 1461, in dispatch_request
    [error]     return self.view_functions[rule.endpoint](**req.view_args)
    [error]   File "d:\\bitbucket\\moin2\\MoinMoin\\apps\\frontend\\views.py", line 637, in modify_item
    [error]     return item.do_modify()
    [error]   File "d:\\bitbucket\\moin2\\MoinMoin\\items\\__init__.py", line 882, in do_modify
    [error]     self.modify(meta, data, comment, contenttype_guessed, **{CONTENTTYPE: contenttype_qs})
    [error]   File "d:\\bitbucket\\moin2\\MoinMoin\\items\\__init__.py", line 496, in modify
    [error]     return self._save(meta, data, contenttype_guessed=contenttype_guessed, comment=comment)
    [error]   File "d:\\bitbucket\\moin2\\MoinMoin\\items\\__init__.py", line 638, in _save
    [error]     return_rev=True,
    [error]   File "d:\\bitbucket\\moin2\\MoinMoin\\storage\\middleware\\protecting.py", line 315, in store_revision
    [error]     rev = self.item.store_revision(meta, data, overwrite=overwrite, return_rev=return_rev, fqname=fqname, **kw)
    [error]   File "d:\\bitbucket\\moin2\\MoinMoin\\storage\\middleware\\indexing.py", line 1135, in store_revision
    [error]     self.indexer.index_revision(meta, content, backend_name)
    [error]   File "d:\\bitbucket\\moin2\\MoinMoin\\storage\\middleware\\indexing.py", line 500, in index_revision
    [error]     writer.update_document(**doc)  # update, because store_revision() may give us an existing revid
    [error]   File "D:\\Bitbucket\\moin2-venv-python\\Lib\\site-packages\\whoosh\\writing.py", line 210, in __exit__
    [error]     self.commit()
    [error]   File "D:\\Bitbucket\\moin2-venv-python\\Lib\\site-packages\\whoosh\\writing.py", line 932, in commit
    [error]     self._commit_toc(finalsegments)
    [error]   File "D:\\Bitbucket\\moin2-venv-python\\Lib\\site-packages\\whoosh\\writing.py", line 879, in _commit_toc
    [error]     toc.write(self.storage, self.indexname)
    [error]   File "D:\\Bitbucket\\moin2-venv-python\\Lib\\site-packages\\whoosh\\index.py", line 716, in write
    [error]     raise TypeError("%s %s=%r ==type=%s " % (e, fieldname, field, type(field)))
    [error] TypeError: can't pickle function objects comment=TEXT(format=Positions(boost=1.0), scorable=True, stored=True, unique=None) ==type=<class 'whoosh.fields.TEXT'>
    

    On the 2016-02-23 traceback. the python virtual env's were used different python versions. In the trraceback above, all virtual env's are using python 2.7.8.

    The 2013-02-19 initial report states that accessing any app before moin 2 causes the pickling error - that is incorrect, only accessing moin 1.9 first causes the error. Accessing the other two apps first has no adverse effect (the png and svg image servers are rather simple and do not use pickle/cpickle).

  8. Matt Chaput repo owner

    I don't understand this output... the prints seem to indicate that everything in comment.__dict__ pickled fine, but then at some later point comment failed to pickle properly?

    Also, earlier in this thread you posted the supposed schema that was causing the problem, but the schema you pasted in doesn't have a comment field? Can you point me to the source code where the comment field is defined?

    Thanks!

  9. Log in to comment