waitress exception when serving file

Issue #219 resolved
Igor Tretyak created an issue

Hello,

Kallithea runs in docker container.

pip modules versions

Babel==1.3
Beaker==1.6.4
FormEncode==1.2.6
Kallithea==0.3
Mako==1.0.0
Markdown==2.2.1
MarkupSafe==0.23
MySQL-python==1.2.5
Paste==2.0.2
PasteDeploy==1.5.2
PasteScript==2.0.2
Pygments==2.1
Pylons==1.0.2
Routes==1.13
SQLAlchemy==0.7.10
Tempita==0.5.3.dev0
URLObject==2.3.4
WebError==0.11
WebHelpers==1.3
WebOb==1.1.1
WebTest==1.4.3
Whoosh==2.5.7
amqplib==1.0.2
anyjson==0.3.3
argparse==1.2.1
celery==2.2.10
chardet==2.0.1
colorama==0.2.5
decorator==4.0.6
docutils==0.11
dulwich==0.9.9
funcsigs==0.4
html5lib==0.999
kombu==1.5.1
mercurial==3.6.3
mock==1.3.0
nose==1.3.7
pbr==1.8.1
psycopg2==2.6.1
py-bcrypt==0.4
pyparsing==1.5.7
python-dateutil==1.5
python-ldap==2.4.10
pytz==2015.7
repoze.lru==0.6
requests==2.2.1
simplejson==3.8.1
six==1.10.0
urllib3==1.7.1
waitress==0.8.8
wheel==0.24.0
wsgiref==0.1.2
# docker version
Client:
 Version:      1.9.1
 API version:  1.21
 Go version:   go1.4.2
 Git commit:   a34a1d5
 Built:        Fri Nov 20 13:25:01 UTC 2015
 OS/Arch:      linux/amd64

Server:
 Version:      1.9.1
 API version:  1.21
 Go version:   go1.4.2
 Git commit:   a34a1d5
 Built:        Fri Nov 20 13:25:01 UTC 2015
 OS/Arch:      linux/amd64

No OS errors, regarding FS or disk, was found.

faced with exeption:

2016-05-24 09:01:30.601 ERROR [waitress] uncaptured python exception, closing channel <waitress.server.TcpWSGIServer listening 0.0.0.0:5000 at 0x7f2c0050f610> (<class 'socket.error'>:[Errno 9] Bad file descriptor [/usr/lib/python2.7/asyncore.py|read|83] [/usr/lib/python2.7/asyncore.py|handle_read_event|443] [/usr/local/lib/python2.7/dist-packages/waitress-0.8.8-py2.7.egg/waitress/server.py|handle_accept|147] [/usr/local/lib/python2.7/dist-packages/waitress-0.8.8-py2.7.egg/waitress/channel.py|__init__|80] [/usr/lib/python2.7/asyncore.py|__init__|250] [/usr/lib/python2.7/socket.py|meth|224])
2016-05-24 09:01:30.601 WARNI [waitress] unhandled close event
2016-05-24 09:01:30.608 ERROR [waitress] Exception when serving /Library-Android/skia/libwebp.git/git-upload-pack
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/waitress-0.8.8-py2.7.egg/waitress/channel.py", line 337, in service
    task.service()
  File "/usr/local/lib/python2.7/dist-packages/waitress-0.8.8-py2.7.egg/waitress/task.py", line 173, in service
    self.execute()
  File "/usr/local/lib/python2.7/dist-packages/waitress-0.8.8-py2.7.egg/waitress/task.py", line 424, in execute
    self.write(chunk)
  File "/usr/local/lib/python2.7/dist-packages/waitress-0.8.8-py2.7.egg/waitress/task.py", line 307, in write
    channel.write_soon(towrite)
  File "/usr/local/lib/python2.7/dist-packages/waitress-0.8.8-py2.7.egg/waitress/channel.py", line 319, in write_soon
    self.outbufs[-1].append(data)
  File "/usr/local/lib/python2.7/dist-packages/waitress-0.8.8-py2.7.egg/waitress/buffers.py", line 253, in append
    self._set_large_buffer()
  File "/usr/local/lib/python2.7/dist-packages/waitress-0.8.8-py2.7.egg/waitress/buffers.py", line 236, in _set_large_buffer
    self.buf = TempfileBasedBuffer(self.buf)
  File "/usr/local/lib/python2.7/dist-packages/waitress-0.8.8-py2.7.egg/waitress/buffers.py", line 116, in __init__
    FileBasedBuffer.__init__(self, self.newfile(), from_buffer)
  File "/usr/local/lib/python2.7/dist-packages/waitress-0.8.8-py2.7.egg/waitress/buffers.py", line 120, in newfile
    return TemporaryFile('w+b')
  File "/usr/lib/python2.7/tempfile.py", line 498, in TemporaryFile
    _os.close(fd)
OSError: [Errno 9] Bad file descriptor

Kallithea sometimes fails on pulling file from same folder. I can't reproduce it manually by executing 'git clone'.

Comments (9)

  1. Mads Kiilerich

    Why not use the latest stable Kallithea version?

    The stacktrace shows all waitress - no Kallithea there ...

    What does 'serving file' mean?

    I guess you see it with Git only? Can you explain in details how to reproduce?

  2. Igor Tretyak reporter

    I haven't seen any waitress or related updates in stable Kallithea. Anyway, tonight I updated Kallithea, Docker and all other system components. We'll see whether the bug gone away.

  3. Thomas De Schampheleire

    Another 'Bad file descriptor' problem (happening on git push of a large file, e.g. 1MB or larger) was reported on the mailing list. I reproduced the issue on stable branch (0.3.5) but the problem is gone on the default branch. In both cases, waitress and sqlite were used. See https://lists.sfconservancy.org/pipermail/kallithea-general/2018q2/002667.html

    There is also issue #229 which looks related.

    In all three cases it seems to be something with waitress. But because backstepping the version of waitress on the default branch to the same version as used on stable (waitress 0.8.8) does not reintroduce the problem, there thus must be some other relevant factor.

    In any case, there are following workarounds: - on stable, use a different web server than waitress - or switch to default branch (with any web server)

    Hence, I will close this issue.

  4. Log in to comment