git clone fails with HTTP-Fehler 417 Expectation failed

Issue #33 resolved
Sven R. Kunze created an issue

This error occurs sometimes (but not always) when doing a pip install -e git+https://....

Comments (19)

  1. Sven R. Kunze reporter

    https:// is a URL refering to our kallithea instance.

    We actually updated from rhodecode to kallithea because of this error. Unfortunately, it persists.

  2. domruf

    Do you have any log entries that shed some light on this problem?

    The problem only happens on git repositories? The problem never happens when you clone the repository regularly?

  3. Sven R. Kunze reporter

    I will see if I can get my hands on some logfiles. I remember us looking through them when using rhodecode, but the error was not comprehensible.

    We indedd serve git repositories only. It happens occasionally and there is no apparent pattern. After talking to my team members, they also incurred it when doing "git clone https://" instead of "pip install -e git+https://....".

  4. Sven R. Kunze reporter

    Btw. pip install -e does not only git clone but also git fetch/git pull. We run into that situation as well as when using pip install -e to create and update of a local git repository.

  5. Mads Kiilerich

    For the --depth clone issue, please try

    --- a/kallithea/lib/vcs/
    +++ b/kallithea/lib/vcs/
    @@ -375,14 +375,20 @@ class SubprocessIOChunker(object):
                 except Exception:
    +            out = ''.join(bg_out)
    -            err = '%s' % ''.join(bg_err)
    -            if err:
    +            err = ''.join(bg_err)
    +            if (err.strip() == 'fatal: The remote end hung up unexpectedly' and
    +                out.startswith('0034shallow ')):
    +                # hack inspired by
    +                bg_out = iter([out])
    +                _p = None
    +            elif err:
                     raise EnvironmentError(
                         "Subprocess exited due to an error:\n" + err)
    -            raise EnvironmentError(
    -                "Subprocess exited with non 0 ret code:%s" % _returncode)
    +            else:
    +                raise EnvironmentError(
    +                    "Subprocess exited with non 0 ret code:%s" % _returncode)
             self.process = _p
             self.output = bg_out
             self.error = bg_err
    @@ -392,7 +398,7 @@ class SubprocessIOChunker(object):
             return self
         def next(self):
    -        if self.process.poll():
    +        if self.process and self.process.poll():
                 err = '%s' % ''.join(self.error)
                 raise EnvironmentError("Subprocess exited due to an error:\n" + err)
  6. Sven R. Kunze reporter

    Is there a testcase to provoke the 417? We would not want to jeopardize our production system.

  7. Mads Kiilerich

    I don't know. You reported the problem ;-)

    You also suggested it could be related to --depth clones ... which this patch seems to fix.

  8. Sven R. Kunze reporter

    Well, I meant something like: py.test -k test_417. I also think that would be a good addition in order to indicate such problems in the future.

    If you are confident that this patch works, we look forward to the next version of kallithea as it will fix other problems as well.

  9. Mads Kiilerich

    This is a collaboration project. We would appreciate contribution of test coverage for this.

    I'm only confident your problem has been solved when you tell me.

  10. Sven R. Kunze reporter

    I understand that. However, we have no idea how to provoke that error. It occurs completely randomly. This might be related as well:

    Right now, we had this:

    error: RPC failed; result=22, HTTP code = 500
    fatal: The remote end hung up unexpectedly

    apache-log shows:

    [Wed Sep 17 11:51:47 2014] [error] [client IP] Premature end of script headers:
    [Wed Sep 17 11:51:47 2014] [error] [client IP] no acceptable variant: /usr/share/apache2/error/HTTP_INTERNAL_SERVER_ERROR.html.var

    Are there some other logs I can sift through?

    I will give your patch a try.

  11. Mads Kiilerich

    I would suggest running it in the foreground with paster and a single client hammering it. That will usually give better debug output.

  12. Sven R. Kunze reporter

    Maybe, this is related to our #32 .

    I have a script running for several hours now after applying that fix and no error so far. I will have this script run for several days in order to make sure it works reliably.

    Do you think the patch above is still necessary when the patch of #32 works as well?

  13. Mads Kiilerich

    I could reproduce something that looked like the failure rhodecode issue you linked to when trying to clone with git --depth ... and the patch above seemed to fix it ... and it is unrelated to the fix for #32.

    I do however not know anything about git (or care that much about it) so I would appreciate if someone with git knowledge could review and/or give it some serious testing.

    It is possible that your initial problem was a duplicate of #32 but you gave a hint towards another and real problem.

  14. Sven R. Kunze reporter

    Another random error:

    error: RPC failed; result=22, HTTP code = 403
    fatal: The remote end hung up unexpectedly

    Apache-Log still reveals nothing.

  15. Log in to comment