git push fails with HTTP 502

Issue #586 resolved
Paweł Widera
created an issue

I have created a new git repo in rhodecode, clone it locally, made some changes and on push back to rhodecode I get: {{{

!python

$> git push http://user@host/code/gittest master error: The requested URL returned error: 502 while accessing http://user@host/code/gittest/info/refs fatal: HTTP request failed }}}

And this is what I see in the rhodecode logs: {{{

!python

Traceback (most recent call last): File "/opt/rhodecode-virtenv/local/lib/python2.7/site-packages/paste/deploy/config.py", line 291, in call return self.app(environ, start_response) File "/opt/rhodecode-virtenv/local/lib/python2.7/site-packages/paste/gzipper.py", line 35, in call return self.application(environ, start_response) File "/opt/rhodecode-virtenv/local/lib/python2.7/site-packages/paste/cascade.py", line 130, in call return self.apps-1 File "/opt/rhodecode-virtenv/local/lib/python2.7/site-packages/paste/registry.py", line 379, in call app_iter = self.application(environ, start_response) File "/opt/rhodecode-virtenv/local/lib/python2.7/site-packages/rhodecode/lib/middleware/https_fixup.py", line 37, in call return self.application(environ, start_response) File "/opt/rhodecode-virtenv/local/lib/python2.7/site-packages/pylons/middleware.py", line 150, in call self.app, environ, catch_exc_info=True) File "/opt/rhodecode-virtenv/local/lib/python2.7/site-packages/pylons/util.py", line 48, in call_wsgi_application app_iter = application(environ, start_response) File "/opt/rhodecode-virtenv/local/lib/python2.7/site-packages/rhodecode/lib/base.py", line 212, in call return self._handle_request(environ, start_response) File "/opt/rhodecode-virtenv/local/lib/python2.7/site-packages/rhodecode/lib/middleware/simplegit.py", line 164, in _handle_request result = self.authenticate(environ) File "/opt/rhodecode-virtenv/local/lib/python2.7/site-packages/paste/auth/basic.py", line 48, in authenticate username, password = auth.split(':', 1) ValueError: need more than 1 value to unpack [pid: 26790|app: 0|req: 1/1] 123.123.123.123 () {44 vars in 667 bytes} [Mon Oct 1 15:10:53 2012] GET /code/gittest/info/refs?service=git-receive-pack => generated 0 bytes in 75 msecs (HTTP/1.0 500) 0 headers in 0 bytes (0 switches on core 0) }}}

Comments (10)

  1. Gilles Bouthenot

    I think I found a bug in paster:

    File "/home/rhodecode/homes/env/lib/python2.6/site-packages/Paste-1.7.5.1-py2.6.egg/paste/auth/basic.py", l:30 2012] [error] [client 195.221.254.2] username, password = auth.split(':', 1)

    This occurs while trying to authenticate with Git using Basic Authentication. The thing is, git does only send this Http header:

    Authorization: Basic base64("mylogin")

    It then waits for the HTTP server to issue a HTTP_UNAUTHORIZED (401), and then it sends the header:

    Authorization: Basic base64("mylogin:mypassword")

    So the first time, paster raises the exception above, because, auth.split(':', 1) can not split, because there is no ":" character.

    I think it is a bug in Paste, but I am a complete newbie in Python. This is the first time I have to do with a python program. Maybe this bug has been corrected in later versions, I don't know.

    Anyway, rhodecode WSGI catches the Exception, and the HTTP_401 is not issued. Wsgi sends a HTTP_500 instead.

    I made this quick and dirty (remember : two days ago I knew NOTHING about python !)

    I tested it, and it works. Cloning a git repo returns a Authentication failed with a wrong password, and success with the right password.

    Mercurial is not impacted, because it correctly sends the right "Authorization" header from the start.

    Hope it helps.

    diff -r 2454ed7f0c21 rhodecode/lib/middleware/simplegit.py

    --- a/rhodecode/lib/middleware/simplegit.py Wed Sep 05 01:39:13 2012 +0200
    +++ b/rhodecode/lib/middleware/simplegit.py Wed Sep 05 23:24:08 2012 +0200
    @@ -79,6 +79,9 @@
    from webob.exc import HTTPNotFound, HTTPForbidden, HTTPInternalServerError, \
    HTTPBadRequest, HTTPNotAcceptable
    +from paste.httpexceptions import HTTPUnauthorized
    +from paste.httpheaders import *
    +
    from rhodecode.lib.utils2 import safe_str
    from rhodecode.lib.base import BaseVCSController
    from rhodecode.lib.auth import get_container_username
    @@ -161,12 +164,20 @@
    if not username:
    self.authenticate.realm = \
    safe_str(self.config['rhodecode_realm'])
    - result = self.authenticate(environ)
    +
    + result = None
    + try:
    + result = self.authenticate(environ)
    + except:
    + pass
    +
    if isinstance(result, str):
    AUTH_TYPE.update(environ, 'basic')
    REMOTE_USER.update(environ, result)
    username = result
    else:
    + head = WWW_AUTHENTICATE.tuples('Basic realm="%s"', safe_str(self.config['rhodecode_realm']))
    + result = HTTPUnauthorized(headers=head)
    return result.wsgi_application(environ, start_response)
    #==============================================================
    

    Oh, I forgot: git --version git version 1.7.2.5 (on a debian stable)

    This problem only appears when using a proxy (not reverse, but a corporate squid proxy).

    tcpdump see this first frame without proxy:

    GET /Sokaris/info/refs?service=git-upload-pack HTTP/1.1
    User-Agent: git/1.7.2.5
    Host: rhodecode.dsi.univ-fcomte.fr
    

    And with proxy:

    GET http://gbouthen@rhodecode.dsi.univ-fcomte.fr/Sokaris/info/refs?service=git-upload-pack
    HTTP/1.1
    User-Agent: git/1.7.2.5
    Host: rhodecode.dsi.univ-fcomte.fr
    

    (and then the proxy forward this to git:)

    GET /Sokaris/info/refs?service=git-upload-pack HTTP/1.0
    User-Agent: git/1.7.2.5
    Host: rhodecode.dsi.univ-fcomte.fr
    Accept: */*
    Pragma: no-cache
    Via: 1.1 proxy2.fcomte.iufm.fr:3128 (squid/2.6.STABLE5)
    X-Forwarded-For: 172.20.16.247
    Authorization: Basic Z2JvdXRoZW4=
    
    

    (Note: Authorization is "Basic gbouthen", there is no password).

    This only occurs while using a proxy. Git sends the username ONLY when he talks to a proxy. When he talks to rhodecode it did not, get a 401, then retry with username+password).

    Again, this is not a bug in rhodecode, more likely in Paste. But since my university enforces proxy usage, I will have to use the patched version.

  2. Gilles Bouthenot

    I said that I found a bug in Paste, because on my installation, the line "username, password = auth.split(':', 1)" throws an Exception. The exception is catched in your case, maybe you are using a more recent version of Paste.

    I bet you are using a proxy. Git HTTP support i strange when using a proxy (see the tcpdump above). Git sends a username to the server, but does not provide a password. In that case, git should issue a HTTP-401, then ask the password, then resend the request, this time with user/password.

    I'm quite sure, this is a git bug (or a bug in squid). It should not send a request with an user name and no password.

    By the way, you can probably work around this "bug?" by NOT PROVIDING your username in the git URL, so git won't send your username. Another test to make is to use an url like "https://username:NOTYOURPASSWORD@server.org/reponame" (replacing your user name, and NOT replacing your password), so git will have a failed password, then ask your password an other time.

    I have no time to test right now.

    (anyway, rhodecode should not crash, so the exception should probably be handled, see my patch above)

  3. Paweł Widera reporter

    That's seems to be a spot on analysis. It is indeed proxy that makes the difference. With proxy disabled, I'm able to push to git repo no problem.

    If I don't provide a user name it works, git asks for both user and a password and then authenticates without problems. Using fake password doesn't work as it results in failed authentication (there is no second try).

    I'm using PasteScript 1.7.5 with UWSGI and this might be the reason why the exception is not silenced.

  4. Marcin Kuzminski repo owner

    I already subclass the paste auth as ```class BasicAuth(AuthBasicAuthenticator):``` I think it's better to apply the fix there, i'll look at some clean solution there

  5. Marcin Kuzminski repo owner

    Proposed initial version of patch, but i must read little more RFC docs before doing this. Not sure if this is the right way to do it

    diff --git a/rhodecode/lib/base.py b/rhodecode/lib/base.py
    --- a/rhodecode/lib/base.py
    +++ b/rhodecode/lib/base.py
    @@ -3,18 +3,17 @@
     Provides the BaseController class for subclassing.
     """
     import logging
     import time
     import traceback
     
     from paste.auth.basic import AuthBasicAuthenticator
     from paste.httpexceptions import HTTPUnauthorized, HTTPForbidden
    -from webob.exc import HTTPClientError
    -from paste.httpheaders import WWW_AUTHENTICATE
    +from paste.httpheaders import WWW_AUTHENTICATE, AUTHORIZATION
     
     from pylons import config, tmpl_context as c, request, session, url
     from pylons.controllers import WSGIController
     from pylons.controllers.util import redirect
     from pylons.templating import render_mako as render
     
     from rhodecode import __version__, BACKENDS
     
    @@ -69,16 +68,34 @@ class BasicAuth(AuthBasicAuthenticator):
         def build_authentication(self):
             head = WWW_AUTHENTICATE.tuples('Basic realm="%s"' % self.realm)
             if self._rc_auth_http_code and self._rc_auth_http_code == '403':
                 # return 403 if alternative http return code is specified in
                 # RhodeCode config
                 return HTTPForbidden(headers=head)
             return HTTPUnauthorized(headers=head)
     
    +    def authenticate(self, environ):
    +        authorization = AUTHORIZATION(environ)
    +        if not authorization:
    +            return self.build_authentication()
    +        (authmeth, auth) = authorization.split(' ', 1)
    +        if 'basic' != authmeth.lower():
    +            return self.build_authentication()
    +        auth = auth.strip().decode('base64')
    +        _parts = auth.split(':', 1)
    +        if len(_parts) < 2:
    +            username = _parts[0]
    +            password = None
    +        elif _parts:
    +            username, password = _parts
    +        if self.authfunc(environ, username, password):
    +            return username
    +        return self.build_authentication()
    +
     
     class BaseVCSController(object):
     
         def __init__(self, application, config):
             self.application = application
             self.config = config
             # base path of repo locations
             self.basepath = self.config['base_path
    
  6. Paweł Widera reporter

    Unfortunately this is not good enough. Firstly, the new authenticate method is not called by simplegit.py. To make it so, the following have to be added to the BaseAuth object:

    __call__ = authenticate
    

    But then, when the URL contains only the username, the password is set to None and this fails to produce a hash inside authfunc:

      File "/opt/rhodecode-virtenv/local/lib/python2.7/site-packages/rhodecode/lib/base.py", line 90, in authenticate
        if self.authfunc(environ, username, password):
      File "/opt/rhodecode-virtenv/local/lib/python2.7/site-packages/rhodecode/lib/auth.py", line 155, in authfunc
        return authenticate(username, password)
      File "/opt/rhodecode-virtenv/local/lib/python2.7/site-packages/rhodecode/lib/auth.py", line 180, in authenticate
        user.password):
      File "/opt/rhodecode-virtenv/local/lib/python2.7/site-packages/rhodecode/lib/auth.py", line 131, in check_password
        return RhodeCodeCrypto.hash_check(password, hashed)
      File "/opt/rhodecode-virtenv/local/lib/python2.7/site-packages/rhodecode/lib/auth.py", line 120, in hash_check
        return bcrypt.hashpw(password, hashed) == hashed
    TypeError: hashpw() argument 1 must be string, not None
    

    To solve that, the build_authentication function has to be called.

        def authenticate(self, environ):
            authorization = AUTHORIZATION(environ)
            if not authorization:
                return self.build_authentication()
            (authmeth, auth) = authorization.split(' ', 1)
            if 'basic' != authmeth.lower():
                return self.build_authentication()
            auth = auth.strip().decode('base64')
            _parts = auth.split(':', 1)
            if len(_parts) == 2:
                username, password = _parts
                if self.authfunc(environ, username, password):
                    return username
            return self.build_authentication()
    
        __call__ = authenticate
    
  7. Log in to comment