Issue #6666 open

Detect git requests by Content-type header, not user agent (BB-8843)

gsalgado
created an issue

Hi,

Bitbucket seems to be restricting non-git User-Agents, and that in turn prevents Launchpad from importing branches. It has first been reported on https://answers.launchpad.net/launchpad/+question/218551 but the problem still exists.

Comments (70)

  1. Jesper Nøhr

    We are not restricting, we are routing based on the user agent, yes. Launchpad will need to send a git-like user agent to talk to our git backends. Anything containing git/ will do.

  2. Jelmer Vernooij

    You're encouraging clients to spoof their user agent, which seems like a bad idea.

    There are plenty of other signals available to tell the git client apart from user clients. Git clients will not request the root of a repository for example, but just a select number of paths below it.

    Git clients that support the smart server protocol (anything after 1.6.6, from 2009) will also set a custom Content-Type header that you could look for.

    Forcing the usage of git/ in the user agent doesn't break just Launchpad but e.g. the git plugins for Mercurial and Bazaar (hg-git and bzr-git) too.

  3. Gordo Lowrey

    this seems really trivial, both on the side of app developers who refuse to add a useragent header, and also on the side of bitbucket refusing to accept non-git useragents.

    what a let down, on all sides.

  4. Gordo Lowrey

    that may be true, but it would be trivial to setup a generic middleware to override the useragent. just something to consider until bitbucket devs get on the ball.

  5. Bret Hash

    Piling on the Tailor issue train. Some update from Bitbucket with an approximate ETA would be appreciated so I should know if I need to move my repositories or not.

  6. marcus_holmes

    Just bought a Chromebook, want to use it for development. If Bitbucket won't host my repos in a way that allows this, I'll be forced to move them. Tried several tools and wondered wtf was going on until Tailor pointed me at this issue.

  7. Thomas White

    marcus_holmes If you unlock dev mode on your Chromebook you get a proper shell and can use git normally, however this wipes the device completely.

    That having been said, this is a huge pain, and I hope it gets fixed.

  8. niceoboe

    This is kind of frustrating.

    I'd rather like to be able to develop on my Chromebook with vanilla Chrome OS, rather than have to resort to workarounds like crouton or going full bore and installing a completely different distro.

    This bug has been around for over a year, but it's marked 'major'. :/

  9. Gordo Lowrey

    ryanackley you are wrong, actually, at least, in regards to Chrome apps, which Tailor is.

    Setting the user-agent is so sooooo trivial. And no, you don't even have to fully override anything... you can even just append "git/YOURAPPNAME" to the end of the normal user-agent as Jesper Nøhr suggested, and it would work.

    For all you people complaining about Chromebooks and Tailor: there are many many many ways around this, and the fact that nobody has yet stepped forward to mention this is rather disappointing. The same goes for people complaining about Launchpad and other back-end apps.

    Routing based on the user-agent may not be a the best design decision on the part of the Bitbucket developers, but it's a trivial thing to work around.

    I just checked the Tailor code repository,and it's severely outdated, like, almost a year old. Maybe that's the problem, ya think? The Adobe Brackets repo is way ahead of the Tailor repo, which is where Tailor was forked from.

    Chrome extensions, and thus apps, absolutely can indeed change the user-agent header, and this is thoroughly documented and there are examples floating around the web of this as well.

    https://developer.chrome.com/extensions/webRequest

    https://stackoverflow.com/questions/10334909/associate-a-custom-user-agent-to-a-specific-google-chrome-page-tab/10339902#10339902

    https://gist.github.com/jugglinmike/1124237

    There are also lower-level socket connections too.

    https://developer.chrome.com/apps/app_network

    Tailor is a wrapped Chrome app, and thus has access to these APIs.

    'nuff said.

  10. ryanackley

    hey Gordo Lowrey, where is the anger coming from? I'm a volunteer remember. Tailor is free. I've never tried to make a cent off of it even though I have put a lot of time into it. I also have no desire to make bitbucket look bad since I worked at Atlassian for 5 years and left on great terms.

    Thanks for figuring out how to do this although I think you have the wrong attitude. It's open source, channel some of your frustration into fixing it yourself instead of abusing me over it.

  11. Gordo Lowrey

    Hi ryanackley,

    No anger, honestly! Sorry for being short and blunt, however. :)

    That being said... it's rather disheartening to see a community of developers all jump in and brigade and finger-point like this, instead of attempting to solve the problem themselves.

    Just adding one more request to get this working with Tailor.

    Piling on the Tailor issue train.

    Another Tailor desperate for a fix

    Tailor needs this

    I'm also trying to use Tailor

    Tailor & bitbucket user here. Please fix this issue.

    Tailor pointed me at this issue

    I don't know who is sending these users here, since the Tailor repository does not have issues enabled, but if you really want to foster community and innovation, maybe enable issues so people can collaborate there.

    I would be happy to take a shot at this... perhaps soon when I have more free time to research further, since I've never written a Chrome app.

    If there's enough people that would use it, I could much more easily throw up a proxy to forward to Bitbucket with git/forwarded as the user-agent header. Users would need to trust that I'm not skimming off their Bitbucket credentials, however.

    Something along these lines on nginx would do the trick, I think:

    location / {
        proxy_pass https://bitbucket.org$request_uri;
        proxy_set_header User-Agent "git/forwarded";
        proxy_set_header Authorization $http_authorization;
        proxy_buffering off;
    }
    
  12. Maksim Lin

    Gordo Lowrey I'm guessing you have not had a chance to try this as yet in practise, as I have and in a Chrome Packaged App you can NOT set the User Agent header, if you try to you get a: "Refused to set unsafe header "User-Agent"" error.

    Those links you provided are actually all for chrome extensions, not packaged apps which have different sets of permissions and available APIs as is actually documented on the https://developer.chrome.com site you linked to.

    A possible change to this policy for Chrome packaged Apps may be considered in the future, you can see this bug report for more: https://code.google.com/p/chromium/issues/detail?id=335934

    BUT in the meantime, there is NO work around available for Chrome Packaged Apps trying to access git repos at BB via HTTPS as even if we did go down the road of implementing a custom HTTP client over the socket api available to Chrome Apps, there is no support for TLS in chrome socket api atm.

    So could BB pls re-consider and not use user-agent based routing.

    thanks.

  13. Gabriel D

    SAM Brickell I don't believe anyone was looking for web based ide's i believe the were looking to use a browser extension that does not require them to have the internet for 90% of the time while using it.

    plus there's also nitrous.io and more c9 isn't the only one...

  14. SAM Brickell

    Wow.

    -----" I don't believe anyone was looking for web based ide's"

    First of all I don't believe you are qualified to define what ""anybody"" could possibly be looking for. I was initially looking for what you describe and when I eventually worked a little with Cloud9 I appreciated it and figured I would share for anybody else.

    ----"i believe the were looking to use a browser extension that does not require them to have the internet for 90% of the time while using it." (Nice capitalization.)

    And we live in an era where the internet is fairly integral to mostly everything. I believe most people who own Chromebooks [anybody interested in this thread] know this. Obviously something that works in addition to online would be better but in a world where you own a Chromebook and use Bitbucket and nothing else works: use the internet. (Oh wait a minute! Also since Cloud9 allows easy folder uploads [even without being zipped] you are able to work offline if you wish.)

    ----"plus there's also nitrous.io and more c9 isn't the only one..." (Nice capitalization.)

    Nor do I believe that anybody of reasonable intelligence could be under the impression that I was indicating that Cloud9 is in fact the only possible web ide.

    Now I was trying to help by telling what I found to be a useful solution to the issue in a way I found helpful and that is why I commented. Why did you? Because it seems more like you wanted to complain and show off your lack of comprehension of proper grammar and capitalization rather than interest in helping people.

  15. SAM Brickell

    Oh, and I would have been happy to help anybody else that replied to this thread and maybe had a question about Cloud9 that I could answer for them or anything else about that. But instead I'm unfollowing and blocking this thread to get away from you Gabby. So congratulations.

  16. marcus_holmes

    well that escalated fast.

    For those interested, I solved this problem: I moved my repos to github. Atlassian, if you don't listen to your customers, you lose them.

    unfollowing the thread to avoid the bile in my inbox.

  17. Tim Caswell

    For what's it worth, there is no technical reason Taylor can't set a custom user agent. Chrome apps have raw TCP access. I have an HTTP implementation in JS I use for js-git at https://github.com/creationix/http-codec. Since taylor's git implementation doesn't have a pluggable backend and is hard-coded to using XHR it gets bit by this issue.

    On the other hand, my goal with js-git is to allow websites to talk to github and bitbucket in which case XHR is the only primitive I have. So I need this fixed as well as the CORS headers issue.

  18. Tóth Róbert

    I wanted to try out Chrome Dev editor again, but i got this issue, which is a little disheartening, because I could clone my github repo without a hitch.

  19. Andrew Tipton

    Just discovered this issue when trying to use the Chrome Dev Editor on my new Chromebook. This issue has been active for, what, a year and a half? And not a single update from Atlassian.

    Shame.

  20. Trever Nightingale

    Chrome OS users:

    This won't help you if you want to use Tailor or CDE, but it may help you to know that as of two nights ago, the following Google developed web app for Chrome has git with https, and therefore works with bitbucket: https://chrome.google.com/webstore/detail/nacl-development-environm/aljpgkjeipgnmdpikaajmnepbcfkglfa?utm_source=chrome-app-launcher-info-dialog

    The NaCl development environment includes many things, vim and emacs amoung them.

  21. Kaleb Elwert staff

    We started routing based on the Content-Type header a while back. However, Chrome Dev Editor does not send that header along with the initial refs discovery (even git doesn't send this header for that part of the protocol). For more information, check here under the "smart" sections: https://git.kernel.org/cgit/git/git.git/tree/Documentation/technical/http-protocol.txt

    We are also looking into alternate ways of routing git requests so additional clients will work.

  22. richard bucker

    you guys are really smart people and while I see that you're making some recent progress I urge you to make this a priority. There are a number of Chromebook applications that are blaming BB for a Git issue which is in-turn forcing me to deploy my own git repo or use you competitor.

  23. Log in to comment