Bitbucket relies on the Referer HTTP header (BB-11761)

Issue #10726 wontfix
Moritz Wilhelmy
created an issue

The Referer header is optional as per HTTP specification. Bitbucket thus breaks the specification.

This is a major annoyance for me, especially since it's not possible to selectively turn on the Referer header only on specific sites.

Comments (22)

  1. John Garcia

    Hey there, to help us understand the scope of this issue, please provide the following:

    • What is it (specifically) that you're trying to do?
    • What is the expected result?
    • What is the actual result?

    Thanks!

  2. Moritz Wilhelmy reporter

    Using firefox, having network.sendRefererHeader set to 0 in about:config, trying to report an issue yields the following error message:

    403
    Forbidden
    CSRF verification failed. Request aborted.
    Please enable 'Referer' headers for this site and try again.
    

    The expected result is that despite not sending the HTTP Referer header, the issue will be reported. If you are looking for reliable CSRF protection which doesn't break the HTTP specification, you could embed an <input type="hidden"> field into the form instead.

    I assume that other parts of bitbucket won't work either, however I haven't checked in detail. Each time I want to report or comment on an issue, like right now, I have to set network.sendRefererHeader to 2.

  3. Anonymous

    Just created a new account and tried to login from Firefox on MAC OS. Got this 403 error.

    CSRF verification failed. Request aborted. Please enable 'Referer' headers for this site and try again.

    Login from the Chrome browser works fine.

  4. Mark V

    It was interesting reading this, I wasn't aware Django did this. Thanks to stackexchange I now understand the issue better.

    It seems that, if TLS is forced everywhere (I think it is here), then referrer checking is redundant. The protection is only against MITM attacks when posting from http to https. So (if this is confirmed to be true), you could theoretically disable referrer checking at no risk. I'm not aware of any setting for this, so you'd have to hack it yourselves. Up to you if that's worth it...

  5. Sean Farley

    From the Django devs:

    this check is absolutely necessary for the security of Django's CSRF protection. Without it, we can't prevent man-in-the-middle attacks on SSL sites. We made the decision that preventing MITM was a more valuable tradeoff than breaking sites for the small minority of users who block the header in a fashion which does not improve privacy.

  6. Mark V

    I don't care about this anymore, feel free to keep it closed, but I'll post this in case anyone else cares.

    That django-dev quote is in the link I posted; my answer was written with that in mind. The problem seems to apply only to mixed http/https sites, not pure https.

    I'm also curious on what they base the claim that this does not improve privacy; some disagree.

  7. Stephan Hilb

    Another reference. For https MITM is prevented by the protocol, so the django dev must have been refering to the case where a http request is MITMed and a https request is forged. One could argue, that you should be using HSTS for that, but the obvious solution is session dependent nonces (over https) since then the MITM has no way of obtaining a valid nonce for the forged request.

    Blocking all referer headers does improve privacy by revealing less information in a request. By this argument even allowing referers only for same-domain-requests reduces privacy compared to blocking all.

    So that django-dev quote is quite misleading to say the least.

  8. Luke Plant

    Stephan:

    This post answers some of your points: https://groups.google.com/d/msg/django-developers/4ZDBMulE-W0/VpKVBom7aAgJ

    You are right that a session-tied nonce for CSRF would fix this. However, that is a difficult trade-off for a very general purpose framework like Django, because it means that CSRF is tied to sessions. It would also mean that we'd need a different mechanism to cope with CSRF attacks that done before a session is established e.g. login CSRF, or require all POST requests to be accompanied by a pre-existing session, which means creating sessions for anonymous visitors, which has obvious downsides.

  9. Stephan Hilb

    Thank you for the reference, since I'm by no means a security specialist please correct me if I'm wrong. The hypothetical attack described there can avoided by simply clearing the csrf-cookie once when starting a session (using https) which is probably a good idea anyway.

    Regarding login csrf: assuming the user only enters his credentials in the correct https login form, there is no way for an attacker to forge a login request without knowing the credentials in the first place. I don't see the problem here. Regarding anonymous requests: think about this for a moment: if you treat visitors as truly anonymous, there is no meaning in protecting them from csrf. The attacker could just have made that forged request by himself. If there is any value gained for the attacker or any harm done by forging a request as a specific user, then that user can't be anonymous by definition.

  10. Mark V

    It seems to me (also not a security expert) that if you assume:

    1. CSRF cookie is reset upon starting a session.
    2. Sessions are started before log in.
    3. The site forces https using HSTS everywhere.
    4. Post requests without session don't need protection.

    then indeed one should be safe... (Assuming the user doesn't login through a http site or another domain, but in that case there's nothing you can do anyway). The CSRF reset is essential because the first visit to the site (ever) might be insecure, nothing can be done against that (from the server side).

    I can see why Django decided that referrer checking should be ON by default, since those assumptions are far from universally met. Also they don't want to create coupling between sessions and CSRF protection, or start sessions before logging in (which might require cookie warnings in some countries like the Netherlands).

    It does seem that there might be cases where one can legitimately turn it off and stay secure. But to prevent session/csrf coupling, not start sessions for anonymous users, and not change or replace part of django internals, possibly it's not worth it after all...

  11. Mark V

    I think I was mistaken about starting sessions before logging in. If we assume the login request is safe from CSRF (since a CSRF attack is unnecessary if you know someone's login credentials), then the session can be started upon login, I think. So that makes the no-referrer way a little more attractive, but still with some downsides.

  12. Mark V

    Ok seems reasonable, login forms need CSRF protection too.

    I agree that this is not really a bitbucket issue, unless django or a reputable external app implements a good solution.

    It's an interesting topic, but I won't pursue it. Too much trouble for too little benefit if it's possible at all. Good luck to anyone who will!

  13. Log in to comment