Problem checking out with TLSv1.1 (BB-1389)

Issue #2552 resolved
created an issue

I cannot hg clone from bitbucket from NetBSD-5.99.47/amd64. NetBSD contains a rather recent openssl snapshot, and another user informed me of the following:

=== begin quote === I started seeing the same error after bitbucket made HTTPS mandatory in December.

I don't know what SSL implementation bitbucket is using, but as far as I can tell it's sending garbage in response to TLSv1.1 requests. They probably haven't noticed because the problem won't affect anyone using older (pre-TLSv1.1) versions of OpenSSL on the client side, but it shows up with either the OpenSSL in -current or with more recent OpenSSL snapshots.

I think bitbucket needs to fix the problem on the server side, but in the meantime I've worked around it by patching Python's ssl module to restrict Python and hg to TLSv1.0 or earlier. With the patch below, everything works fine for me.

Equivalently (more or less), I expect that your command-line test will work if you add the -tls1 option, viz.: openssl s_client -tls1 -connect

The -debug option to s_client is also useful for seeing how the server's responses differ with and without -tls1.

diff python/Modules/_ssl.c python/Modules/_ssl.c --- python/Modules/_ssl.c
+++ python/Modules/_ssl.c
@@ -365,7 +365,7 @@ newPySSLObject(PySocketSockObject Sock, char key_file, char *cert_file, }

 /* ssl compatibility */
  • SSL_CTX_set_options(self->ctx, SSL_OP_ALL);
  • SSL_CTX_set_options(self->ctx, SSL_OP_ALL|SSL_OP_NO_TLSv1_1);

    verification_mode = SSL_VERIFY_NONE;
    if (certreq == PY_SSL_CERT_OPTIONAL)
    === end quote ===

(The patch won't apply since it will have whitespace issues from cut'n'paste.) I have tried connecting with -tls1 and it does indeed fix the problem.

For comparison:

openssl s_client -connect

CONNECTED(00000006) 140187580655852:error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol:/archive/cvs/src/crypto/external/bsd/openssl/dist/ssl/s23_clnt.c:705:

no peer certificate available

No client certificate CA names sent

SSL handshake has read 7 bytes and written 145 bytes

New, (NONE), Cipher is (NONE) Secure Renegotiation IS NOT supported Compression: NONE Expansion: NONE

openssl s_client -tls1 -connect CONNECTED(00000006) depth=0 C = NL, O =, OU = GT16385137, OU = See (c)09, OU = Domain Control Validated - QuickSSL(R), CN = verify error:num=20:unable to get local issuer certificate verify return:1 depth=0 C = NL, O =, OU = GT16385137, OU = See (c)09, OU = Domain Control Validated - QuickSSL(R), CN = verify error:num=27:certificate not trusted verify return:1 depth=0 C = NL, O =, OU = GT16385137, OU = See (c)09, OU = Domain Control Validated - QuickSSL(R), CN = verify error:num=21:unable to verify the first certificate verify return:1

Certificate chain 0 s:/C=NL/ (c)09/OU=Domain Control Validated - QuickSSL(R)/ i:/C=US/O=Equifax/OU=Equifax Secure Certificate Authority

Server certificate -----BEGIN CERTIFICATE----- MIIDLDCCApWgAwIBAgIDDZCWMA0GCSqGSIb3DQEBBQUAME4xCzAJBgNVBAYTAlVT MRAwDgYDVQQKEwdFcXVpZmF4MS0wKwYDVQQLEyRFcXVpZmF4IFNlY3VyZSBDZXJ0 aWZpY2F0ZSBBdXRob3JpdHkwHhcNMDkxMDI1MTE0MTA4WhcNMTIwMTI2MjAyNDA1 WjCBtjELMAkGA1UEBhMCTkwxFjAUBgNVBAoTDWJpdGJ1Y2tldC5vcmcxEzARBgNV BAsTCkdUMTYzODUxMzcxMTAvBgNVBAsTKFNlZSB3d3cuZ2VvdHJ1c3QuY29tL3Jl c291cmNlcy9jcHMgKGMpMDkxLzAtBgNVBAsTJkRvbWFpbiBDb250cm9sIFZhbGlk YXRlZCAtIFF1aWNrU1NMKFIpMRYwFAYDVQQDEw1iaXRidWNrZXQub3JnMIGfMA0G CSqGSIb3DQEBAQUAA4GNADCBiQKBgQDnAS4Nvgi9BnlG56hRO5Km5kBztLQZoIRH NzrRqxMyh7DkT3O4xP62D5MNgRf5HHExCHZtdoBWMIdPgyI3tkpZQtv32/PvhIwT +a8MLF7o19H3jc4T/I4hxa5lYY1H7nWfo/ulh9LOujaaYid7tkHdlxp4XgJfRW+W PAoOmRvO8QIDAQABo4GuMIGrMA4GA1UdDwEB/wQEAwIE8DAdBgNVHQ4EFgQUN2Hq zv4GppKRpHm7ZYv26HnMzfowOgYDVR0fBDMwMTAvoC2gK4YpaHR0cDovL2NybC5n ZW90cnVzdC5jb20vY3Jscy9zZWN1cmVjYS5jcmwwHwYDVR0jBBgwFoAUSOZo+SvS spXXR9gjIBBPM5iQn9QwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMCMA0G CSqGSIb3DQEBBQUAA4GBADqv1xIEp+haWoiWIC5JiM1NmkEB4zUsj6JS4As3KV/o Vh+G5XP2jefYNT5epMLhckhnJHF+11tI7XHqIPzgF94sjCWW7sWKsfOIsW0Q97GN Za0Or9iSqn1O90EB030B6M3DmR8uTisoiMZ+DUI8/bUyU9M38OLI5GiiwvpJyzLt -----END CERTIFICATE----- subject=/C=NL/ (c)09/OU=Domain Control Validated - QuickSSL(R)/ issuer=/C=US/O=Equifax/OU=Equifax Secure Certificate Authority --- No client certificate CA names sent --- SSL handshake has read 1534 bytes and written 407 bytes --- New, TLSv1/SSLv3, Cipher is DHE-RSA-AES256-SHA Server public key is 1024 bit Secure Renegotiation IS supported Compression: NONE Expansion: NONE SSL-Session: Protocol : TLSv1 Cipher : DHE-RSA-AES256-SHA Session-ID: B6F24C90CC3BA4FA84F7A829AFCCECF53124C4365FE87773B0FB4E7439859C8D Session-ID-ctx: Master-Key: 2A08F25A5DBD4511002CE8C538A71AF81DCA6226FD3AB93BE8E4FAEC594A63F5F8357458F462E6E9CD60D5046486B907 Key-Arg : None PSK identity: None PSK identity hint: None TLS session ticket: 0000 - 1a d6 28 14 70 18 8b 06-0f c7 2a 37 e9 39 9d 43 ..(.p.....*7.9.C 0010 - 0c 65 4d e9 87 b5 81 07-32 3c 8d ce 78 c4 8b b0 .eM.....2<..x... 0020 - be 74 8c 33 82 77 5f 4b-d7 e8 70 3e 6e bd 42 c4 .t.3.w_K..p>n.B. 0030 - 63 a7 99 1a e6 3d 22 98-a1 c0 bb 2e 1b 4f 43 a6 c....="......OC. 0040 - 05 fb 58 88 5f a5 6b af-54 c8 e5 d1 a5 db ea c0 ..X._.k.T....... 0050 - d9 d8 1d d2 69 c9 94 13-a2 d5 23 e0 16 aa 6c f8 ....i.....#...l. 0060 - eb ff 99 a4 8e dc 62 d4-0b ff 81 7c 2b cf 3c 0c ......b....|+.<. 0070 - e1 a2 de d2 8b eb 46 8d-a8 f0 43 71 22 2f 28 ef ......F...Cq"/(. 0080 - 4a 75 7d 7b 49 3c 97 84-f0 b1 0d 98 e5 fa 3a 73 Ju}{I<........:s 0090 - e3 0d d1 26 10 98 c3 f8-09 6d be e3 49 6b a2 97 ...&.....m..Ik..

Start Time: 1299334101
Timeout   : 7200 (sec)
Verify return code: 21 (unable to verify the first certificate)

Please fix this!

Comments (6)

  1. Mads Kiilerich

    What exactly is the problem you see? Can you give an example?

    (And please make sure you use some kind of markup so it is readable. You might want to re-post some of the content above.)

  2. Brodie Rao

    I'm able to reproduce the issue with openssl-SNAP-20110309. It looks like the issue only occurs when talking to HAProxy; connecting to one of the front ends directly works fine.

    I'm not sure if it's an issue in OpenSSL, an issue with our HAProxy config, or a bug in HAProxy. I'll have to do some more investigation.

  3. Brodie Rao

    The issue was with our HAProxy config.

    We allow SSH connections over port 443 for users who can't connect via port 22 on their network. The check we have in place to differentiate SSL connections and SSH connections only recognized SSL 2-3.1 connections. I've expanded that check to SSL 2-3.3. You should be able to connect using TLS 1.1 now.

  4. Log in to comment