New TLS certificate for einsteintoolkit.org is self-signed and gives browser warnings/errors

Create issue
Issue #2249 resolved
Ian Hinder created an issue

See https://www.sslshopper.com/ssl-checker.html?hostname=einsteintoolkit.org.

einsteintoolkit.org resolves to 130.39.21.27

Server Type: Apache/2.4.25 (Debian)

The certificate will expire in 364 days.

The hostname (einsteintoolkit.org) is correctly listed in the certificate.

The certificate is self-signed. Users will receive a warning when accessing this site unless the certificate is manually added as a trusted certificate to their web browser. You can fix this error by buying a trusted SSL certificate
    Common name: einsteintoolkit.org
Organization: Louisiana State University Org. Unit: Center for Computation and Technology
Location: Baton Rouge, Louisiana, US
Valid from April 24, 2019 to April 23, 2020
Serial Number: 10477637556223891899 (0x91680bd8300459bb)
Signature Algorithm: sha256WithRSAEncryption
Issuer: einsteintoolkit.org

Safari throws up a big warning and doesn't let you visit the site unless you confirm a security exception with your login password.

Didn't we have an automated tool to check these things and report them?

Comments (12)

  1. Steven R. Brandt

    We had a failure of the server where the machines were hosted, and the docker images were restarted on different servers but the certificates were lost. We are in the process of getting new ones.

  2. Steven R. Brandt

    I have a tool which monitors websites and certs (it's a 50 line python script). If anyone is interested, I can share it.

  3. Ian Hinder reporter

    Yes please! I have been interested recently in monitoring solutions, and have played with Icinga (a nagios fork). Unfortunately, its certificate checking is restricted to checking expiry dates, not certificate chains. Maybe your tool could be hooked up to it instead? Would it have caught this problem? I found an SSL/TLS checker (https://github.com/drwetter/testssl.sh), but it is very full-featured, takes a long time to run (maybe minutes), and doesn't return a simple exit code for "good" or "bad", probably because most sites will fail at least some of its checks (e.g. using old ciphers). Probably a simple grep on the output would catch the things we need, and maybe taking a few minutes doesn't matter if we only run it infrequently, but if your solution is much simpler, then I like it!

    Was there no backup of the Einstein Toolkit server, in which the certificates could be found?

  4. Ian Hinder reporter

    For reference, the following docker command will run a test on einsteintoolkit.org and catch the current problem. It takes 26 s to run, which is better than the 3 min taken by the full suite of tests.

    mkdir logs
    time sudo docker run -ti -v $PWD/logs:/tmp/logs drwetter/testssl.sh --server-defaults --color 0 --quiet --severity HIGH -oC /tmp/logs -oJ /tmp/logs -oL /tmp/logs -oH /tmp/logs einsteintoolkit.org
    

    The standard output is human readable:

     Start 2019-04-26 12:36:50        -->> 130.39.21.27:443 (einsteintoolkit.org) <<--
    
     rDNS (130.39.21.27):    einsteintoolkit.org.
     Service detected:       HTTP
    
    
     Testing server defaults (Server Hello) 
    
     TLS extensions (standard)    "server name/#0" "renegotiation info/#65281"
                                  "EC point formats/#11" "session ticket/#35"
                                  "heartbeat/#15"
                                  "application layer protocol negotiation/#16"
     Session Ticket RFC 5077 hint 300 seconds, session tickets keys seems to be rotated < daily
     SSL Session ID support       yes
     Session Resumption           Tickets: yes, ID: yes
     TLS clock skew               Random values, no fingerprinting possible 
     Signature Algorithm          SHA256 with RSA
     Server key size              RSA 2048 bits
     Server key usage             --
     Server extended key usage    --
     Serial / Fingerprints        91680BD8300459BB / SHA1 A06674AE0582BD65DFA9E5A43F28F32B59A29FF4
                                  SHA256 80412717A02D2DE5FA41D6D26DE378397278F2983BE33AFEA9C6E6B75F1E8FCE
     Common Name (CN)             einsteintoolkit.org
     subjectAltName (SAN)         missing (NOT ok) -- Browsers are complaining
     Issuer                       self-signed (NOT ok)
     Trust (hostname)             via CN only -- Browsers are complaining (same w/o SNI)
     Chain of trust               NOT ok (self signed)
     EV cert (experimental)       no 
     "eTLS" (visibility info)     not present
     Certificate Validity (UTC)   363 >= 60 days (2019-04-24 21:33 --> 2020-04-23 21:33)
     # of certificates provided   1
     Certificate Revocation List  --
     OCSP URI                     --
                                  NOT ok -- neither CRL nor OCSP URI provided
     OCSP stapling                not offered
     OCSP must staple extension   --
     DNS CAA RR (experimental)    not offered
     Certificate Transparency     --
    
    
     Done 2019-04-26 12:37:14 [  25s] -->> 130.39.21.27:443 (einsteintoolkit.org) <<--
    

    And the CSV file can be parsed by awk to give:

    $ awk -vFPAT='([^,]*)|("[^"]+")' -vOFS="\t" '{print $1,$2,$4,$5}' logs/einsteintoolkit.org_p443-*.csv | column -t -s $'\t'
    
    "id"                   "fqdn/ip"                           "severity"  "finding"
    "service"              "einsteintoolkit.org/130.39.21.27"  "INFO"      "HTTP"
    "cert_subjectAltName"  "einsteintoolkit.org/130.39.21.27"  "HIGH"      "No SAN, browsers are complaining"
    "cert_caIssuers"       "einsteintoolkit.org/130.39.21.27"  "CRITICAL"  "selfsigned"
    "cert_trust"           "einsteintoolkit.org/130.39.21.27"  "HIGH"      "via CN only (same w/o SNI)"
    "cert_chain_of_trust"  "einsteintoolkit.org/130.39.21.27"  "CRITICAL"  "failed (self signed)."
    "cert_revocation"      "einsteintoolkit.org/130.39.21.27"  "HIGH"      "Neither CRL nor OCSP URI provided"
    
  5. Roland Haas

    If all we want is a simple checker then Frank's code (behind) https://www.cct.lsu.edu/~knarf/cgi-bin/monitor.cgi will do as well as the python code that Steve mentioned. Note that this would still need to run somewhere so if that host running it goes down we are no better off than now. Note that for the current instance of self-signed certificates this was not something that happened without people being aware. As far as I know Steve was aware of the certificates being incorrect from the beginning but given that the servers were brought back online at 9pm local time, getting back the original certs was delayed.

    There was unfortunately no explicit email to the mailing list (once it was back up) stating "we are sorry, einsteintoolkit.org and docs.einsteintoolkit.org are down" and "einsteintoolkit.org is back but you will have to accept the temporary certificate".

    Speaking of which: having everything back up and running before Tuesday next week would be very nice.

    I am not sure if one can restrict what to watch to particular components in bb but I can change my filter to let "website" components through and only throw out the others (I need the filter b/c bitbucket makes me auto-watch all tickets to which I have responded).

  6. Roland Haas

    As a side note: this certificate chain missing is not caught by: https://www.cct.lsu.edu/~knarf/cgi-bin/monitor.cgi (and the “not all” is correct, my browser is happy with the website right now).

    The raw

    openssl s_client -showcerts -connect einsteintoolkit.org:443 </dev/nul
    

    does report “Verify return code: 21 (unable to verify the first certificate)” though.

  7. Ian Hinder reporter

    Thanks! The direct parent certificate, AddTrust External CA Root, seems to be using a SHA1 cert, known to be weak since 2012, as reported by ssl-checker. I suppose this is probably beyond our control, but it’s a bit disappointing not to get “full marks”! Apparently there are lots of questions and confusion on the net about intermediate certs being flagged, and how in some cases, it’s ok for some of the certs (maybe only the root?) to use SHA1. Safari, at least, is perfectly happy with it.

    One of the certificates is signed with a SHA1 signature. We recommend that you reissue or replace this certificate with one that uses a SHA-2 signature. Contact your SSL provider about how to do this. Read more about the SHA-1 deprecation here.

  8. Log in to comment