Issue #48 resolved
Jan Galkowski
created an issue

http://www.eia.gov

This is importance for both governance, whether at federal or state level, and as a mark of progress. Because the Clean Power Plan demands states reduce their carbon emissions, EIA also took on a complementary task of tracking greenhouse gas emissions, principally CO2, from generation and use sectors.

These data might be at risk. They are important. (I use them all the time.) I do not know the size of the dataset. If we run into space issues on the server, I'll copy down sections of interest to me, and keep them locally.

Comments (18)

  1. Jan Galkowski reporter

    So, I think we got our first ``block'': https://www.eia.gov is issuing a "503" when I try to wget from the server, but if I do it from home, it proceeds fine. It is possible that the wait=0 value in Scott's wg script is causing attention to the process. Dunno. Restarting from home. This is the fourth restart, because of the problems with server #1 and /media/jan-one and /home/jan/local_data yesterday.

  2. Sakari Maaranen

    I hope it goes well this time. If you get HTTP 503 errors, you should inspect possible Retry-After header in the response. Both servers are now running without issues. We should prefer local storage. The issue on azi01 was caused by a CIFS mount of Hetzner Storage Box. Local storage is independent of network mounts.

  3. Jan Galkowski reporter

    FTP portion of this site is uploaded to

    /home/maxwell/local_data/jan_spillover/ftp.eia.gov
    

    on server #1.

    Checksums are uploaded here, as are tar packing and unpacking listings.

  4. Log in to comment