Issue #40 duplicate
Borislav Iordanov
created an issue

https://www1.ncdc.noaa.gov/pub/ for a few days now. First I almost filled the drive on my own server and I had to stop it. Then kicked off a download on Sakari's server, but the connection is so slow I ended up getting a separate machine with a 4 terabyte volume on AWS. So now I'm trying it there. On the climatemirror.org only a part of this is claimed. And on our bitbucket I noticed issues #5, #10 and #17 which are about the same organization (NOAA NCEI ).

I will report with comments on progress.

Comments (8)

  1. Anonymous

    Basing all of my info off of ftp.ncdc.noaa.gov/pub which should be the same stuff as on www1.ncdc.noaa.gov but over FTP.

    /pub/has/ftpusage.txt shows:

    Filesystem       1K-blocks        Used   Available Use% Mounted on
    /dev/simfs     76235669504 12081838080 64153831424  16% /ftp/pub/has
    

    Which indicated ~12TB used on /pub/has.

    In addition, running a recursive size summation (in FileZilla) of /pub/data grows to >1TB pretty quickly (The asos-* dirs alone are ~1.5TB).

  2. Sakari Albert Maaranen

    @John Baez has limited our budget to longer term four server that is about 40 (or 44) terabytes total. Let's prioritize according to the email I sent earlier. Technically there is no reason why we couldn't do this, but it would hog most of our capacity, and with our approach we would need to split it to three servers.

    Let climate experts prioritize.

  3. Log in to comment