1. Azimuth Backup
  2. Azimuth Backup Project
  3. azimuth-inventory
  4. Issues
Issue #10 open

NOAA's National Centers for Environmental Information (NCEI), FTP site

Jan Galkowski
created an issue

FTP data from

#!shell


ftp://eclipse.ncdc.noaa.gov/

mostly NCDF files, to

#!shell

azi01:/home/jan/local_data/eclipse.ncdc.noaa.gov-pub-ftp

Comments (9)

  1. Jan Galkowski reporter

    Going to restart this on the server. It's probably not going to make it on my box.

    Using

    wget --dns-timeout=10 --connect-timeout=20 --read-timeout=120 --wait=12 --random-wait --prefer-family=IPv4 --tries=40 --timestamping=on --recursive --level=8 --no-remove-listing --follow-ftp -nv --output-file=ftp-oar-noaa-gov-ftp.log --no-check-certificate ftp://eclipse.ncdc.noaa.gov

  2. Jan Galkowski reporter

    When restarted, forgot to do a cd .., so will need to do an mv when this is finished to organize the directories. Current layout is:

    #!shell
    
    total 20
    drwxrwxr-x 3 jan jan 4096 Dec 20 20:46 .
    drwxr-xr-x 5 jan jan 4096 Dec 20 20:42 ..
    drwxrwxr-x 3 jan jan 4096 Dec 20 20:46 eclipse.ncdc.noaa.gov
    -rw-rw-r-- 1 jan jan 1036 Dec 20 20:47 ftp-oar-noaa-gov-ftp.log
    -rw-rw-r-- 1 jan jan  234 Dec 20 20:45 ftp-oar-noaa-gov-pub-ftp.log
    
    ./eclipse.ncdc.noaa.gov:
    total 16
    drwxrwxr-x 3 jan jan 4096 Dec 20 20:46 .
    drwxrwxr-x 3 jan jan 4096 Dec 20 20:46 ..
    drwxrwxr-x 3 jan jan 4096 Dec 20 20:46 cdr
    -rw-rw-r-- 1 jan jan  367 Dec 20 20:46 .listing
    
    ./eclipse.ncdc.noaa.gov/cdr:
    total 16
    drwxrwxr-x 3 jan jan 4096 Dec 20 20:46 .
    drwxrwxr-x 3 jan jan 4096 Dec 20 20:46 ..
    drwxrwxr-x 2 jan jan 4096 Dec 20 20:47 amsu-ch7
    -rw-rw-r-- 1 jan jan 2676 Dec 20 20:46 .listing
    
    ./eclipse.ncdc.noaa.gov/cdr/amsu-ch7:
    total 24
    drwxrwxr-x 2 jan jan 4096 Dec 20 20:47 .
    drwxrwxr-x 3 jan jan 4096 Dec 20 20:46 ..
    -rw-rw-r-- 1 jan jan  883 Jan 16  2015 AMSU-CH7-RO-CAL-BT-CDR_V01R00_ANOM_latest_Aggregation.ncml
    -rw-rw-r-- 1 jan jan  360 Jan 16  2015 AMSU-CH7-RO-CAL-BT-CDR_V01R00_CLIM_latest.ncml
    -rw-rw-r-- 1 jan jan  880 Jan 16  2015 AMSU-CH7-RO-CAL-BT-CDR_V01R00_MON_latest_Aggregation.ncml
    -rw-rw-r-- 1 jan jan  543 Dec 20 20:46 .listing
    
  3. Log in to comment