1. Azimuth Backup
  2. Azimuth Backup Project
  3. azimuth-inventory
  4. Issues
Issue #23 new

NASA PDS: The Planetary Data System

Sakari Maaranen
created an issue

2016-12-18 Hetzner storage box in Germany

wget --dns-timeout=10 --connect-timeout=20 --read-timeout=120 \
 --wait=12 --random-wait --follow-ftp --progress=dot:mega \
 --prefer-family=IPv4 --tries=40 --timestamping=on --recursive --level=inf \
 --no-remove-listing --no-check-certificate \
 --output-file=pds.nasa.gov.log \
 -H -Dpds.nasa.gov https://pds.nasa.gov

Comments (4)

  1. Sakari Maaranen reporter

    Also this one seems to have stuck into some dynamically created pages, traversing through infinite links. I am assuming the valuable content has already been downloaded, but cannot be sure without someone familiar with the data inspecting it.

    Interrupting the download.

  2. Sakari Maaranen reporter

    The following directories and files were created as a part of this job.

    0 Dec 18 23:17 atmos.pds.nasa.gov
    0 Dec 20 12:47 geo.pds.nasa.gov
    0 Dec 21 03:27 img.pds.nasa.gov
    0 Dec 21 00:34 mgmt.pds.nasa.gov
    0 Dec 19 03:05 naif.pds.nasa.gov
    0 Dec 21 02:25 pds.nasa.gov
    139824623 Jan  9 10:20 pds.nasa.gov.log
    0 Jan  5 13:38 ppi.pds.nasa.gov
    0 Dec 18 23:29 rings.pds.nasa.gov
    0 Dec 21 07:31 sbn.pds.nasa.gov
    

    All the above directories and files should now be moved together, away from datarefuge. I have a disk usage command running and will report size as soon as it completes.

  3. Sakari Maaranen reporter
    [sam@azi03 datarefuge]$ nice ionice du -s -c -b *
    364     atmos.pds.nasa.gov
    1949771752      geo.pds.nasa.gov
    167670838       img.pds.nasa.gov
    257551594       mgmt.pds.nasa.gov
    50579609        naif.pds.nasa.gov
    466522276       pds.nasa.gov
    139824623       pds.nasa.gov.log
    2946156955381   ppi.pds.nasa.gov
    16165   rings.pds.nasa.gov
    621398926403    sbn.pds.nasa.gov
    
  4. Log in to comment