Issue #24 resolved
Jan Galkowski
created an issue

Site: http://oceandata.sci.gsfc.nasa.gov

Fetch

http://oceandata.sci.gsfc.nasa.gov

using

wget --dns-timeout=10 --connect-timeout=20 --read-timeout=120 --wait=12 --random-wait --prefer-family=IPv4 --tries=40 --timestamping=on --recursive --level=8 --no-remove-listing  --follow-ftp -nv   --output-file=oceandata-sci-gsfc-nasa-gov-web.log --no-check-certificate http://oceandata.sci.gsfc.nasa.gov

to

oceandata.sci.gsfc.nasa.gov

Comments (17)

  1. Jan Galkowski reporter

    I discovered belatedly that the "open dap" means of getting access to this site is via https://oceandata.sci.gsfc.nasa.gov/opendap/. I am now using: .

    httrack "https://oceandata.sci.gsfc.nasa.gov/opendap/" -O . --mirror --depth=8 --ext-depth=3 --max-rate=100000000 %c500 --sockets=30 \
            --retries=30 --host-control=0 TN 60 --near --robots=0 %s
    
  2. Jan Galkowski reporter

    These live on azi03 where they will remain, per Sakari's instructions.

    The oceandata.sci.gsfc.nasa.gov_o was a partial run against the entire site using wget. The 6oceandata-sci-gsfc-nasa-gov-web` is the result of runninghttrackagainsthttps://oceandata.sci.gsfc.nasa.gov/opendap/` per the comment above, which is the substance of the site.

    These are completed.

    SHA sums attached. Updating per Issue #81.

  3. Log in to comment