No transfer log, didn't generate one and don't want to restart the mirroring process (it took nearly seven days to complete).
Suggestion for comparing completeness and accuracy of mirrors:
Download the SHA512SUMS file I've attached to this ticket.
Copy into the root directory (in this case, ftp.cdc.noaa.gov/) of your mirror.
Execute the command sha512sum -c SHA512SUMS, which will compare the SHA-512 hashes of your files against those in the file. Any which don't match will be flagged as incorrect. This means that your files don't match mine. This doesn't mean that mine are right and yours are wrong, it means that they don't match. We'll need to figure out what to do about that later.
Turns out, the wget missed a bunch of the NetCDF files, for some reason. I am re-downloading them to my workstation using pure FTP which seems to be going well, and then will upload to the server, and recalculation the SHA512 sums.
Latest comparison of Bruce's upload effort with mine. There are a few files missing, but they don't look important. (Mostly systemish ".listing"-like files.) I can go back and try to get the individual NetCDFs again, and see if I have better luck doing an SFTP.
Both my /media/jan-one/ and /home/jan/local_data/ are getting kinda full, especially /media/jan-one/. Any possibility of tossing more storage there? I'd go to another box, but there are wgets running heading there, and don't want to interrupt. I could of course, after taking some time to copy things off.
Download completed. Calculating SHA sums. Bruce and I each did this, and that was continued to try to ascertain how much variability there might be among different people downloading the same thing.
At some point this should be revisited and we should get a unique set of files. I don't mind as these reanalysis data are critically important. The main datafiles are in subdirectories:
drwx------ 2 jan jan 4096 Jan 704:33 www.esrl.noaa.gov
drwxrwxr-x 2 jan jan 135168 Dec 2714:48 surface_gauss
drwxrwxr-x 2 jan jan 36864 Dec 2714:47 surface
drwxrwxr-x 2 jan jan 36864 Dec 2714:46 other_gauss
drwxrwxr-x 2 jan jan 20480 Dec 2714:46 pressure
drwxrwxr-x 2 jan jan 4096 Dec 2714:45 tropopause
drwxrwxr-x 2 jan jan 20480 Dec 2714:43 spectral
and there are replicas in the subdirectories Datasets and old.
Download completed, but there may be redundant information here. But we do not need access to the original to ascertain that. Accordingly, postponing until later. Can use the present time to download more.