I re-ran the example using current master code and got this graph for the error (terr value, plotted as line number vs. column 2):
However looking at the values of the coefficient (or the solution in psi for that matter) reveals that not all is well:
namely of the 5 Gaussian blobs mentioned in the parfile only 2 show up.
Some digging reveals that this is b/c of a not-backwards compatible change to CT_Analytic in d07b160 - (tag: ET_2015_11_v0) Reduced number of terms of gaussian superposition in Analytic.m in the first version of the thorn in the toolkit.
This is in principle ok since it never supported more than 2 Gaussian blobs while in the ET but we likely should
remove the non-functional parameters from param.ccl
regenerate the gallery example with the correct data
We agreed in the ET call today to:
1. remove the non-functional parameters from the thorn
1. regenerate the example data using the current thorn. Will compare that the result are comparable to what was produced by the original code when using the same number of Gaussians.
In the call on 2018-0802 a suggestion was made to mabye re-instantiate the 5 Gaussians. param.ccl right now claims that 20 gaussians are supported, so supporting 5 is not actually fixing the "bug" but instead only making the gallery example work.
I new pull request enabling 20 gaussians is here:
I have quickly checked on runtime and find that run time of the generated routine is:
number of Gaussians
total run time (s)
So basically the CT_Analytic function's runtime is scaling almost linearly with the number of Gaussians. Note that this is a version of the poisson.par file where the max. number of iterations in the ML scheme has been reduced to 2 from 200 so presumably in a real run of the parfile CT_Analytic would not take >50% of the runtime since it only runs once and not once per grid sweep.
I will thus prepare and test a commit allowing up to 5 Gaussians which lets one run the gallery example as shown.
Only relative numbers matter as the absolute numbers would change with processor speed etc.
Commands to create tarball and upload to bitbucket:
I pushed a change to support 5 amp values in git hash 2563eba "CT_Analytic: regenrate code" of ctthorns. This makes the result look like what the gallery shows. I have updated the gallery in git hash 71f3c7a "gallery/ns: update plot and data after re-run" of wwww.
I obtained the above results from running the poisson gallery example, using the “terr_norm_eqn0.asc” and “ct_analytic-ct_testc0.file_*.h5” files respectively. Would you still like me to upload all the results onto bitbucket?
We uploaded the data of the latest test and the corresponding information on the website.
@Bill Gabella and his group agreed to test this for ET_2020_05.
Tested several times and once I used 1 node, 8 mpi tasks, and 1 cpu per task, with 48 GB it ran in 20-25 minutes. Did not use simfactory, so the output tarball reflects that. Removed big files and added some plot files, a python script to make the err.png graphic, a Readme, and left in the 11 Oct 2019 run script and err.png with _20191011 appended to them. The command file might be useful to other HPCs. Included our Slurm script and the Lmod script it uses to load modules.
Used a more manual curl command to add to the Downloads directory, see Roland’s above,
psi2.session, with the directories removed from the data files
[Warning save your long entry before attaching any files…just lost a ton of text when I attached psi2.session. Ack!]
Have been unable to install Visit either on my laptop or on our HPC cluster. Found a docker image for version 2.12.3 (current as of now is 3.1.2) at https://github.com/symerio/visit-docker . Cannot run docker on my laptop because Fedora 32 uses CGRoups V2 and currently runc from docker throws an error. I can run podman as
podman run -it -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix:rw symerio/visit
or with more -v local directories mapped to container directories.
From the HPC cluster, where we use singularity, I use the command
singularity run docker://symerio/visit
from within the Poisson data directory, the one containing the two plotted data files ct_analytic-ct_testc0.file_0.h5 and ct_multilevel-psi.file_0.h5 . Copy the psi2.session file, and run above command, and restore the session from File > Restore Session and select the psi2.session file. This should recreate the images with the sources in white and the slice through the Psi potential with color contours. With the new versions of the Toolkit, running this gallery example, I get an image that is indistinguishable to the eye from the previous one.