Primaries unphysically lost in first lattice element

[Ben Ledroit]

I am using BDSIM for tracking of a Geant4 generated halo distribution in a CSV file. My lattice is originating from MAD-X and converted into GMAD using the pybdsim converter. The CSV input file contains 1e6 primary coordinates and seems to be properly initialized in BDSIM (Primary count and phasespace matches the original Geant4 output). However, the majority of my primaries (~50-99%) is lost in the first lattice element, which is a drift section. Analysis shows that primary losses appear within given element, while no primary hit or energy deposition is recorded. There are also no error messages during simulation. Running the simulation with --verbose_event didn't reveal any insight. I attached my project files (excluding the CSV file).

[Laurie Nevay]

Hello Ben,

Can you attach or email me an example csv file you use for the beam distribution please - preferably just a few particles that produce the problem. I need a set of coordinates that reproduces the problem. A reference beam works fine. I've also tried preparing the model from the tfs file you supplied and the optics agree well.

Thanks, Laurie

[Ben Ledroit]

I attached the CSV file for the issue.

Thanks for the quick reply! Ben

[Laurie Nevay]

Hello Ben,

Thanks for the file. So I find the first set of warnings at event 64931 with seed 1552034545. I've been able to reproduce tracking warnings and stuck particles from that point onwards.

I found a bug in that recreation with a user file didn't recreate correctly - it didn't advance to the right point in the file, so this is fixed.

For the warnings, I believe you can very likely get around them just now with offsetting the start of the beam backwards by a just very small amount. For example:

beam, Z0=-0.1*um;

The warnings I see are due to numerical rounding issues and starting on a boundary. I believe I have a proper fix for this by using a slightly larger margin between volumes in beam pipes transversely. This requires some testing so likely ready in a few days.

Please try this quick fix above and let me know if that helps for now.

I'll let you know about the further fix.

Cheers, Laurie.

[Ben Ledroit]

Hi Laurie,

thank you for your advice! I ran the same scenario including your quick fix for several times and it seems it does something. I did not dig deeper into details, but the count of lost particles now appears to be the same from run to run instead of randomly lost amounts. This does for sure not fix the issue, but i can wait for your updates and i am happy about your engagement!

Thank you and have a nice weekend! Cheers, Ben

[Ben Ledroit]

Hi Laurie,

is there any news about the fix?

Cheers, Ben

[Laurie Nevay]

Hello Ben,

Apologies for the delay. There was a bug with white space reading in the input file so I've fixed this. I have a few remaining checks to do with your model. Some seeds complete fine, but others find an issue only after a very large number of events and then I believe the navigators get in a bad state after that. I'm testing some changes for this and should have a result by the end of the week. I'll update you then either way.

Cheers, Laurie

[Laurie Nevay]

Hello Ben,

I think I have fixed this now. I have made a few changes to make things much more robust to any problems and also have removed the cause of these particular tracking problems. A quick summary of the changes:

• Geant4.10.5 looping limits lowered as they were > the primary energy
• Field now not attached to beam pipe container volume to avoid field in small gaps
• Yoke field more tolerant of very small steps
• Checked reset of geometry navigators at event and run level
• Better catching of event exceptions
• Fixes for user input distribution reading (although this didn't affect you)

These are in a branch called v1.3.2-rc. I'm able to run 200k events from your file with (so far) 3 different seeds without any warnings or problems. I'm currently running the same data with 100 different seeds on our cluster and will have the results tomorrow.

You're quite welcome to try this out with the branch - you can do

git fetch
git checkout v1.3.2-rc

from your copy of the BDSIM source code and update your build. Nominally, I'd make a new clean build of the code and use that one for testing.

I need to wait for our nightly testing to complete but I expect I'll make a release for this tomorrow or the day after.

Please let me know how you get on.

Best, Laurie

[Ben Ledroit]

Hi Laurie,

i have checked out the branch and have been playing around with v1.3.2-rc for the last days. Running the whole 1M still results in stuck particles, so i've split my CSV into 8 separate files (i have 8 available threads for parallel computing), each containing 125k particles. This seems to relax the situation a bit and in the worst case, i had rerun half of it, and in the end the runs were eventually successful. There seem to be some cases, where runs responded differently to different seeds. One 125k-run is very resistant to seed variations and aborts halfway through every time. I am currently getting along with this solution although it's not very satisfying in terms of usability.

Best regards, Ben

[Laurie Nevay]

Hello Ben,

Ok, I need to be able to reproduce your setup. Can you post a file with one event in it. This will be small and will give me the information of all the dependent software so I can recreate your setup. Can you also zip and re-upload your model to be sure I'm using the exact same one as you.

Also, which operating system are you using?

Just to check, with the build of the branch, did you do the install step as well? If you've environmental paths set to an install but haven't re-installed, you'll pick up the old shared libraries.

It's highly surprisingly you see no change in the behaviour. I ran 100 seeds with 200k events from your file without problems at all on SL68 with geant4.10.p02. I also ran 3 seeds of 200k on my mac laptop without problem. I got around 40 single small warnings about a stuck secondary in all of those events but no stuck or looping particles - such small warnings at a very low rate are typical for geant4 and not a problem at all.

Since I have the whole file of coordinates, I can just offset into that file for the particular 125k run if you can post the index of where to start please and roughly which event you see it go off.

This problem is overall quite unique and particular to your model as we've not encountered problems like this in recent generation of billions of events. However, there is no reason it should not work given your input so it'll be good to fix.

I'll hold off the release until we resolve this.

Best, Laurie

[Laurie Nevay]

Hello Ben,

Did you check about the paths for the shared libraries?

Could you upload a BDSIM output root file with 1 event please (from any model).

If the problem persists, can you also upload or email me a zip of the model you're using.

Thanks, Laurie

[Ben Ledroit]

Hi Laurie,

i have to apologize, this was my mistake! I re-checked my installation and noticed that i accidentally checked out the wrong branch. Now everything is up and running. I ran all my simulations to be sure, no problems so far.

I'm sorry for the confusion and i owe you a big thank you! I guess we can consider this issue closed :)

Best regards, Ben

[Laurie Nevay]

Hello Ben,

Great to hear it's resolve for you! Please let me know if you have any further problems. For now, I'll close this issue and make the release.

Best, Laurie

[Laurie Nevay]

Fixed in v1.3.2.

Main changes are

• not attaching field to beam pipe
• increasing transverse margin in beam pipe
• better event exception catching
• fix looping limits for Geant4.10.5 when using low energy primary
• yoke field integrator tolerant of possible discontinuities and small steps
• improved user file loading with better errors