Parallel Random number generators

[Scott Baden]

What is the recommended procedure for generating random numbers? We need a random number generator that will work will many millions of ranks.

[Dan Bonachea]

Any serial C++ PRNG should work (including rand() and those provided by C++), although you may want to seed it with upcxx::rank_me() if you want to ensure each rank has a different pseudorandom sequence.

[Scott Baden]

But if you always use rank_me you'll always start with the same random number sequence. Adding in the time of day (captured from just 1 rank) can avoid this problem. Sprng is specially designed for MPI, I believe that with just 1 seed it arranges for each rank to traverse a mutually disjoint path. Qustion is 2^48 enough? With 2^28 threads, that's just 2^20 random numbers.

[john bachan]

Scott, the determinacy you describe is a desired behavior. Seeding with timeofday is probably a bad idea. Your app should ask for a random seed at program startup (in the environment) and use that in addition to the rank number to seed the local PRNG. C++ provides std::seed_seq for combining the information from multiple integers into the seed of a PRNG.

unsigned user_seed = 0xdeadbeef;

std::seed_seq seed{(unsigned)upcxx::rank_me(), user_seed};

std::default_random_engine prng(seed);

If you want a fast and lightweight PRNG that ensures ranks each start from very distant places in the deterministic sequence, consider xoroshiro https://en.wikipedia.org/wiki/Xoroshiro128%2B. That one has a period of 2^128, and the ability to skip over 2^64 elements at a time (for spacing the ranks). Otherwise the Mersenne twister seeded appropriately is probably fine since the period is so enormous the chance that any two ranks will land near each other is small. I would just do the latter since its in the standard #include <random> header.

[Scott Baden]

But that begs the question.. how do you know that your so called "random seed" at startup time is in fact random?

[john bachan]

Those values don't have to look "random", they only need to be different. Its the job of the seeding algorithm to sufficiently mix incoming bits before initializing the PRNG. For instance, I could seed a PRNG by taking the MD5 hash of consecutive integers. MD5 can make any pattern look effectively random since its a provably difficult function to invert. std::seed_seq does something similar.