This is a revival of Clustermatic 5. We have already got a newer bproc working; to avoid confusion with what is still out there and in use, we are calling it xproc, since it derives features from xcpu and bproc.
We've removed all kernel code; this system will work on, e.g., OSX and other systems in future.
It's fast! The speed is a credit to Erik Hendrix's excellent coding skills.
It's now recursive! The bpmaster can serve as a slave node controller; bpslave is deprecated and going away soon.
It will scale better than bproc! We're targeting one million nodes, which is one reason we dropped the original bpoc model; it does not scale well past about 4000 nodes.
It's more secure! We now create a private name space (CLONE_NEWNS) and mount a private directory (/xproc). Your files live there and are visible to no one else. To create files that are shared, use /tmp.
You can also (via the -f switch) specify files and directories to bring over on demand. This has proven incredibly powerful. For example, what if we want to trace a binary with the strace command. In bproc there was no really good way to do this. But now we can do this:
bpsh -f /usr/bin/date strace usr/bin/date
So we move both strace and the date command, then trace it. Very, very nice.
Things we are working on adding
1. Still don't have tree spawn. That's ok, we're going to do it a different way.
You can join the mailing list.