sim setup should create a section for the current machine

[Roland Haas]

When calling sim setup on a machine in the machine data base (eg bluewaters in my case) it it still creates only a [default] section in defs.local.ini which means that the account information and possibly non-default sourcebasedir are not used for bluewaters leading to hard to understand error messages later on.

I would thus like to suggest to change sim setup such that instead (or in addition to) a "[default]" section it creates a section for the detected machine if it already knows this machine.

Keyword:

[Ian Hinder]

Unfortunately, modifying "sim setup" is very awkward. Instead of direct python code which asks questions and sets up a data structure which is eventually written to defs.local.ini, which would be easy to modify, the questions and actions are defined in terms of a "decision tree" via configuration files. This means that adding logic which was not originally intended to the setup process requires the additional step of adding such logic to the generic decision tree language, then implementing the change in the configuration file. "sim setup" starts off in sim-util.py in command_setup, where we have

dt = simdt.DecisionTree("defaults")
dt.setupTree()
dt.begin()

Here, "defaults" refers to the file lib/dt/defaults.ini, which contains the configuration for the "defaults" decision tree. It contains things like

[action:1]
prompt = Determining local machine name
action = %GET_LOCAL_MACHINE_NAME()%
dest = @machine@
printresults = yes

which means that the code should ask for the local machine name, then execute a "macro" called GET_LOCAL_MACHINE_NAME (maybe for the default?), then store the result in a section of the ini file, expanded as @..@. I'm not sure which machine is referred to here.

Anyway, there is a facility for setting machine-specific options; this is the question at the end of the setup process which asks if you want to set explicit machine-specific options. One way to work around the current problem would be to ask users to set up their sourcebasedir here, but this is obviously suboptimal.

Perhaps what we want is to copy all the settings from [default] to [<machine>] after the setup process completes (though presumably not overriding any which had been explicitly set by the user). If we could access the ini file from command_setup, we might be able to try this, but by the time dt.begin() exits, the file has already been written. There's also no way to hook in to the system between writing the [defaults] and [<machine>] sections.

In the spirit of the original code, we probably want a concept of copying everything that was set in one part of the decision tree ([default]) into another part ([<machine>]).

One suggestion would be to replace the entire "sim setup" code with a simpler implementation. Unless I am missing something, it seems like it should be less than a couple of pages of code. We might lose some "advanced" functionality, but I suspect we don't use that functionality anyway.

Is anyone already familiar with this code, and would like to take a stab at fixing the problem in a way which is consistent with the original implementation? Erik?

[Roland Haas]

Before we re-write everything from scratch yet again, would this one: https://bitbucket.org/simfactory/simfactory2/pull-requests/19/rhaas-defaultsect/diff server? I have only tested this once on my own workstation (and it took ~30 minutes to write for 9 lines of code, although that is only possible because Ian already found out what to do and described the code :-) ).

[Roland Haas]

Now that the release is done, would someone want to review this (Ian, Erik are candidates for reviewers).

[Roland Haas]

Making a [<machine>] section is actually dangerous on the clusters. Simfactory's setup command (both the interactive and the silent variant) create settings for sourcebasedir (probably ok since they set it to the parent of the current directory) and also basedir, and the latter one is likely wrong since the default that simfactory uses is $HOME/simulations which is almost certainly wrong on clusters.

So basically we have the situation where half of the information is what we want and half is not.

The fix is likely to try and see if one can make the defaults be the values from an existing machine.ini file if one find oneself on a known machine. The tricky part is probably to try and see if one can get a mdb table for the current machine without an existing defs.local.ini.

[Steven R. Brandt]

Roland, what should we do with this ticket? What are you currently doing on BW?

[Roland Haas]

I think the code proposed (if there is any) should be changed to either pre-populate from the existing machine entry or only set "safe" values in the machine entry (eg user name, allocation, maybe sourcebasedir).

Right now on BW I manually edit defs.local.ini after simfactory's setup code is done.