UPCXX_CODEMODE must be set

[Scott Baden]

I'm getting this warning message when I attempt to build any target of scott/rb1d/upcxx on kotten

UPCXX_CODEMODE must be set to one of: O3 (default), debug

The output is generated by this part of the Makefile. I added this at some point, but I don't see it in any of the PG examples. if I remove it, all is fine. Just wondering if I should keep it around as it seems useful.

ifeq ($(wildcard $(UPCXX_INSTALL)/bin/upcxx-meta),) $(error Please set UPCXX_INSTALL=/path/to/upcxx/install) endif

Shoudl I remove it? Scott

[Steven Hofmeyr]

This is exactly how it is done in the example/prog-guide/Makefile. It causes no problems for me. Not sure what your problem is.

[Scott Baden]

On Kotten do

module load upcxx/previous/clang-5.0.0

Then go to the PG directory, typing "Make"

If you don't have access to kotten, please ask Eric to set you up with an account

[Paul Hargrove]

@sbbaden This is a "feature" of the environment module on Kotten.
To force you to think about debug vs O3, the module file sets it to unset to trigger this warning.

When using Kotten, please set UPCXX_CODEMODE to O3 or debug as the message says.

[Paul Hargrove]

This is the intended behavior of the environment on Kotten.

[Scott Baden]

Where is this documented?

[Dan Bonachea]

Where is this documented?

INSTALL.md:

* `UPCXX_CODEMODE=[O3|debug]`: `O3` is for highly compiler-optimized
  code. `debug` produces unoptimized code, includes extra error checking
  assertions, and is annotated with the symbol tables needed by debuggers. The
  default value is always `O3`.

I encouraged Paul to disable the default on our local installs, because we want to encourage our users (especially our local developers) to make an explicit, informed decision about whether they should be running in debug mode or not - each has its proper place depending on what you are doing, and neither is right all the time.

[Scott Baden]

A few lines of explanation, outlining what you and Paul had in mind, would go a long way to help users understand why they are getting this obscure message. I looked at INSTALL.md, but it never occurred to me that I should be looking at the section "Backends," so I skipped over it.

This is not really an installation issue. I encountered via the module command, which is hardly an installation the user takes part in actively

[Paul Hargrove]

To repeat:

This is a "feature" of the environment module on Kotten.

Only users on Kotten and Dirac will ever see this behavior unless they also chose to intentionally set UPCXX_CODEMODE to an invalid value as I have done to defeat the default O3 behavior.

It has nothing to do with INSTALL.md or any other file in the upcxx repo.
So, I don't think it is Steve's problem to address in the upcxx documentation.

Dan provided the requested "few lines of explanation" regarding our modules in a previous comment.
If the motivation is still not clear, let us know.
If this is a problem, let Dan and me know as the sysadmins, and we can discuss adjusting how the modules work.

[Scott Baden]

What you said earlier: "To force you to think about debug vs O3, the module file sets it to unset to trigger this warning."

contradicts what is said in the INSTALL.md file

"The default value is always O3."

That isn't the case.

There's no way for anyone to know that the installation on Kotten and Dirac is different from that stated in INSTALL.md. The message tells you to make a choice, but again that isn't what is said in INSTALL.md.

What is also a needs a quick explanation (if I'm interpreting the doucmentation correctly) is that the the other 2 variables that affect what backend upcxx-meta uses are set to the defrault value, presumably when they are not defined. That behavior is different from what governs UPCXX_CODEMODE's behavior. That one has to be defined.

I'm also not sure why this behavior is different on kotten and dirac than from other installations, but that's another discussion when we have more time.

[Paul Hargrove]

Short version:

The default is still O3, and is still used when UPCXX_CODEMODE is not set.

Therefore, if using (t)csh, you need only to unsetenv UPCXX_CODEMODE to restore the default.
For users of bash or dash: unset UPCXX_CODEMODE.

However, Dan and I would encourage you to instead set it to O3 or to debug so you've made a conscious choice to compile for performance (O3) or for correctness (debug), instead of depending on the default.

Additionally, one is free to ignore the module files entirely and set only UPCXX_INSTALL as you might on other systems (such as the Mac's we share).

Full version:

The behavior on Kotten and Dirac is different because Dan and I have chosen in the environment module file (which is not part of UPC++) to set UPCXX_CODEMODE=unset. This triggers an error message from UPC++ because it is not a valid value. We thought that the resulting message
UPCXX_CODEMODE must be set to one of: O3 (default), debug
was clear and that the user would simply set the variable to one of the permitted values and move on.

What we want, very much, is to encourage the users on our systems to stop and consider if they are currently focused on performance (the O3 default) or on correctness (the debug value). We felt that if we had not chosen to "defeat" the default in this manner, then we anticipated users would remain largely unaware that they were developing code without the aide of debugging checks. We cannot begin to count the number of times we've seen bug reports against Berkeley UPC which could have been identified as invalid user code it they had only turned on debugging (which is a simple matter of passing -g to the UPC compiler in that case).

contradicts what is said in the INSTALL.md file
"The default value is always O3."
That isn't the case.

The default in UPC++ is still O3, and there is no contradiction with the documentation.
The default value is the one used if the variable has not been set.
However, the admins for this site have elected to set this environment variable.

If you remove the setting of UPCXX_CODEMODE from the environment then the default is used.
However, instead of removing it, we recommend setting it to O3 or debug depending on your needs at the time.

I'm also not sure why this behavior is different on kotten and dirac than from other installations,

This has been answered multiple times already:
Dan and I chose to force the user to pick instead of using the default behavior.
This was accomplished by setting UPCXX_CODEMODE=unset in the environment module.

The environment module is external to UPC++ and one can choose not to use it, or can return to the as-installed behavior by removing this setting from ones environment.

[Scott Baden]

That is clearer. Here is what was confusing

"The default value is the one used if the variable has not been set. However, the admins for this site have elected to set this environment variable."

"Only users on Kotten and Dirac will ever see this behavior unless they also chose to intentionally set UPCXX_CODEMODE to an invalid value as I have done to defeat the default O3 behavior."

Users aren't intentionally setting the value. You are. It's a lot harder for teh reader to understand that you are setting it to "unset" which will cause upcxx-meta to complain, but not to the default of O3.

Why not just say what you did

"On Dirac and kotten we disabled the default setting of the UPCXX_CODEMODE environment variable so that the user is required to specify which mode they desire before they may compile or link upcxx code. They will be sent a warning any time they attempt to compile or link code until they've set this variable; the upcxx-meta command, which sets up the include and link command line arguments, requires a valid setting in order to operate correctly"

I can appreciate this with years of experience wiht users, you know what's best for them :-)

But in many years of teach students about how to use software installation in large classes, if I don’t give them some background, I tend to get many many more questions. This is like what you are saying about making sure that users know that there is an available debugging capability. So knowing the motivation is actually part of the learning process.

But users also get floored by complicated explanations. It may be easier to just say what you did

[Paul Hargrove]

@sbbaden wrote:

Why not just say what you did

That is exactly what I intended in my first posting to this issue:

This is a "feature" of the environment module on Kotten.
To force you to think about debug vs O3, the module file sets it to unset to trigger this warning.

When using Kotten, please set UPCXX_CODEMODE to O3 or debug as the message says.

I avoided as many unnecessary (IMO) details of the "how" and "why" as possible.
I described (briefly) what had been done that was non-standard, and how to proceed.

@sbbaden wrote:

But in many years of teach students about how to use software installation in large classes, if I don’t give them some background, I tend to get many many more questions.

In my 28 years (since Fall 1989) of supporting users, I find that too much background can be distracting or overwhelming if unsolicited. Any more than the basics and the message can be lost.

Many users (not all) just want the steps needed to move forward toward their goal with little or no concern for why they are the right steps. I had initially assumed from your question (in which you asked if you should remove some logic from Makefile) that your goal was simply to move forward with building on Kotten, and provided an answer tailored to that goal.

One can always answer the additional questions later if they arise. I do acknowledge that answering the questions later would not work as well for a large class as in a one-on-one support scenario such as this. Additionally, in a class setting the true goal is to learn and so information beyond what is required to complete any given assignment is highly appropriate.