`make tests-clean` does not clean `make tests`

[Dan Bonachea]

On dirac and develop @ 67be530e with a clean build tree:

{pcp-d-5} make tests
[...]
Compiling test-hello_upcxx-smp                                         SUCCESS
Compiling test-rpc_ff_ring-smp                                         SUCCESS
Compiling test-uts_ranks-smp                                           SUCCESS
Compiling test-global_ptr-smp                                          SUCCESS
Compiling test-barrier-smp                                             SUCCESS
Compiling test-uts_hybrid-smp                                          SUCCESS
Compiling test-rput_thread-smp                                         SUCCESS
Compiling test-rput-smp                                                SUCCESS
Compiling test-rpc_barrier-smp                                         SUCCESS
Compiling test-local_team-smp                                          SUCCESS
Compiling test-hello_upcxx-ofi                                         SUCCESS
Compiling test-dist_object-smp                                         SUCCESS
Compiling test-vis-smp                                                 SUCCESS
Compiling test-collectives-smp                                         SUCCESS
Compiling test-view-smp                                                SUCCESS
Compiling test-global_ptr-ofi                                          SUCCESS
Compiling test-barrier-ofi                                             SUCCESS
Compiling test-rpc_ff_ring-ofi                                         SUCCESS
Compiling test-local_team-ofi                                          SUCCESS
Compiling test-rpc_barrier-ofi                                         SUCCESS
Compiling test-rput-ofi                                                SUCCESS
Compiling test-vis_stress-smp                                          SUCCESS
Compiling test-uts_ranks-ofi                                           SUCCESS
Compiling test-dist_object-ofi                                         SUCCESS
Compiling test-hello_upcxx-udp                                         SUCCESS
Compiling test-collectives-ofi                                         SUCCESS
Compiling test-uts_hybrid-ofi                                          SUCCESS
Compiling test-rput_thread-ofi                                         SUCCESS
Compiling test-atomics-smp                                             SUCCESS
Compiling test-global_ptr-udp                                          SUCCESS
Compiling test-vis-ofi                                                 SUCCESS
Compiling test-rpc_ff_ring-udp                                         SUCCESS
Compiling test-barrier-udp                                             SUCCESS
Compiling test-local_team-udp                                          SUCCESS
Compiling test-view-ofi                                                SUCCESS
Compiling test-uts_ranks-udp                                           SUCCESS
Compiling test-atomics-ofi                                             SUCCESS
Compiling test-rpc_barrier-udp                                         SUCCESS
Compiling test-rput-udp                                                SUCCESS
Compiling test-collectives-udp                                         SUCCESS
Compiling test-dist_object-udp                                         SUCCESS
Compiling test-vis_stress-ofi                                          SUCCESS
Compiling test-rput_thread-udp                                         SUCCESS
Compiling test-uts_hybrid-udp                                          SUCCESS
Compiling test-hello_upcxx-ucx                                         SUCCESS
Compiling test-vis-udp                                                 SUCCESS
Compiling test-global_ptr-ucx                                          SUCCESS
Compiling test-rpc_ff_ring-ucx                                         SUCCESS
Compiling test-barrier-ucx                                             SUCCESS
Compiling test-atomics-udp                                             SUCCESS
Compiling test-local_team-ucx                                          SUCCESS
Compiling test-view-udp                                                SUCCESS
Compiling test-rput-ucx                                                SUCCESS
Compiling test-rpc_barrier-ucx                                         SUCCESS
Compiling test-vis_stress-udp                                          SUCCESS
Compiling test-dist_object-ucx                                         SUCCESS
Compiling test-uts_ranks-ucx                                           SUCCESS
Compiling test-hello_upcxx-ibv                                         SUCCESS
Compiling test-collectives-ucx                                         SUCCESS
Compiling test-uts_hybrid-ucx                                          SUCCESS
Compiling test-rput_thread-ucx                                         SUCCESS
Compiling test-global_ptr-ibv                                          SUCCESS
Compiling test-barrier-ibv                                             SUCCESS
Compiling test-rpc_ff_ring-ibv                                         SUCCESS
Compiling test-vis-ucx                                                 SUCCESS
Compiling test-local_team-ibv                                          SUCCESS
Compiling test-uts_ranks-ibv                                           SUCCESS
Compiling test-view-ucx                                                SUCCESS
Compiling test-atomics-ucx                                             SUCCESS
Compiling test-rpc_barrier-ibv                                         SUCCESS
Compiling test-rput-ibv                                                SUCCESS
Compiling test-dist_object-ibv                                         SUCCESS
Compiling test-collectives-ibv                                         SUCCESS
Compiling test-vis_stress-ucx                                          SUCCESS
Compiling test-rput_thread-ibv                                         SUCCESS
Compiling test-uts_hybrid-ibv                                          SUCCESS
Compiling test-vis-ibv                                                 SUCCESS
Compiling test-atomics-ibv                                             SUCCESS
Compiling test-view-ibv                                                SUCCESS
Compiling test-vis_stress-ibv                                          SUCCESS

[...]
{pcp-d-5 ~/UPC/bxx-gcc112} ls
total 1200828
    4 Makefile               15952 test-dist_object-ofi*  12736 test-rpc_barrier-smp*  12888 test-uts_hybrid-udp*
    0 bin/                   14372 test-dist_object-smp*  14572 test-rpc_barrier-ucx*  14744 test-uts_ranks-ibv*
    4 bld/                   16212 test-dist_object-ucx*  13732 test-rpc_barrier-udp*  13196 test-uts_ranks-ofi*
    4 config.log             15368 test-dist_object-udp*  14816 test-rpc_ff_ring-ibv*  11628 test-uts_ranks-smp*
    4 inst-develop/          15128 test-global_ptr-ibv*   13264 test-rpc_ff_ring-ofi*  13456 test-uts_ranks-ucx*
 1896 job.out                13580 test-global_ptr-ofi*   11696 test-rpc_ff_ring-smp*  12608 test-uts_ranks-udp*
20348 test-atomics-ibv*      12008 test-global_ptr-smp*   13524 test-rpc_ff_ring-ucx*  18428 test-view-ibv*
18796 test-atomics-ofi*      13836 test-global_ptr-ucx*   12680 test-rpc_ff_ring-udp*  17036 test-view-ofi*
17192 test-atomics-smp*      12996 test-global_ptr-udp*   16048 test-rput-ibv*         15452 test-view-smp*
19052 test-atomics-ucx*      14380 test-hello_upcxx-ibv*  14492 test-rput-ofi*         17336 test-view-ucx*
18212 test-atomics-udp*      12836 test-hello_upcxx-ofi*  12924 test-rput-smp*         16452 test-view-udp*
15508 test-barrier-ibv*      11264 test-hello_upcxx-smp*  14752 test-rput-ucx*         17996 test-vis-ibv*
13960 test-barrier-ofi*      13092 test-hello_upcxx-ucx*  13912 test-rput-udp*         16444 test-vis-ofi*
12380 test-barrier-smp*      12252 test-hello_upcxx-udp*  15528 test-rput_thread-ibv*  14872 test-vis-smp*
14216 test-barrier-ucx*      16036 test-local_team-ibv*   14136 test-rput_thread-ofi*  16700 test-vis-ucx*
13376 test-barrier-udp*      14492 test-local_team-ofi*   12556 test-rput_thread-smp*  15860 test-vis-udp*
17792 test-collectives-ibv*  12912 test-local_team-smp*   14436 test-rput_thread-ucx*  20320 test-vis_stress-ibv*
16240 test-collectives-ofi*  14748 test-local_team-ucx*   13556 test-rput_thread-udp*  18772 test-vis_stress-ofi*
14668 test-collectives-smp*  13908 test-local_team-udp*   14868 test-uts_hybrid-ibv*   17196 test-vis_stress-smp*
16496 test-collectives-ucx*      0 test-results/          13476 test-uts_hybrid-ofi*   19028 test-vis_stress-ucx*
15652 test-collectives-udp*  15860 test-rpc_barrier-ibv*  11900 test-uts_hybrid-smp*   18188 test-vis_stress-udp*
17496 test-dist_object-ibv*  14316 test-rpc_barrier-ofi*  13780 test-uts_hybrid-ucx*
{pcp-d-5} make tests-clean
Removing tests for smp.
Removing tests for ofi.
Removing tests for udp.
Removing tests for mpi.
Removing tests for ucx.
Removing tests for ibv.
{pcp-d-5} ls
total 139036
    4 Makefile        1896 job.out                14436 test-rput_thread-ucx*  13780 test-uts_hybrid-ucx*
    0 bin/               0 test-results/          13556 test-rput_thread-udp*  12888 test-uts_hybrid-udp*
    4 bld/           15528 test-rput_thread-ibv*  14868 test-uts_hybrid-ibv*
    4 config.log     14136 test-rput_thread-ofi*  13476 test-uts_hybrid-ofi*
    4 inst-develop/  12556 test-rput_thread-smp*  11900 test-uts_hybrid-smp*

As shown above, make tests is building test-rput_thread and test-uts_hybrid but make tests-clean is not cleaning these tests. They are also not cleaned by dev-tests-clean, but that's expected since they use a different naming pattern.

There's something else odd going on with the test set: These two tests (test-rput_thread, test-uts_hybrid) are BUILT by make check, but not RUN by make check or make run-tests.

[Paul Hargrove]

Those two tests appear in $(test_exclude_seq). So, my hypothesis is that some logic error is leading to the exclusion of these from both the clean and run targets, while they are included in the tests target.

A quick glance at tests.mak turned up something which looked suspicious to me. I don't have time to test, but am 75% sure the following change (swapping test_sources_par and test_sources_seq on 2 lines) is what is needed:

--- a/bld/tests.mak
+++ b/bld/tests.mak
@@ -28,8 +28,8 @@ test_sources_par = \
        test/uts/uts_hybrid.cpp \
        test/view.cpp

-test_progs_seq = $(patsubst %.cpp,%,$(patsubst %.sh,%,$(filter-out $(tests_filter_out_seq),$(test_sources_par))))
-test_progs_par = $(patsubst %.cpp,%,$(patsubst %.sh,%,$(filter-out $(tests_filter_out_par),$(test_sources_seq))))
+test_progs_seq = $(patsubst %.cpp,%,$(patsubst %.sh,%,$(filter-out $(tests_filter_out_seq),$(test_sources_seq))))
+test_progs_par = $(patsubst %.cpp,%,$(patsubst %.sh,%,$(filter-out $(tests_filter_out_par),$(test_sources_par))))

 ###
 # Section 2: Maintainer/development tests

[Dan Bonachea]

glance at tests.mak turned up something which looked suspicious to me.

I agree this looks suspicious.

Note that test_sources_par includes all three of: test/rput_thread.cpp test/uts/uts_hybrid.cpp test/view.cpp, but test/view does NOT appear to be affected, unless my testing is somehow flawed..

[Paul Hargrove]

Yes, I also noticed the discrepancy with respect to view. I'd say that accounts for about 10% of my 25% uncertainly that the 2-line change is the right (or only) fix.

[Paul Hargrove]

OK, I think I get it now. Because view is not present either in $(test_exclude_seq) or $(test_exclude_par) it was not filtered-out when generating $(test_progs_par). Meanwhile the other two are par-only, and the "suspicious bit" therefore removed them from $(test_progs_par) when erroneously filtering-out on $(test_exclude_seq) instead of $(test_exclude_par).

In my testing on my laptop (so smp conduit by default), the 2-line patch above resolved the problem. I see 16 tests compiled, 16 tests run, and 16 tests cleaned (matching the 16 enumerated in "Section 1" of bld/tests/mak).

@Dan Bonachea if you can confirm it works for you too, then I'll IPR that change with the addition of a commit message referencing this issue.

[Dan Bonachea]

Patch confirmed to fix the problem I saw

[Paul Hargrove]

Fix incorrect treatment of two tests

This commit resolves issue #542 in which is was observed that make tests and make check would compile test-rput_thread and test-uts_hybrid, but make check did not run them and make tests-clean did not remove them.

The problem was an inadvertent reversal of seq-vs-par in the logic to generate filtered lists of tests.

→ <<cset 42e2b6db4d42>>