Clarifying pointers to shared arrays, and multi-D shared arrays

[Former user]

Originally reported on Google Code with ID 3

Evidence for pointer-to-local interpretation
--------------------------------------------

The following parts of the UPC spec and C99 spec indicate that a pointer to an
array-of-shared is NOT a pointer-to-shared:

UPC 1.2 3.2.3 - "shared array: an array with elements that have shared qualified
type"

UPC 1.2 3.4 - "pointer-to-shared: a pointer whose referenced type is
shared-qualified"

UPC 1.2 6.5.2.1.4 - "In an array declaration, the type qualifier applies to the
elements."

C99 6.7.3.8 - "If the specification of an array type includes any type
qualifiers, the element type is so qualified, not the array type."

Because the pointer's target is an array and the array itself cannot be shared 
qualified, such a pointer must be a pointer-to-local.  This conclusion is very
non-intuitive to most users.


Evidence for pointer-to-shared interpretation
---------------------------------------------

Other parts of the spec appear to (very indirectly) imply that such a pointer is
indeed a pointer-to-shared:

UPC 1.2 6.5.2.1.5 - "For any shared array, a, upc_phaseof( &a ) is zero."

UPC 1.2 7.2.3.2 - "size_t upc_phaseof( shared void *ptr );"

C99 6.5.3.2.3 - "if the operand is the result of a [] operator, neither the &
operator nor the unary * that is implied by the [] is evaluated and the result
is as if the & operator were removed and the [] operator were changed to a +
operator."

If the call to upc_phaseof( &a ) is legal, then &a must be a pointer-to-shared.  
However, &a is distinct from &a[0] and simply "a" given the quoted section of 
C99, which establishes the equivalence between &a[0] and a.  &a is a pointer to
an array.


Recommendation
--------------

The spec should be clarified.  The pointer-to-local interpretation is the most 
consistent with the rest of the spec, but users seem to expect a 
pointer-to-shared.  If it IS a pointer-to-shared, then there are other issues to
resolve with regards to pointer-to-shared arithmetic on such a pointer.

Reported by johnson.troy.a on 2012-03-13 15:49:16

[Former user]

``` The following code illustrates one interesting aspect of this ambiguity:

typedef int T1; shared T1* p1;

typedef int T2[2]; shared T2* p2;

Under the pointer-to-local interpretation, p1 is a pointer-to-shared but p2 is a pointer-to-local. Therefore, for a programmer, "shared <TYPE>*" is an ambiguous pointer-to-something until they track down the typedef and check if it is an array.

The pointer-to-shared interpretation treats both p1 and p2 as a pointer-to-shared, but then what does it mean to do pointer arithmetic on p2? Consider these examples:

shared [10] int A[THREADS][10];

typedef int T[10]; shared [10] T* p;

p = &A[0]; p = p + 1; Does p == &A[1] afterward? (*p)[3] = 1; Does this set A[1][3]?

-------------------

shared [5] int A[20][10];

typedef int T[10]; shared [5] T* p;

p = &A[0]; p = p + 1; (*p)[9] = 1; Does this set A[1][9]? ```

Reported by `johnson.troy.a` on 2012-03-18 17:53:22

[Former user]

``` Does anyone have any comments on this issue? This is one issue that I'd like to see clarified sooner than later because different compilers currently resolve this issue differently. ```

Reported by `johnson.troy.a` on 2012-05-29 15:24:41

[Former user]

``` Attached, is a test program which attempts to codify the examples shown above. Please review and correct the example if it doesn't accurately demonstrate the issue.

When compiled with GUPC, it fails as follows.

$ upc -fupc-threads-2 -fupc-debug -W -Werror -Wpedantic -O2 -o array_test array_test.upc array_test.upc: In function ‘main’: array_test.upc:28:3: error: UPC operator upc_blocksizeof applied to a non-shared type assert (upc_blocksizeof &B[1] == 0); ^

When compiled with BUPC, it fails as follows.

$ $ bld/upcr-packed-dbg/dbg/upcc -network=smp -T2 -o array_test array_test.upc

upcc: error during UPC-to-C translation (sgiupc stage): array_test.upc: In function `main': array_test.upc:26: warning: assignment from incompatible pointer type array_test.upc:28: upc_blocksizeof applied to a nonshared type array_test.upc:29: warning: comparison of distinct pointer types lacks a cast array_test.upc:32: warning: assignment from incompatible pointer type

Here is the text of the program.

1 #include <upc.h> 2 #include <assert.h> 3 4 shared [5] int A[5*THREADS]; 5 shared [10] int B[THREADS][10]; 6 shared [5] int C[20][10]; 7 8 typedef int TB[10]; 9 shared [10] TB* pb; 10 11 typedef int TC[10]; 12 shared [5] TC* pc; 13 14 int main() 15 { 16 size_t bs_a, bs_a_0; 17 size_t phase_a, phase_a_0; 18 bs_a = upc_blocksizeof A; 19 bs_a_0 = upc_blocksizeof A[0]; 20 assert (bs_a == 5); 21 assert (bs_a == bs_a_0); 22 phase_a = upc_phaseof (A); 23 phase_a_0 = upc_phaseof (&A[0]); 24 assert (phase_a == 0); 25 assert (phase_a == phase_a_0); 26 pb = &B[0]; 27 pb = pb + 1; Does pb == &B[1] afterward? 28 assert (upc_blocksizeof &B[1] == 0); 29 assert (pb == &B[1]); 30 (*pb)[3] = 1; Does this set B[1][3]? 31 assert (B[1][3] == 1); 32 pc = &C[0]; 33 pc = pc + 1; 34 (*pc)[9] = 1; 35 assert (C[1][9] == 1); 36 return 0; 37 }

They agree on the interpretation of line 28 as an error. (Note: the introduction of parentheses after the upc_blocksizeof operator will not change the meaning or the result, due to operator precedence.)

```

Reported by `gary.funck` on 2012-05-29 16:27:25

<hr>

• *Attachment: [array_test.upc](https://storage.googleapis.com/google-code-attachments/upc-specification/issue-3/comment-3/array_test.upc)*

[Former user]

``` So first, for completeness, Cray emits this message (which is very oddly worded -- I'll have to look into that):

CC-1172 craycc: WARNING File = gary.c, Line = 28 Argument of upc_blocksizeof is a pointer to a shared type (not shared type itself).

assert (upc_blocksizeof &B[1] == 0); ^

When I run the executable:

a.out: gary.c:28: main: Assertion `upc_blocksizeof( &B[1] ) == 0' failed.

--------

You chose upc_blocksizeof for this example code. If you switch upc_blocksizeof to upc_phaseof or upc_threadof, the error issued by BUPC goes away. Does GUPC produce a consistent error for all three? ```

Reported by `johnson.troy.a` on 2012-05-29 16:46:09

[Former user]

``` On this virtual Monday, it seems that I need another cup of coffee. Attached an updated example with two corrections:

- check for correct block size (ie, 10) of B.

- require indirection to check for the blocksize B[1]: assert (upc_blocksizeof (*&B[1]) == 10);

- add a check for the block size of *pb: assert (upc_blocksizeof (*pb) == 10);

GUPC compiles this without complaint, but fails at runtime:

$ upc -fupc-threads-2 -fupc-debug -W -Werror -Wpedantic -O2 -o array_test array_test.upc $ array_test array_test: at array_test.upc:30, UPC error: Invalid conversion of shared address to local pointer; thread does not have affinity to shared address

BUPC issues some warnings:

$ bld/upcr-packed-dbg/dbg/upcc -network=smp -T2 -o array_test array_test.upc array_test.upc: In function `main': array_test.upc:26: warning: assignment from incompatible pointer type array_test.upc:30: warning: comparison of distinct pointer types lacks a cast array_test.upc:33: warning: assignment from incompatible pointer type

and fails at runtime with an assertion.

$ bld/upcr-packed-dbg/dbg/upcrun array_test array_test: array_test.upc:29: main: Assertion `upc_blocksizeof (*pb) == 10' failed. array_test: array_test.upc:29: main: Assertion `upc_blocksizeof (*pb) == 10' failed.

I'm still not sure if I have got this example right. Please review/comment/correct as needed.

```

Reported by `gary.funck` on 2012-05-29 16:47:27

<hr>

• *Attachment: [array_test.upc](https://storage.googleapis.com/google-code-attachments/upc-specification/issue-3/comment-5/array_test.upc)*

[Former user]

``` For completeness, here is the new example.

1 #include <upc.h> 2 #include <assert.h> 3 4 shared [5] int A[5*THREADS]; 5 shared [10] int B[THREADS][10]; 6 shared [5] int C[20][10]; 7 8 typedef int TB[10]; 9 shared [10] TB* pb; 10 11 typedef int TC[10]; 12 shared [5] TC* pc; 13 14 int main() 15 { 16 size_t bs_a, bs_a_0; 17 size_t phase_a, phase_a_0; 18 bs_a = upc_blocksizeof A; 19 bs_a_0 = upc_blocksizeof A[0]; 20 assert (bs_a == 5); 21 assert (bs_a == bs_a_0); 22 phase_a = upc_phaseof (A); 23 phase_a_0 = upc_phaseof (&A[0]); 24 assert (phase_a == 0); 25 assert (phase_a == phase_a_0); 26 pb = &B[0]; 27 pb = pb + 1; Does pb == &B[1] afterward? 28 assert (upc_blocksizeof (*&B[1]) == 10); 29 assert (upc_blocksizeof (*pb) == 10); 30 assert (pb == &B[1]); 31 (*pb)[3] = 1; Does this set B[1][3]? 32 assert (B[1][3] == 1); 33 pc = &C[0]; 34 pc = pc + 1; 35 (*pc)[9] = 1; 36 assert (C[1][9] == 1); 37 return 0; 38 }

```

Reported by `gary.funck` on 2012-05-29 16:50:04

[Former user]

``` Troy asks:

You chose upc_blocksizeof for this example code. If you switch upc_blocksizeof to upc_phaseof or upc_threadof, the error issued by BUPC goes away. Does GUPC produce a consistent error for all three?

I used upc_blocksizeof, because it should issue a compile-time error message if passed an argument that is not shared qualified. upc_phaseof() and upc_threadof() are implemented as functions, and will evaluate their argument (a pointer-to-shared) at runtime. Both cases may be useful to illustrate this issue under discussion.

Please make corrections/additions to illustrate your points and submit as an attachment.

As far as consistency, I haven't looked in detail at the warnings issued by BUPC to see if GUPC should be following suit, or if in fact the ambiguity of interpretation is leading to the warnings. It is unexpected that GUPC may be silently casting a shared pointer to local, and that issue requires further investigation. But first, I think it is worthwhile to have a complete test case for which there is consensus that it illustrates the point of this issue, and that it should be a valid UPC test program under a reasonable interpretation of the language specification.

```

Reported by `gary.funck` on 2012-05-29 17:03:31

[Former user]

``` I understand. Your latest test case looks okay and compiles with Cray UPC without any messages. It produces this error at runtime:

a.out: gary.c:30: main: Assertion `pb == &B[1]' failed.

Cray UPC currently follows the pointer-to-local interpretation in Comment #0, but we don't diagnose improper usage under that interpretation very well. For example, to return to a simpler compile-only test:

1. include <upc.h>

shared int x[THREADS];

int foo( void ) { return upc_phaseof( &x ); }

Under the pointer-to-local interpretation, the &x there should be either x or &x[0] to be legal. Cray is prepared to make this an error in the next release if the community consensus is the pointer-to-local interpretation in Comment #0. Cray is also prepared to make this code actually work as expected under the pointer-to-shared interpretation if that's what everyone decides upon. We don't believe that either interpretation is clearly expressed by the UPC 1.2 spec. ```

Reported by `johnson.troy.a` on 2012-05-29 17:22:00

[Former user]

``` The latest test case compiles and runs without any messages using HP UPC.

Gary wrote: I used upc_blocksizeof, because it should issue a compile-time error message if passed an argument that is not shared qualified.

The specification does not require an error message. Should it?

The language in the specification just says that the operator applies only to shared types; it is silent on how the compiler should treat operands that do not fit that criterion. HP UPC does not currently issue an error message, but I now think it should do so, and think requiring an error message in this case is prudent. ```

Reported by `brian.wibecan` on 2012-05-29 17:52:43

[Former user]

``` Two additional data points: sgiupc and MTU's mupcc

Use of Gary's first test case w/ the SGI compiler yields:

$ sgiupc -c -LANG:upc_threads=2 test.upc "test.upc", line 28: warning: argument of upc_blocksizeof is a pointer to a shared type (not shared type itself) assert (upc_blocksizeof &B[1] == 0); ^

And MTU's mupcc: $ mupcc -fthreads 2 test.c "test.c", line 28: warning: argument of upc_blocksizeof is a pointer to a shared type (not shared type itself) assert (upc_blocksizeof &B[1] == 0); ^

Both of which are in agreement w/ GUPC, BUPC and Cray. The message is both cases is identical to the one from the Cray compiler.

Unfortunately, in addition to that warning, the sgiupc version I have access to also ICEs on both examples. So, I can't tell what the runtime result on the second test case might be.

When given the second testcase mupcc compiles and runs without any complaints.

Aside: I agree with Brian (in comment #9) that the warning that most of the compilers have issued about the argument to upc_blocksizeof does NOT appear to be required by the spec. I also agree that a compiler *should* issue this warning, but DISAGREE that the spec should require it. ```

Reported by `[e-mail]` on 2012-05-29 18:28:48

[Former user]

``` Brian wrote:

The specification does not require an error message. Should it? The language in the specification just says that the operator applies only to shared types; it is silent on how the compiler should treat operands that do not fit that criterion. HP UPC does not currently issue an error message, but I now think it should do so, and think requiring an error message in this case is prudent.

I will file a separate issue on this, if we determine that an error is not currently required, or if the behavior is currently unspecified, and there is some (possibly non-unanimous) support for defining this usage as an error.

Howver, my reading of the specification is that this usage is defined as an error.

The C99 specification in Section 4 ("Conformance") states:

4. Conformance 1 In this International Standard, ‘‘shall’’ is to be interpreted as a requirement on an implementation or on a program; conversely, ‘‘shall not’’ is to be interpreted as a prohibition. 2 If a ‘‘shall’’ or ‘‘shall not’’ requirement that appears outside of a constraint is violated, the behavior is undefined. Undefined behavior is otherwise indicated in this International Standard by the words ‘‘undefined behavior’’ or by the omission of any explicit definition of behavior. There is no difference in emphasis among these three; they all describe ‘‘behavior that is undefined’’.

The UPC 1.2 Specification lists the following _constraint_ under section 6.4.1.3 ("The upc_blocksizeof operator").

1 The upc blocksizeof operator shall apply only to shared-qualified expressions or shared-qualified types. All constraints on the sizeof operator [ISO/IEC00 Sec. 6.5.3.4] also apply to this operator.

Given the use of the term "shall" within a constraint, does the specification not already require that the argument of upc_blocksizeof must be a shared qualified expression. Thus, any other usage is an error?

In the C99 specification, 'sizeof' lists this constraint:

1 The sizeof operator shall not be applied to an expression that has function type or an incomplete type, to the parenthesized name of such a type, or to an expression that designates a bit-field member.

Is the use of 'shall not' some how a stronger restriction, and therefore a prescription that these excluded cases are definitely erroneous uses, but the "shall" language of the UPC speicification is somehow more permissive?

If that is the case (and interpretation), I would like to see the UPC specification amended to use the "shall not" form of description, and will submit a separate tracking issue to discuss that proposal. Otherwise, it seems to me that this erroneous use case is already covered.

```

Reported by `gary.funck` on 2012-05-29 23:35:23

[Former user]

``` Gary wrote:

Given the use of the term "shall" within a constraint, does the specification not already require that the argument of upc_blocksizeof must be a shared qualified expression. Thus, any other usage is an error?

It's clear that any other usage is relying on undefined behavior, which is incorrect usage. The question is whether the compiler is required to issue an error message. Theoretically, undefined behavior can be handled however the implementation wants. In this case, no, the compiler has not up to now been required to diagnose and report this situation. If it were, the behavior would change from "undefined" to "defined: produces a compile-time error message". ```

Reported by `brian.wibecan` on 2012-05-30 19:29:41

[Former user]

``` Brian wrote:

It's clear that any other usage is relying on undefined behavior, which is incorrect usage. The question is whether the compiler is required to issue an error message.

Brian/all, is there any place in the UPC specification that states explicitly that a translation error must be issued for a particular use case? I see only footnote 7 referenced from section 6.3.3 ("UPC MAX BLOCK SIZE"), which states:

7 e.g. shared [UPC MAX BLOCK SIZE+1] char x[UPC MAX BLOCK SIZE+1] and shared [*] char x[(UPC MAX BLOCK SIZE+1)*THREADS] are translation errors

When I read the constraint "shall apply only to shared-qualified expressions", it seems to me that passing a non-shared qualified expression value to upc_blocksizeof should be considered a translation error.

```

Reported by `gary.funck` on 2012-05-30 21:29:33

[Former user]

``` Back to the main line of this issue, I tried Troy's example in comment #8 on both GUPC and BUPC.

1 #include <upc.h> 2 3 shared int x[THREADS]; 4 5 int main() 6 { 7 return upc_phaseof( &x ); 8 }

There were no compiler complaints and the program returned 0 on all threads. It is possible that both compilers are handling this case of & applied to a shared array specially, or some other part of the shared GCC infra-structure is stepping in and interpreting &x to be equivalent to &x[0]. Probably, we would need to dump the internal tree representation to see how this is being handled.

Note that regarding the longer example that I submitted in comment #5 and the varying compiler responses to same, there may be other implementation issues that need to be understood and resolved besides this issue under discussion.

```

Reported by `gary.funck` on 2012-05-30 23:16:04

[Former user]

``` As an additional data point, consider the following "C" program.

$ cat -n array_qual.c 1 2 const int x[100]; 3 4 int 5 addr_eq_x (const int *p) 6 { 7 return p == x; 8 } 9 10 int main() 11 { 12 return !addr_eq_x (&x); 13 }

When compiled with GCC.

$ gcc -o array_qual array_qual.c array_qual.c: In function ‘main’: array_qual.c:12:5: warning: passing argument 1 of ‘addr_eq_x’ from incompatible pointer type [enabled by default] return !addr_eq_x (&x); ^ array_qual.c:5:1: note: expected ‘const int *’ but argument is of type ‘const int (*)[100]’ addr_eq_x (const int *p) ^

However, if the example is changed to: return !addr_eq_x (x); all is well.

Does that help to shed any further light on the issue?

```

Reported by `gary.funck` on 2012-05-30 23:27:16

[Former user]

``` In comment #15, I indicated that just passing 'x', was OK. Perhaps that was a cockpit error. Here is the actual output.

$ cat -n array_qual.c 1 2 const int x[100]; 3 4 int 5 addr_eq_x (void *p) 6 { 7 return p == x; 8 } 9 10 int main() 11 { 12 return !addr_eq_x (x); 13 }

Above the only change is to pass 'x' instead of '&x'.

$ gcc -std=c99 -W -Wpedantic -o array_qual array_qual.c array_qual.c: In function ‘main’: array_qual.c:12:5: warning: passing argument 1 of ‘addr_eq_x’ discards ‘const’ qualifier from pointer target type [enabled by default] return !addr_eq_x (x); ^ array_qual.c:5:1: note: expected ‘void *’ but argument is of type ‘const int *’ addr_eq_x (void *p) ^

```

Reported by `gary.funck` on 2012-05-30 23:39:46

[Former user]

``` Regarding Comment #15, yes, it is instructive to experiment with how C handles pointers to arrays and it shows the exact problem. &x in your example has type const int (*)[100], which in English is a pointer to an array of 100 const ints. If you were to ask the question, "does this pointer type point to something that is const qualified?", the answer is No according to C99 6.7.3.8: "If the specification of an array type includes any type qualifiers, the element type is so qualified, not the array type." The type is a pointer to an array and that array is not const qualified.

Now, replace "const" with "shared" and one concludes that a pointer to an array of shared-qualified objects is not a pointer-to-shared. Therefore, it must be a plain old C pointer because there is no third alternative. The real question is, is that what we want? Is there some utility to going contrary to C99 and deciding that it is a pointer-to-shared? ```

Reported by `johnson.troy.a` on 2012-05-31 14:51:49

[Former user]

``` It seems ill-advised to go contrary to the C99 specification, because UPC is generally defined as an extension to "C", not a revision of "C".

Perhaps one way out is for UPC to disallow application of & to an operand expression that has type that is a shared array?

Are there other contexts where this "shared array" and "shared array element" dichotomy come into play? Is that list sufficiently small that it can be compactly addressed in the UPC specification?

```

Reported by `gary.funck` on 2012-05-31 15:08:17

[Former user]

``` I think it is the main situation in which the dichotomy arises and I'm not aware of others. I've seen people try to code essentially this:

shared [M*N] int xyz[THREADS][M][N];

typedef shared [M*N] int (*ptype)[M][N]; ptype p;

...

p = &xyz[t];

The intention here is to point p at the part of array xyz that is local to thread t, i.e.

Thread 0 Thread 1 Thread t p N N | N ------------ ------------ ... ------------ ...

M | | | | | |

------------ ------------ ... ------------ ...

such that p[0 <= i < M][0 <= j < N] reference data on Thread t.

Given what I just wrote in Comment #17, p is a pointer to an array of M of an array of N shared ints. Because that makes it a normal C pointer, p has no thread or phase associated with it, so the fact that it points to data on Thread t cannot be represented and it cannot participate in pointer-to-shared arithmetic.

To go contrary to C99, we'd need to decide that code similar to the above is sufficiently useful. ```

Reported by `johnson.troy.a` on 2012-05-31 16:45:26

[Former user]

``` Er, that should be "(*p)[0 <= i < M][0 <= j < N] reference data on Thread t." ```

Reported by `johnson.troy.a` on 2012-05-31 16:47:54

[Former user]

``` For an example of "real code" that tries to use a pointer to an array of shared, see the UPC Run Time Error Detection Test suite from ISU: http://rted.public.iastate.edu/UPC/

Where tests such as RTED_UPC/UPC/C_Outofbnds/c_C_10_1_a_F.upc have declarations like:

shared [K] float (* arrA)[K]; ```

Reported by `johnson.troy.a` on 2012-06-05 19:06:33

[Former user]

``` I'd like to see this answered in 1.3. ```

Reported by `johnson.troy.a` on 2012-06-15 17:09:30 - Labels added: Milestone-Spec-1.3

[Former user]

``` As an additional data point, clang and GCC appear to differ on the interpretation of whether the target of a pointer to an array which in turns has type qualified elements is itself type qualified or not.

Consider:

const int array[10]; void *ptr = &array;

Here is what a recent version of clang does with the test case.

$ clang -cc1 -std=c99 t.c t.c:2:7: warning: initializing 'void *' with an expression of type 'const int (*)[10]' discards qualifiers void *ptr = &array; ^ ~~~~~ 1 warning generated.

which indicates that clang propagated the element type qualifiers to the array type that is the target of the declared pointer type.

A recent version of GCC on the other hand will compile the test case above without complaint (even with full pedantic warnings enabled).

Thus, GCC supports the interpretation cited by Troy, and clang does not.

GCC will issue a warning given this variation.

const int array[10]; void *ptr = array;

$ gcc -std=c99 -c t.c t.c:2:13: warning: initialization discards ‘const’ qualifier from pointer target type [enabled by default]

This makes sense, because per the C99 specification 'array' on its own is interpreted as &array[0] which is a pointer to the first element of 'array'.

```

Reported by `gary.funck` on 2012-06-29 23:18:52

[Former user]

``` As a follow up to comment #12 above, regarding a compiler's response to a constraint violation ("shall" or "shall not"), the following section of the C99 specification is applicable.

5.1.1.3 Diagnostics

1 A conforming implementation shall produce at least one diagnostic message (identified in an implementation-defined manner) if a preprocessing translation unit or translation unit contains a violation of any syntax rule or constraint, even if the behavior is also explicitly specified as undefined or implementation-defined. Diagnostic messages need not be produced in other circumstances.

Per above, doesn't the specification require that a UPC compiler issue a diagnostic if the operand of upc_blocksizeof is not a shared qualified expression?

```

Reported by `gary.funck` on 2012-06-30 00:14:36

[Former user]

``` An additional example, using upc_forall:

1. include <upc.h>
2. include <stdio.h>
3. include <assert.h>

int main(void) { static shared [10] int array[THREADS][10]; upc_forall(int i = 0; i < THREADS; ++i; array[i]) { int *p = (int *)array[i]; for (int j = 0; j < 10; ++j) { p[j] = MYTHREAD * 10 + (j + 1); } } upc_barrier; if (MYTHREAD == 0) { for (int i = 0; i < THREADS; ++i) for (int j = 0; j < 10; ++j) { printf ("array[%d][%d] = %d\n", i, j, array[i][j]); assert (array[i][j] == i * 10 + (j + 1)); } printf ("Test passed.\n"); } return 0; }

This test case passes when compiled with GUPC.

BUPC issues the following error:

forall_array.upc: In function `main': forall_array.upc:8: Affinity expression is not an integral/shared address expression

which makes sense. Since GUPC is probably using logic common to GCC, I'm guessing that the expression was promotoed to the address of the array.

```

Reported by `gary.funck` on 2012-07-05 20:58:57

[Former user]

``` "As an additional data point, clang and GCC appear to differ on the interpretation of whether the target of a pointer to an array which in turns has type qualified elements is itself type qualified or not."

I'm guessing clang's warning message is just a bit unclear and that they're still simply saying that the element type 'const int' is qualified, and by casting the pointer to void *, that qualification is lost. The message is simply ambiguous as to what part of the type is qualified. I can't imagine them ignoring C99 6.7.3.8 that blatantly.

)

```

Reported by `[e-mail]` on 2012-08-14 14:59:21

[Former user]

``` Steve wrote: "I'm guessing clang's warning message is just a bit unclear and that they're still simply saying that the element type 'const int' is qualified, and by casting the pointer to void *, that qualification is lost."

Agreed. This version of the test compiles without complaint.

int main(void) { const int array[10]; const void *ptr = &array; return 0; } ```

Reported by `gary.funck` on 2012-08-14 15:40:34

[Former user]

``` As a more motivating example of the implications of this, consider trying to dynamically allocate a multidimensional shared array:

shared int (*array0)[10]; shared int (*array1)[10]; array0 = (shared int (*)[10]) upc_all_alloc( 10 * THREADS, sizeof( int ) ); array1 = array0 + 1;

Where does array1 point, and why?

Is it even possible to dynamically allocate a multidimensional shared array under the pointer-to-local interpretation, given that the return from all the dynamic shared allocation routines returns a pointer-to-shared that would be immediately cast to pointer-to-local? ```

Reported by `[e-mail]` on 2012-08-14 16:29:18

[Former user]

``` Set default Consensus to "Low". ```

Reported by `gary.funck` on 2012-08-19 23:26:19 - Labels added: Consensus-Low

[Former user]

``` Note that code affected by this is very common in some publicly available test suites--notably the Run-time Error Detection suite from Iowa State University (http://rted.public.iastate.edu/UPC/) and the GWU Unified Testing Suite suite from (The) George Washington University (http://threads.hpcl.gwu.edu/sites/guts). It appears that they are expecting the "pointer-to-shared" interpretation, as the code doesn't work as documented under the "pointer-to-local" interpretation. ```

Reported by `[e-mail]` on 2012-09-04 16:45:20

[Former user]

``` Another small program to consider. I'm not sure what this argues for, as the results do not seem intuitive to me under either interpretation.

1. include<stdio.h>
2. include<upc.h>

typedef int MYTYPEA; typedef int MYTYPEB[2];

shared [2] MYTYPEA a[2*THREADS]; shared [2] MYTYPEB b[2*THREADS];

int main() { if ( MYTHREAD == 0 ) { if ( upc_threadof( &a[0] ) == upc_threadof( &a[1] ) ) { printf( "&a[0] and &a[1] point to the same thread\n" ); } else { printf( "&a[0] and &a[1] point to different threads\n" ); }

if ( upc_threadof( &b[0] ) == upc_threadof( &b[1] ) ) { printf( "&b[0] and &b[1] point to the same thread\n" ); } else { printf( "&b[0] and &b[1] point to different threads\n" ); } } }

Under the local pointer interpretation, &b[0] and &b[1] are NOT shared pointers, and it is therefore erroneous to pass them to upc_threadof().

Under the shared pointer interpretation, when run with more than 1 thread, it should print

&a[0] and &a[1] point to the same thread &b[0] and &b[1] point to different threads

which seems really odd until you look at the typedefs. Perhaps EXAMPLE 4 under Array declarators could be expanded upon to make this result more clear? ```

Reported by `[e-mail]` on 2012-09-04 21:56:14

[Former user]

``` Some additional observations, consider UPC v1.2 6.4.2 4:

"In addition, the correspondence between shared and local addresses and arithmetic is defined using the following constructs:

T *P1, *P2; shared T *S1, *S2;

P1 = (T*) S1; /* allowed if S1 has affinity to MYTHREAD */ P2 = (T*) S2; /* allowed if S2 has affinity to MYTHREAD */"

Assume T is defined as

typedef int T[2];

Then it is erroneous to use the results of those casts in the general case, as (*P1)[1] (or (*P2)[1]) may point to unused memory as the shared object may have no additional elements on the local thread.

Also, consider C99 6.5.2.1 3:

"Successive subscript operators designate an element of a multidimensional array object. If E is an n-dimensional array (n >= 2) with dimensions ixjx...xk then E (used as other than an lvalue) is converted to a pointer to an (n-1)-dimensional array with dimensions jx...xk. If the unary * operator is applied to this pointer explicitly, or implicitly as a result of subscripting, the result is the pointed-to (n-1)-dimensional array, which itself is converted into a pointer if used as other than an lvalue."

Indexing of multidimensional arrays is defined using pointers to arrays, and therefore, indexing of multidimensional shared arrays is defined using pointers to shared arrays. Therefore, this issue affects not only pointers to shared arrays, but any non-trivial use of multidimensional shared arrays. ```

Reported by `[e-mail]` on 2012-09-07 16:43:54

[Former user]

``` Getting back to the original issue here..

It seems to me that at a high-level this problem arises because UPC defines "shared" as a C type qualifier (mostly for syntactic convenience), but it's really much more than that. When C99's type qualifiers (const, restrict, volatile) are applied to an object (eg an array element) they truly qualify and enforce a property upon that specific element (eg const states that element may not be modified), and the property is orthagonal to the (cast-free) expression used to reach the object. UPC's shared qualifier is different, because it specifies not only a property of the qualified object (ie it lives in shared space) but ALSO a property of the expression used to reach that object (ie a pointer-to-shared must be a "wide" pointer, with a different representation than a pointer-to-local). So when you declare a pointer-to-shared (shared int *p), it is technically the *referent* type that is shared-qualified, but the representation and type compatibility of the pointer itself is also affected. I believe there is no analogue to this in C99. The UPC shared qualifier is already "much more" than a plain C99 type qualifier, and most of UPC 6.4 is dedicated to explaining the monstrous additional properties of our "super qualifier". I don't see it as a serious additional conflict with C99 to extend the specification of our super-qualifier a tiny bit further to include a special relationship with arrays.

IMHO it seems linguistically cleanest (and also most intuitive) to treat pointer-to-shared-array similarly to pointer-to-shared-scalar, ie follow the "pointer-to-shared" interpretation suggested in comment #0. This has the benefit of maintaining the duality of pointers and arrays that most C programmers intuitively take for granted, extended to shared types.

Proposal:

Keeping in mind: C99 6.7.3.8 - "If the specification of an array type includes any type qualifiers, the element type is so qualified, not the array type."

Update the following text from UPC 1.2 6.5.2.1.4:

"In an array declaration, the type qualifier applies to the elements."

to read something like:

"As specified in [C99 6.7.3.8], if the specification of an array type includes any type qualifiers (possibly via one or more typedefs), the element type is so qualified, not the array type. Additionally, the shared qualifier ALSO applies to the array itself\footnote{This implies that a pointer to an array-of-shared is itself a pointer-to-shared.}"

Much of the discussion in prior comments seems to implicitly assume we should follow the pointer-to-local interpretation, but aside from quoting existing spec language I don't see any strong arguments against taking the pointer-to-shared interpretation suggested in this proposal. Everyone seems to agree this is the most intuitive behavior, so is there anything that "breaks" if we clarify to take this option?

```

Reported by `danbonachea` on 2012-09-07 17:56:09

[Former user]

``` "As specified in [C99 6.7.3.8], if the specification of an array type includes any type qualifiers (possibly via one or more typedefs), the element type is so qualified, not the array type. Additionally, the shared qualifier ALSO applies to the array itself\footnote{This implies that a pointer to an array-of-shared is itself a pointer-to-shared.}"

I think it would be much simpler to change the definition of 'pointer-to-shared' (UPC 3.4.1) to be 'a pointer whose referenced type is shared qualified or is a shared array type'. Then we don't need an exception to C99 6.7.3.8. ```

Reported by `[e-mail]` on 2012-09-07 20:10:56

[Former user]

``` RE Comment 33: "Much of the discussion in prior comments seems to implicitly assume we should follow the pointer-to-local interpretation, but aside from quoting existing spec language I don't see any strong arguments against taking the pointer-to-shared interpretation suggested in this proposal. Everyone seems to agree this is the most intuitive behavior, so is there anything that "breaks" if we clarify to take this option?"

I'm fairly neutral on what option UPC /should/ support. The pointer-to-local interpretation has some limited utility in allowing a thread to indirectly refer to one of many arrays of shared that have the same shape, but it can point only to the start of a whole array since it lacks affinity and phase information. The pointer-to-shared interpretation has greater utility in providing access to slices of multidimensional arrays of shared objects, but then again multidimensional arrays are generally accepted to be a pain in C and I'm not sure that it was a goal of UPC to address that in any way.

Given public codes that I have seen, there is some user desire for the pointer-to-shared interpretation. What "breaks" if we choose it is not anything else in the UPC spec, although for clarity the section on pointer addition could benefit from a note about how it applies to a pointer to array of shared. What breaks is existing compiler type systems. The pointer-to-local interpretation is what naturally falls out by following the C rules, at least for Cray and GCC. So settling on the pointer-to-shared interpretation for UPC 1.3 is basically saying to at least two implementers "go change your type system." I don't mind having to make the change, I just wonder if that's the sort of change that people expected in a 1.2->1.3 version bump. ```

Reported by `johnson.troy.a` on 2012-09-13 16:55:11

[Former user]

``` "So settling on the pointer-to-shared interpretation for UPC 1.3 is basically saying to at least two implementers "go change your type system.""

The same is true for settling on the pointer-to-local interpretation.

"I think it would be much simpler to change the definition of 'pointer-to-shared' (UPC 3.4.1) to be 'a pointer whose referenced type is shared qualified or is a shared array type'."

We would also need a clarification to the definition of "shared array type":

Original: "shared array: an array with elements that have shared qualified type" Suggestion 1: "shared array: an array with elements that have shared qualified or shared array type" Suggestion 2: "shared array: an array with an underlying element type that is shared qualified"

Without this change, it is not clear that AAA in

shared int AAA[3][4][5][6];

is a "shared array" type, because the elements of AAA are "array [4][5][6] of shared int", and are thus not shared qualified. (Question: "underlying element type" is a well-understood concept, isn't it? It wouldn't need explanation?)

Dan's suggested change to what is considered "shared qualified" covers this case by making each of the array components "shared qualified" recursively.

I think both wordings are good. I have a slight preference for keeping the concept of "shared qualified" attached to the underlying element type, but I like the simplicity of Dan's suggestion, too. ```

Reported by `brian.wibecan` on 2012-09-13 17:39:43

[Former user]

``` "The same is true for settling on the pointer-to-local interpretation."

Yep, certainly true. Cray UPC follows pointer-to-local and notes in documentation that the UPC spec is ambiguous, but I'm sure there are other UPC implementations that follow pointer-to-shared. It would be interesting to have a count of each, but I'm just raising the question of if it's a UPC 1.2->1.3 level of change to actually pick one. I've investigated the changes enough that I believe it's not a massive undertaking for our implementation, so I'm fine with pointer-to-shared for 1.3, but I could sympathize with other implementations where it might require more work or where they were not expecting a type system change. ```

Reported by `johnson.troy.a` on 2012-09-13 17:57:25

[Former user]

``` "(Question: "underlying element type" is a well-understood concept, isn't it? It wouldn't need explanation?)"

Actually, we already have this problem elsewhere in the spec, for instance 6.5.1.10 says (emphasis mine):

"The layout qualifier dictates the blocking factor for the type being shared qualified. This factor is the nonnegative number of consecutive ELEMENTS (when evaluating pointer-to-shared arighmetic and array declarations) which have affinity to the same thread..." ```

Reported by `[e-mail]` on 2012-09-13 18:00:21

[Former user]

``` "so is there anything that "breaks" if we clarify to take this option?"

We'll probably have to clean-up the pointer-to-shared arithmetic section, as there's nothing in there that clearly states how it should work in this case. For the expected behavior, I believe we'd have to point out that the increment would need to be scaled by the number of elements in the array type when determining the new phase and thread. ```

Reported by `[e-mail]` on 2012-09-13 18:04:35

[Former user]

``` Steve proposed: "I think it would be much simpler to change the definition of 'pointer-to-shared' (UPC 3.4.1) to be 'a pointer whose referenced type is shared qualified or is a shared array type'. Then we don't need an exception to C99 6.7.3.8."

I'm concerned Steve's suggested wording may be too strong (or possibly just ambiguous). We are trying to clarify that an array of shared elements is itself a shared-qualified type, and thus a POINTER to an array of shared is a pointer-to-shared. Steve's wording could be interpreted (misread?) to imply that a shared array type IS a pointer-to-shared, which I don't think is what we want. In the spirit of C99 where pointers and arrays are often interchangeable, we want shared arrays and pointer-to-shared to be interchangeable in similar ways - but that is weaker than stating they are EQUIVALENT types, which they definitely are not (prime example being when you pass one to sizeof()). I think he probably meant to write something more like 'a pointer whose referenced type is shared qualified or WHOSE REFERENCED TYPE is a shared array type'

In any case, Steve's proposal is also incomplete - both in the ways pointed out by Brian in comment #36 wrt multi-D arrays, and more importantly in that it references a "shared array type", which as we've already discussed is a type that does not exist under C99 6.7.3.8, until we add an explicit exception.

So I stand by my original proposal in comment #33.

```

Reported by `danbonachea` on 2012-09-13 18:12:11

[Former user]

``` I don't like the proposal in comment #33, because it explicitly permits a shared-qualified type that doesn't obey affinity rules, doesn't permit a layout specifier, and could span multiple threads. It also needlessly violates C99 6.7.3.8. Perhaps the wording in comment #34 is bad, but that can be fixed without adding new shared-qualified types that don't act like other shared-qualified types.

In general, there are a lot of places in the 1.2 spec that assume all shared arrays have a single dimension and are ambiguous if they have more than one dimension. For instance (emphasis mine),

3.2.3.1 "shared array an array with ELEMENTS that have shared qualified type"

3.9.1 "phase an unsigned integer value associated with a pointer-to-shared which indicates the ELEMENT-offset within an affinity block; ..."

6.4.2.4 { see comment 32 }

6.4.2.5 "...The expression ((ptrdiff_t) upc_addrfield(S2) - (ptrdiff_t) upc_addrfield(S1)) shall evaluate to the same value as ((P2 - P1) * sizeof(T))"

This fails miserably if T is an array type.

6.4.3.6 "Shared objects with affinity to a given thread can be accessed by either pointers-to-shared or pointers-to-local of that thread."

What is the affinity of a shared object that spans multiple threads?

6.5.1.1.10 "... This factor is the nonnegative number of consecutive ELEMENTS (when evaluating pointer-to-shared arighmetic and array declarations) which have affinity to the same thread..."

6.5.2.1.3 "ELEMENTS of shared arrays are distributed in a round robin fashion, by chunks of block-size ELEMENTS, such that the i-th ELEMENT has affinity with thread (floor (i/block_size) mod THREADS)"

Fortran 2008 side-stepped this issue nicely by making dimensions that span images distinct from dimensions that span memory addresses. UPC made things ridiculously complicated by supporting multiple block distributions that are barely used because they are so complicated. ```

Reported by `[e-mail]` on 2012-09-13 18:57:46

[Former user]

``` "I don't like the proposal in comment #33, because it explicitly permits a shared-qualified type that doesn't obey affinity rules, doesn't permit a layout specifier, and could span multiple threads."

This is a fair point.. which leaves us with no complete proposal that gives us everything we want.

Perhaps we should consider just DIRECTLY stating the property we're looking for, with something like:

Change 1.2 6.5.2.1.4: "In an array declaration, the type qualifier applies to the elements."

to read something like:

"As specified in [C99 6.7.3.8], if the specification of an array type includes any type qualifiers (possibly via one or more typedefs), the element type is so qualified, not the array type. Additionally, a pointer to an array of shared elements is a pointer-to-shared.\footnote{This also applies to multi-dimensional arrays, so a pointer to an array of array of shared elements is a pointer-to-shared.}"

With the second sentence being the direct statement of the clarification we are attempting to provide. The footnote clarifies the rule also applies to multi-D.

"In general, there are a lot of places in the 1.2 spec that assume all shared arrays have a single dimension and are ambiguous if they have more than one dimension."

This is a valid point and seems worthy of study, but is also somewhat off-topic. Steve can you please create a new separate issue to address those ambiguities? ```

Reported by `danbonachea` on 2012-09-13 19:32:42

[Former user]

``` "Steve can you please create a new separate issue to address those ambiguities?"

Issue 85 has been created. ```

Reported by `[e-mail]` on 2012-09-13 19:41:28

[Former user]

``` I'll handle this one. ```

Reported by `ga10502` on 2012-09-14 21:26:48

[Former user]

Reported by `ga10502` on 2012-09-14 21:27:38

[Former user]

``` Sorry about the confusion. It's Friday evening and I hit the wrong key. Obviously. ```

Reported by `ga10502` on 2012-09-14 21:28:11

[Former user]

``` "I'll handle this one."

Didn't I get the action item on this (to write up the proposed changes with Dan's help)? ```

Reported by `[e-mail]` on 2012-09-14 21:29:03

[Former user]

``` fixed. ```

Reported by `danbonachea` on 2012-09-14 21:36:07

[Former user]

Reported by `[e-mail]` on 2012-09-21 19:30:50 - Blocking: #85

[Former user]

Set critical priority on the unresolved technical issues slated for 1.3 which are blocking
progress on ratification.

Reported by danbonachea on 2013-01-24 07:40:02 - Labels added: Priority-Critical - Labels removed: Priority-Medium

[Former user]

Below are the proposed additions and modifications (against v1.2) to clarify how multi-dimensional
shared arrays and pointers-to-shared whose referenced type is an array type work. 
Once I manage to get latex working correctly on our development machines, I'll post
the formal proposal.

[ADDED]
{3.*}
ultimate element
of a scalar is the scalar itself.  For arrays, the ultimate element is the ultimate
element of an element of the array.

{3.*}
ultimate element type
for non-array types, the type itself.  For array types, the ultimate element type of
an element of the array.

{3.*}
shared type
a type whose ultimate element type is shared-qualified.

{3.*}
local type
a type whose ultimate element type is not shared-qualified.

{6.4.2 *}
The affinity of a non-null pointer-to-shared whose reference type is a shared array
type is that of its first ultimate element, as a shared array type may have ultimate
elements on more than one thread.

{6.4.3 *}
If a non-null pointer-to-shared with definite blocksize is cast to a pointer-to-local
whose referenced type is not the ultimate element type of the referenced type of the
pointer-to-shared, the result is undefined.

{6.5.1.1 *}
EXAMPLE 4: shared array

    shared TYPE array[THREADS]; /* NOTE: 'upc_affinityof( &array[i] )' could be something
other than 'i' if TYPE is an array type */


[MODIFIED]
{3.2.3}
shared array
an array whose ultimate element type is shared-qualified.

{3.4}
pointer-to-shared
a pointer whose referenced type is a shared type.

{3.5}
pointer-to-local
a pointer whose referenced type is a local type.

{3.6.1}
shared access
an access using an expression whose type is a shared type.

{3.6.2}
local access
an access using an expression whose type is a local type.

{6.4.1.2 1}
The upc_localsizeof operator shall only apply to shared types or expressions whose
type is a shared type.  All constraints on the sizeof operator [ISO/IEC00 Sec. 6.5.3.4]
also apply to this operator.

{6.4.1.3 1}
The upc_blocksizeof operator shall only apply to shared types or expressions whose
type is a shared type.  All constraints on the sizeof operator [ISO/IEC00 Sec. 6.5.3.4]
also apply to this operator.

{6.4.1.4 1}
The upc_elemsizeof operator shall only apply to shared types or expressions whose type
is a shared type.  All constraints on the sizeof operator [ISO/IEC00 Sec. 6.5.3.4]
also apply to this operator.

{6.4.1.4 2}
The upc_elemsizeof operator returns the size, in bytes, of the ultimate element type
of a shared type or expression whose type is a shared type.  The result of upc_elemsizeof
is an integer constant.

{6.4.2 2}
.... If the shared array has a definite block size, then the following example describes
pointer arithmetic:

    shared [B] TYPE (*p), (*p1); /* B a positive integer, TYPE is a local type */
    int i;

    p1 = p + i;

{6.4.2 3}
...:

    size_t elem_delta = i * (sizeof(TYPE) / upc_elemsizeof(TYPE))

    upc_phaseof(p1) == (upc_phaseof(p) + elem_delta) mod B
    upc_threadof(p1) == (upc_threadof(p) + (upc_phaseof(p) + elem_delta) div B) mod
THREADS

{6.4.2 4}
...:

    LT *p1, *p2;
    shared ST (*s1), (*s2); /* LT is the non-shared-qualified version of the ultimate
element type of ST */

    p1 = (LT*) s1; /* Allowed if s1 has affinity to MYTHREAD */
    p2 = (LT*) s2; /* Allowed if s2 has affinity to MYTHREAD */

{6.4.2 5}
...:
    ...
    * The expression (((ptrdiff_t) upc_addrfield(s2)) - ((ptrdiff_t) upc_addrfield(s1)))
shall evaluate to the same value as ((p2 - p1) * sizeof(LT)).

{6.5.1.1 10}
....  This factor is the nonnegative number of consecutive ultimate elements (when
evaluating pointer-to-shared arithmetic and array declarations) which have affinity
to the same thread....

{6.5.2 8}
...and no array object with automatic storage duration shall have an ultimate element
type that is shared-qualified.

{6.5.2.1 3}
Ultimate elements of shared arrays are distributed in a round-robin fashion, by chunks
of block-size ultimate elements, such that the i-th ultimate element has affinity with
thread (floor(i/block_size) mod THREADS).

{6.5.2.1 4}
In an array declaration, the type qualifier applies to the ultimate elements.

{6.5.2.1 9}
...:
   ...

shared [3] applies to the ultimate element type of T, which is int, regardless of the
typedef. ...

   ...

{7.2.3.5 4}
In the case of a statically allocated shared object with declaration:

    shared [b] t d[s];

the totalsize argument shall be s * sizeof(t) and the nbytes argument shall be b *
upc_elemsizeof(t). ...

Reported by [e-mail] on 2013-02-05 17:58:01

[Former user]

Thanks Steve - I think we're headed in the right direction here. I know I'll have more
comments on the proposed changes once I see the full diff against the current draft
and have some more time to study it, but here are some initial thoughts..

>ultimate element type
>ultimate element 

In the interests of simplicity, can we get away with just the term "ultimate element
type", and eliminate the second term (the instantiated version)? The former is a very
useful concept, but the latter seems only tangentially useful and the definition is
somewhat ambiguous:

> ultimate element of a scalar is the scalar itself.  For arrays, the ultimate element
is the ultimate element of an element of the array.

This wording is confusing because "the ultimate element" of an array is not a unique
entity - ie it should probably read "an ultimate element" not "the ultimate element",
but even then the definition seems shaky. Also, the term "scalar" is not what we want
here, because it excludes struct and union types (probably wanted "non-array object"
instead). Overall seems better to ditch this term entirely and always specify things
in terms of "object of ultimate element type".

> shared type
> local type

Similarly, since these two are mutually exclusive sets, it seems sufficient to only
define "shared type", and use the phrase "not a shared type" instead of "local type".
This wording is also nice because it seems self-evident that any vanilla C99 type is
"not a shared type", but a novice might wonder if their C99 type constitutes a "local
type".

> The affinity of a non-null pointer-to-shared whose reference type is a shared array
type is that of its first ultimate element, as a shared array type may have ultimate
elements on more than one thread.

This paragraph conflates types and instantiated objects (eg a "shared array type" does
not have any instantiated "ultimate elements", "on" more than one thread or anywhere
else). Also, the phrase "affinity of a pointer" generally means the affinity of the
pointer object itself, not to be confused with the affinity of the referenced object
which is what we mean here.
I'm not sure I fully grasp what this statement is trying to say, but perhaps we could
use a more direct statement? Eg:
   "A pointer to a shared array is a pointer-to-shared that points to the initial element
of the array object."

> If a non-null pointer-to-shared with definite blocksize is cast to a pointer-to-local
whose referenced type is not the ultimate element type of the referenced type of the
pointer-to-shared, the result is undefined.

This is way too strong as currently written. 
At least need to insert: "is not compatible with a qualified or unqualified version
of the ultimate element type" 
Also, casts to (void *) need to be permitted.
It might be easier/safer to rephrase this as a prohibition against casting to pointer-to-local-array,
since I think that's what you're trying to prohibit here?

> EXAMPLE 4: shared array
>    shared TYPE array[THREADS]; /* NOTE: 'upc_affinityof( &array[i] )' could be something
other than 'i' if TYPE is an array type */

This example is trying to illustrate a property of multi-dimensional shared arrays,
so it might be clearer to directly show one?

> The upc_*sizeof operator shall only apply to shared types or expressions whose type
is a shared type.

Suggested rewording: "expressions with shared type." (4 occurrences)

>     size_t elem_delta = i * (sizeof(TYPE) / upc_elemsizeof(TYPE))

This part of the specification is ill-defined because TYPE is not a "shared type",
and therefore an invalid argument to upc_elemsizeof(). Suggest replacing it with: upc_elemsizeof(array)

> {6.5.2 8}...and no array object with automatic storage duration shall have an ultimate
element type that is shared-qualified.

Why not simply replace this whole paragraph with something like:
"No object with automatic storage duration shall have shared type."

> {7.2.3.5 4}
> In the case of a statically allocated shared object with declaration:
>    shared [b] t d[s];
> the totalsize argument shall be s * sizeof(t) and the nbytes argument shall be b
* upc_elemsizeof(t). ...

Another invalid use of upc_elemsizeof(). Should probably be: "b * upc_elemsizeof(d)"

> {6.5.2.1 4}In an array declaration, the type qualifier applies to the ultimate elements.

Both the old and new text are a bit too "loose" here. Qualifiers are applied to types,
not objects (or elements). It also notably doesn't cover typedefs or types appearing
in casts.
Further, this is the paragraph discussed in earlier comments, which I proposed to rewrite
to something like:

"As specified in [C99 6.7.3.8], if the specification of an array type includes any
type qualifiers (possibly via one or more typedefs), the element type is so qualified,
not the array type. Additionally, a pointer to an array of shared elements is a pointer-to-shared.\footnote{This
also applies to multi-dimensional arrays, so a pointer to an array of array of shared
elements is a pointer-to-shared.}" 

Related to this, the proposal doesn't seem to currently address the central question
in issue 3 - ie what do you get when you apply the address-of operator to a shared
array? This was discussed at length in earlier comments, but doesn't seem to be covered
by any of the proposed changes, and seems important enough to be directly specified.
I thought we had agreed to specify the pointer-to-shared interpretation, with something
like comment #42.

Here's some concrete examples for discussion:

1:  shared int x[5];
2:  shared int y[5][5];
3:  shared int *p1;
4:  shared int (*p2)[5];
...
5:  p1 = x;     // should be legal..
6:  p1 = &x[0]; // equivalent to this, also legal
7:  p1 = &y[0][0]; // also also legal
8:  upc_threadof(&x[0]); // must be permitted
9:  upc_threadof(&y[0][0]); // must be permitted
10: upc_threadof(p1);    // must be permitted
11: upc_threadof(x);     // silly but should probably also be permitted
12: p2 = &x;             // should be legal?
13: upc_threadof(&x);    // legal for PTS interpretation, constraint violation if the
expression type is pointer-to-local
14: p2 = y;     // another problematic case
15: upc_threadof(p2); // more questionable cases...
16: upc_threadof(*p2);
17: upc_threadof(&y[2]);

We could potentially eliminate half of the problem here by prohibiting the address-of
operator on expressions of shared array type. However, we still need to clarify whether
p2 is a pointer-to-shared or a pointer-to-local, and which of the lines above are legal
or prohibited. 

Crazy Idea: A stronger and simpler approach would be to prohibit the type pointer-to-shared-array
entirely, since objects of that type have very limited utility anyhow. This approach
would force users to always use a pointer to ultimate element type (in which case p2
and lines 12-17 would all be rejected by the typechecker). I'm guessing this idea is
controversial, and I'm not sure what the practical impact would be on users. Do we
have important use cases that motivate continuing to allow this problematic type?


Some relevant text from C99:
C99 6.3.2.1 - Implicit conversion for array operands
"Except when it is the operand of the sizeof operator, or the unary & operator, or
is a
string literal used to initialize an array, an expression that has type ‘‘array of
type’’ is
converted to an expression with type ‘‘pointer to type’’ that points to the initial
element of
the array object and is not an lvalue."

C99 6.5.3.2-3 - Semantics of &
"The unary & operator yields the address of its operand. If the operand has type ‘‘type’’,
the result has type ‘‘pointer to type’’. If the operand is the result of a unary *
operator,
neither that operator nor the & operator is evaluated and the result is as if both
were
omitted, except that the constraints on the operators still apply and the result is
not an
lvalue. Similarly, if the operand is the result of a [] operator, neither the & operator
nor
the unary * that is implied by the [] is evaluated and the result is as if the & operator
were removed and the [] operator were changed to a + operator. Otherwise, the result
is
a pointer to the object or function designated by its operand."

A very helpful page discussing the nitty-gritty details of C arrays and pointers:
http://c-faq.com/~scs/cgi-bin/faqcat.cgi?sec=aryptr

A C99-compliant example showing some of the relevant things C99 permits:
int x[5];
int y[5][5];
int main() {
  int *p1 = x;
  int (*p2)[5];
  p1 = &x[0];
  p2 = &x;
  (*p2)[2] = 7;
  p2 = y;
  p2 = &y[1];
  return 0;
}

Reported by danbonachea on 2013-02-11 09:45:23

[Former user]

"Related to this, the proposal doesn't seem to currently address the central question
in issue 3 - ie what do you get when you apply the address-of operator to a shared
array? This was discussed at length in earlier comments, but doesn't seem to be covered
by any of the proposed changes, and seems important enough to be directly specified.
I thought we had agreed to specify the pointer-to-shared interpretation, with something
like comment #42."

This naturally falls out of the new definition of pointer-to-shared.  Since a shared
array is a shared-type (its ultimate element type is shared-qualified), any pointer
to it (due to taking its address for instance...) is a pointer-to-shared.

Reported by [e-mail] on 2013-02-11 18:47:16

[Former user]

"In the interests of simplicity, can we get away with just the term "ultimate element
type", and eliminate the second term (the instantiated version)? The former is a very
useful concept, but the latter seems only tangentially useful and the definition is
somewhat ambiguous:"

Sure.  I just added it to save some typing on my part.

"This wording is confusing because "the ultimate element" of an array is not a unique
entity - ie it should probably read "an ultimate element" not "the ultimate element",
but even then the definition seems shaky. Also, the term "scalar" is not what we want
here, because it excludes struct and union types (probably wanted "non-array object"
instead). Overall seems better to ditch this term entirely and always specify things
in terms of "object of ultimate element type"."

Sounds good to me.

"This is way too strong as currently written. 
At least need to insert: "is not compatible with a qualified or unqualified version
of the ultimate element type" 
Also, casts to (void *) need to be permitted."

Yes, I forgot to permit casting to compatible local types--that should definitely be
allowed.

Reported by [e-mail] on 2013-02-11 18:53:56

[Former user]

> This naturally falls out of the new definition of pointer-to-shared.  Since a shared

> array is a shared-type (its ultimate element type is shared-qualified), any pointer

> to it (due to taking its address for instance...) is a pointer-to-shared.

I agree the framework is compatible with this interpretation, but it still seems worth
explicit clarification. Also, simply saying it is a shared type doesn't really specify
the semantics of an example like:

shared [5] int a[1000][THREADS];
upc_blocksizeof(&a);

Is the blocksize of the ultimate element type propagated to pointers to the array?
If so where does it say that?

Reported by danbonachea on 2013-02-11 18:58:52

[Former user]

"Is the blocksize of the ultimate element type propagated to pointers to the array?
If so where does it say that?"

Ah, I forgot to add a sentence to upc_blocksizeof() stating that it returned the blocksize
of the the ultimate element type, similar to the language for upc_elemsizeof().

Reported by [e-mail] on 2013-02-11 19:03:48

[Former user]

Ignore my example from comment 55 - as written it's a constraint violation of upc_blocksizeof()

Reported by danbonachea on 2013-02-11 19:42:32

[Former user]

Issue 85 has been merged into this issue.

Reported by [e-mail] on 2013-02-20 00:00:05

[Former user]

Reported by [e-mail] on 2013-02-20 00:00:05 - No longer blocking: #85

[Former user]

Attached is a first pass at the proposed changes in the spec.  The changes mostly follow
those in comment 51, with some corrections from comments 52-56.

Reported by [e-mail] on 2013-02-20 00:07:18 - Labels added: Consensus-Medium - Labels removed: Consensus-Low

<hr> * Attachment: upc-lang-spec-draft.pdf

[Former user]

Reported by [e-mail] on 2013-02-20 00:07:51 - Status changed: Started

[Former user]

> Attached is a first pass at the proposed changes in the spec.  

Could you please attach a Latex diff?

Reported by danbonachea on 2013-02-20 04:34:33

[Former user]

> Could you please attach a Latex diff?

I made all the changes with the \xadded and \xreplaced macros, so they should be explicitly
spelled out in the PDF, along with the old text where applicable.  Since there's a
lot of small changes all over the place, it's probably easier to review by checking
the change bars than to go over the latex diff.  Still, I'll make diffs and attach
them tomorrow.

Reported by [e-mail] on 2013-02-20 06:01:32

[Former user]

Attached are svn diffs against the current trunk (r205) for the three changed files
(upc-language.tex, upc-lib-core.tex, upc-terms-and-defs.tex) used to produce the PDF
in comment 60.

Reported by [e-mail] on 2013-02-20 15:34:08

<hr> * Attachment: upc-language.dif * Attachment: upc-lib-core.dif * Attachment: upc-terms-and-defs.dif

[Former user]

As promised at our last telcom, attached is a test-case Cray used for testing while
updating our compiler for the changes proposed here.  It tests various ways of indexing
a multidimensional shared array.  Thread 0 prints the array out for each method of
indexing as the checking occurs.

As an example of incorrect output, consider the output when compiled with CCE 8.1 and
run with 2 threads.

array2d:
    0@0:0    1@0:1    2@0:2    3@0:3    4@0:4    5@1:0    6@1:1
    7@1:2    8@1:3    9@1:4   10@0:0   11@0:1   12@0:2   13@0:3
   14@0:4   15@1:0   16@1:1   17@1:2   18@1:3   19@1:4   20@0:0
   21@0:1   22@0:2   23@0:3   24@0:4   25@1:0   26@1:1   27@1:2
   28@1:3   29@1:4   30@0:0   31@0:1   32@0:2   33@0:3   34@0:4
   35@1:0   36@1:1   37@1:2   38@1:3   39@1:4   40@0:0   41@0:1

array2d (derefi):
    0@0:0    1@0:1    2@0:2    3@0:3    4@0:4    5@1:0    6@1:1
    0@0:0    1@0:1    2@0:2    3@0:3    4@0:4    5@1:0    6@1:1
    0@0:0    1@0:1    2@0:2    3@0:3    4@0:4    5@1:0    6@1:1
    0@0:0    1@0:1    2@0:2    3@0:3    4@0:4    5@1:0    6@1:1
    0@0:0    1@0:1    2@0:2    3@0:3    4@0:4    5@1:0    6@1:1
    0@0:0    1@0:1    2@0:2    3@0:3    4@0:4    5@1:0    6@1:1

array2d (derefj):
    0@0:0    1@0:1    2@0:2    3@0:3    4@0:4    5@1:0    6@1:1
    7@1:2    8@1:3    9@1:4   10@0:0   11@0:1   12@0:2   13@0:3
   14@0:4   15@1:0   16@1:1   17@1:2   18@1:3   19@1:4   20@0:0
   21@0:1   22@0:2   23@0:3   24@0:4   25@1:0   26@1:1   27@1:2
   28@1:3   29@1:4   30@0:0   31@0:1   32@0:2   33@0:3   34@0:4
   35@1:0   36@1:1   37@1:2   38@1:3   39@1:4   40@0:0   41@0:1

array2d (derefij):
    0@0:0    1@0:1    2@0:2    3@0:3    4@0:4    5@1:0    6@1:1
    0@0:0    1@0:1    2@0:2    3@0:3    4@0:4    5@1:0    6@1:1
    0@0:0    1@0:1    2@0:2    3@0:3    4@0:4    5@1:0    6@1:1
    0@0:0    1@0:1    2@0:2    3@0:3    4@0:4    5@1:0    6@1:1
    0@0:0    1@0:1    2@0:2    3@0:3    4@0:4    5@1:0    6@1:1
    0@0:0    1@0:1    2@0:2    3@0:3    4@0:4    5@1:0    6@1:1

array2dptr:
    0@0:0    1@0:1    2@0:2    3@0:3    4@0:4    5@1:0    6@1:1
   12@0:0   13@0:1   14@0:2   20@0:3   21@0:4   17@1:0   18@1:1
   24@0:0   30@0:1   31@0:2   32@0:3   33@0:4   29@1:0   35@1:1
   41@0:0    0@0:1    0@0:2    0@0:3    0@0:4    0@1:0    0@1:1
    0@0:0    0@0:1    0@0:2    0@0:3    0@0:4    0@1:0    0@1:1
    0@0:0    0@0:1    0@0:2    0@0:3    0@0:4    0@1:0    0@1:1

Thread 0 failed 3 tests!
Thread 1 failed 3 tests!

For each indexing method, thread 0 prints each element in the form {value}@{thread}:{phase}.
 If things work correctly, each method should generate the same output.  That is clearly
not the case here.  With the latest development build of CCE however, we do get the
expected results.

array2d:
    0@0:0    1@0:1    2@0:2    3@0:3    4@0:4    5@1:0    6@1:1
    7@1:2    8@1:3    9@1:4   10@0:0   11@0:1   12@0:2   13@0:3
   14@0:4   15@1:0   16@1:1   17@1:2   18@1:3   19@1:4   20@0:0
   21@0:1   22@0:2   23@0:3   24@0:4   25@1:0   26@1:1   27@1:2
   28@1:3   29@1:4   30@0:0   31@0:1   32@0:2   33@0:3   34@0:4
   35@1:0   36@1:1   37@1:2   38@1:3   39@1:4   40@0:0   41@0:1

array2d (derefi):
    0@0:0    1@0:1    2@0:2    3@0:3    4@0:4    5@1:0    6@1:1
    7@1:2    8@1:3    9@1:4   10@0:0   11@0:1   12@0:2   13@0:3
   14@0:4   15@1:0   16@1:1   17@1:2   18@1:3   19@1:4   20@0:0
   21@0:1   22@0:2   23@0:3   24@0:4   25@1:0   26@1:1   27@1:2
   28@1:3   29@1:4   30@0:0   31@0:1   32@0:2   33@0:3   34@0:4
   35@1:0   36@1:1   37@1:2   38@1:3   39@1:4   40@0:0   41@0:1

array2d (derefj):
    0@0:0    1@0:1    2@0:2    3@0:3    4@0:4    5@1:0    6@1:1
    7@1:2    8@1:3    9@1:4   10@0:0   11@0:1   12@0:2   13@0:3
   14@0:4   15@1:0   16@1:1   17@1:2   18@1:3   19@1:4   20@0:0
   21@0:1   22@0:2   23@0:3   24@0:4   25@1:0   26@1:1   27@1:2
   28@1:3   29@1:4   30@0:0   31@0:1   32@0:2   33@0:3   34@0:4
   35@1:0   36@1:1   37@1:2   38@1:3   39@1:4   40@0:0   41@0:1

array2d (derefij):
    0@0:0    1@0:1    2@0:2    3@0:3    4@0:4    5@1:0    6@1:1
    7@1:2    8@1:3    9@1:4   10@0:0   11@0:1   12@0:2   13@0:3
   14@0:4   15@1:0   16@1:1   17@1:2   18@1:3   19@1:4   20@0:0
   21@0:1   22@0:2   23@0:3   24@0:4   25@1:0   26@1:1   27@1:2
   28@1:3   29@1:4   30@0:0   31@0:1   32@0:2   33@0:3   34@0:4
   35@1:0   36@1:1   37@1:2   38@1:3   39@1:4   40@0:0   41@0:1

array2dptr:
    0@0:0    1@0:1    2@0:2    3@0:3    4@0:4    5@1:0    6@1:1
    7@1:2    8@1:3    9@1:4   10@0:0   11@0:1   12@0:2   13@0:3
   14@0:4   15@1:0   16@1:1   17@1:2   18@1:3   19@1:4   20@0:0
   21@0:1   22@0:2   23@0:3   24@0:4   25@1:0   26@1:1   27@1:2
   28@1:3   29@1:4   30@0:0   31@0:1   32@0:2   33@0:3   34@0:4
   35@1:0   36@1:1   37@1:2   38@1:3   39@1:4   40@0:0   41@0:1

Reported by [e-mail] on 2013-02-21 18:05:05

<hr> * Attachment: t_2darr.c

[Former user]

Steve,

Thanks for the test case.  I made the following small mod. to highlight the cells where
the actual value is not equal to the expected value.

$ diff -u t_2darr.c cray-2d-array-test.c 
--- t_2darr.c   2013-02-21 13:48:05.156551400 -0800
+++ cray-2d-array-test.c        2013-02-21 13:57:46.630987352 -0800
@@ -60,7 +60,8 @@
     shared [BLKS] VTYPE *ptr = &(expr); \
     VTYPE actual = *ptr; \
     if ( actual != EXPECTED(i,j) ) failed = 1; \
-    t0_print( "  " VFMT "@" TFMT ":" PFMT, \
+    t0_print( "  %s" VFMT "@" TFMT ":" PFMT, \
+              actual != EXPECTED(i,j) ? "*" : " ", \
               actual, \
               (TTYPE)upc_threadof( ptr ), \
               (PTYPE)upc_phaseof( ptr ) )

Currently, GUPC doesn't pass this test.  Here is the example output.

array2d:
     0@0:0     1@0:1     2@0:2     3@0:3     4@0:4     5@1:0     6@1:1
     7@1:0     8@1:1     9@1:2    10@1:3    11@1:4    12@0:0    13@0:1
    14@0:0    15@0:1  * 21@0:2  * 22@0:3  * 23@0:4    19@1:0    20@1:1
    21@0:0    22@0:1    23@0:2    24@0:3    25@0:4    26@1:0    27@1:1
    28@1:0    29@1:1  * 35@1:2  * 36@1:3  * 37@1:4    33@0:0    34@0:1
    35@1:0    36@1:1    37@1:2    38@1:3    39@1:4    40@0:0    41@0:1

Reported by gary.funck on 2013-02-21 22:00:59

[Former user]

> Currently, GUPC doesn't pass this test.  Here is the example output.

Attached is an updated version of the test that puts asterisks by invalid elements,
and additionally checks that the addrfield, phase, and thread are valid as well.  I
see in GUPC's output, the thread and phase (and likely addrfield) are not correct,
even where it gets the correct value.

Reported by [e-mail] on 2013-02-21 22:45:00

<hr> * Attachment: t_2darr.c

[Former user]

Steve,

Thanks for the improved checking.

Unfortunately, the new code runs into GUPC's current errant shared address calculation.

array2d:
t_2darr: at t_2darr_v2.c:93, UPC error: Invalid conversion of shared address to local
pointer;
thread does not have affinity to shared address
thread 1 terminated with signal: 'Aborted'
Terminated


The offending code is here:

    91      shared [BLKS] VTYPE *p_first = &array2d[p_first_i][p_first_j];
    92
    93      size_t local_diff = (((VTYPE *)ptr) - ((VTYPE *)p_first)) *
    94                          sizeof( VTYPE );

Perhaps something like the following is good enough?

size_t local_diff = (upc_threadof(ptr) == MYTHREAE && upc_threadof(pfirst) == MYTHREAD)
            ? (((VTYPE *)ptr) - ((VTYPE *)p_first))+ sizeof( VTYPE )
            : (ROWS * COLS) ;

The idea above is to set local_diff to something outside of the normal computation.
Alternatively, local_diff might be made an 'int' and the calculation above could then
return -1.

Reported by gary.funck on 2013-02-21 23:19:35

[Former user]

I was just going to change the if checks to only compute and check local_diff if it
is on the same thread.  Thus:

    if ( addr_diff != exp_diff ) {
        failed = 1;
    }
    else if ( upc_threadof( ptr ) == MYTHREAD ) {
        size_t local_diff = (((VTYPE *)ptr) - ((VTYPE *)p_first)) *
                            sizeof( VTYPE );
        if ( local_diff != exp_diff ) {
            failed = 1;
        }
    }

Reported by [e-mail] on 2013-02-21 23:24:49

[Former user]

Removed illegal pointer-to-local cast from check_addrfield() using change from comment
69.

Reported by [e-mail] on 2013-02-21 23:31:20

<hr> * Attachment: t_2darr.c

[Former user]

OK, the fix in Comment 70 avoids the runtime check, thanks.

$ t_2darr -n2
array2d:
    0@0:0    1@0:1    2@0:2    3@0:3    4@0:4    5@1:0    6@1:1
**  7@1:0**  8@1:1**  9@1:2** 10@1:3** 11@1:4** 12@0:0** 13@0:1
** 14@0:0** 15@0:1** 21@0:2** 22@0:3** 23@0:4** 19@1:0** 20@1:1
** 21@0:0** 22@0:1** 23@0:2** 24@0:3** 25@0:4** 26@1:0** 27@1:1
** 28@1:0** 29@1:1** 35@1:2** 36@1:3** 37@1:4** 33@0:0** 34@0:1
   35@1:0   36@1:1   37@1:2   38@1:3   39@1:4   40@0:0   41@0:1
[...]

Reported by gary.funck on 2013-02-21 23:56:23

[Former user]

In order to make this test fit better with the conventions used within the Berkeley
test harness, I'd suggest something along the lines of the following.

diff -u t_2darr_v{3,4}.c
--- t_2darr_v3.c        2013-02-21 15:52:19.900938400 -0800
+++ t_2darr_v4.c        2013-02-21 16:15:58.442793237 -0800
@@ -233,6 +233,7 @@
 main()
 {
     int failed = 0;
+    static shared int fail_count[THREADS];

     if ( THREADS > MAX_THREADS ) {
         t0_print( "Please run with %d or fewer threads.\n",
@@ -247,9 +248,21 @@
     failed += print_array2d_derefj();
     failed += print_array2d_derefij();
     failed += print_array2dptr();
+    fail_count[MYTHREAD] = failed;

-    if ( failed != 0 ) {
-        printf("Thread %d failed %d tests!\n",MYTHREAD,failed);
+    upc_barrier;
+
+    if (MYTHREAD == 0) {
+      int passed = 1;
+      for (int t = 0; t < THREADS; ++t) {
+        if (fail_count[t] > 0) {
+          passed = 0;
+          printf("Thread %d failed %d tests!\n",t,fail_count[t]);
+        }
+      }
+      printf("2D array test %s\n",
+             passed ? "passed" : "failed");
+      if (!passed) upc_global_exit(1);
     }

     return 0;

This changes causes a single pass/fail line to be printed if any of the threads failed
and issues a non-zero exit if a failure is detected.  (I realize that exit codes have
been deemed unreliable.  That is one reason that many of the harness tests issue an
affirmative diagnostic.)

Reported by gary.funck on 2013-02-22 00:21:05

[Former user]

Steve - Can you verify the 17-line program in comment #52 should be fully permitted
by the rules we're specifying? Does the updated Cray compiler accept that program?

Here are my comments on the latest proposal:

+ For array types, recursively, the
+ ultimate element type of the element type of the array type.

Possible rewording:
"For an array type ``array of T'', the ultimate element type of T."

 \ssterm{shared array}%
- an array with elements that have shared qualified type.
+ an array \xreplaced[id=DB]{3}{whose ultimate element type is
+ shared-qualified}{with elements that have shared qualified type}.

Given the new definitions above, wouldn't it be equivalent and cleaner to replace this
entire definition with "an array with shared type"?

 \np The {\tt upc\_elemsizeof} operator returns the size, in bytes,
- of the highest-level (leftmost) type that is not an array. For
+ of the \xreplaced[id=DB]{3}{ultimate element type of an expression
+ with shared type}{highest-level (leftmost)
+ type that is not an array. For
non-array objects, {\tt upc\_elemsizeof} returns the same value as
- sizeof. The result of {\tt upc\_elemsizeof} is an integer constant.
+ sizeof}. The result of {\tt upc\_elemsizeof} is an integer constant.

The proposed semantic description is currently biased toward a /unary-expression/ operand,
and leaves the /type-name/ operand case somewhat underspecified. It also removes the
(redundant but handy) clarification that despite its misleading name, this operator
also accepts non-array operands.
How about this replacement text:

  The {\tt upc\_elemsizeof} operator returns the size, in bytes, of the ultimate element
type of its operand.
  For non-array operands, {\tt upc\_elemsizeof} returns the same value as sizeof.

 \npf A shared type qualifier shall not appear in a type cast
where the corresponding pointer component of the type of the expression
- being cast is not shared-qualified.\footnote{i.e., pointers-to-local
+ being cast is not \xreplaced[id=DB]{3}{a shared type}{shared-qualified}.
+ \footnote{i.e., pointers-to-local
cannot be cast to pointers-to-shared.} 

Might also want to remove "qualifier" from the first sentence. This has the beneficial
effect of clarifying the restriction still applies when the shared qualifier is introduced
via typedefs.

+\xadded[id=DB]{3}{
+\np A non-null pointer-to-shared whose reference type is an array type points
+ to the first object of the ultimate element type of the referenced array
+ type in the pointed-to shared array object.
+}

I believe this description is correct, but it is difficult to parse and doesn't really
convey the restriction we're trying to impose. It also seems redundant with this later
paragraph:

+\xadded[id=DB]{3}{
+\np When the unary {\tt \&} is applied to an expression with a shared
+ array type, the result is a pointer-to-shared that points to the
+ beginning of the pointed-to shared array, whose referenced type
+ matches that of the expression the unary {\tt \&} was applied to.
+}
+

We also have a very closely-related existing paragraph in 6.5.2.1: "For any shared
array, a, upc_phaseof (&a) is zero."

Could we perhaps merge all three into a single, clear statement? Something like:

  \np When the unary {\tt \&} operator is applied to an expression with shared
      array type, the result is a pointer-to-shared with zero phase that points to
the
      address of the first byte of the initial array element. The referenced type of
the 
      pointer matches the operand type.

I think that captures everything we are trying to specify? Ie:
  - &a is a pointer-to-shared
  - upc_phaseof(&a) is zero
  - upc_threadof(&a) is equal to upc_threadof(&a[0]) (and also upc_threadof(a))
  - &a actually points to the beginning of the initial *element* (and not eg to an
array descriptor, which don't exist in UPC). This specifically implies you can cast
&a directly to an element type and use it to access an element
The last sentence is completely redundant with C99 6.5.3.2-3 and could optionally be
removed, but might be worth keeping for clarity.

- T *P1, *P2;
- shared T *S1, *S2;
+ LT *P1, *P2;
+ shared ST *S1, *S2; /* LT is the ultimate element type of ST */

- P1 = (T*) S1; /* allowed if S1 has affinity to MYTHREAD */
- P2 = (T*) S2; /* allowed if S2 has affinity to MYTHREAD */
+ P1 = (LT*) S1; /* allowed if S1 has affinity to MYTHREAD */
+ P2 = (LT*) S2; /* allowed if S2 has affinity to MYTHREAD */
...
\item S1 and P1 shall point to the same object.
\item S2 and P2 shall point to the same object. 
 \item The expression (({\tt (ptrdiff\_t) upc\_addrfield} (S2) - {\tt (ptrdiff\_t)
upc\_addrfield(S1))} shall
- evaluate to the same value as ((P2 - P1) * sizeof(T)).
+ evaluate to the same value as ((P2 - P1) * sizeof(\xreplaced[id=DB]{3}{LT}{T})).


The replacement text is a bit problematic when ST and LT are permitted to differ (eg
consider ST=(int[10]) and LT=(int))
In that case the expression: (LT*) S1  is at least "fishy", since it casts (shared
int (*)[10]) directly to (int *), which technically has an incompatible referenced
type. I believe this is still permitted by C99 (although the alignment restrictions
in 6.3.2.3 might technically be problematic?), but utilizing that corner case for one
of the axioms that defines our pointer arithmetic seems a bit "sketchy". More importantly,
*P1 and *S1 no longer have the same unqualified type, so it is incorrect to say that
"S1 and P1 shall point to the same object", as one might actually point to a sub-object
of the other.

Is it necessary to change the variable definitions at all? Couldn't we just leave them
unchanged and simply change the final expression to:
((P2 - P1) * upc_elemsizeof(*S1))

+\np \xreplaced[id=DB]{3}
+ {Objects of the ultimate element type of the array are distributed
+ in a round robin fashion, by chunks of block-size objects, such that
+ the i-th object}

This is the first paragraph in a new section, and refers to "the array", a seemingly
prior entity which is not defined. Further, the proposed change removes the qualification
that this paragraph applies to (and only to) shared arrays. Perhaps add an introductory
sentence: "The elements of a shared array are laid out as follows."

Also while we're changing that paragraph, I suggest we change the equation:
   (floor (i/block_size) mod THREADS)
to use the mathematical \lfloor\rfloor operator defined by C99 3.19, ie:
   ($\lfloor i/block\_size \rfloor$ {\tt mod THREADS})
The latter is a bit more precise since the floor() function(s) in math.h return a floating-point
value, which is not relevant here.

Editorial stuff:
----------------
> \xreplaced[id=DB]
We're not currently displaying change authors, but please use your own initials in
the annotation id ;-)
You might need to add your name to common/upc-spec-preamble.tex.

- {\tt nbytes} argument shall be {\tt b * sizeof (t)}. If the block
- size is indefinite, {\tt nbytes} shall be 0.
+ {\tt nbytes} argument shall be {\tt b * \xreplaced[id=DB]{3}
+ {upc\_elemsizeof (d)}{sizeof (t)}. If the block size is indefinite,
+ {\tt nbytes} shall be 0.

Missing a closing } for the \tt that breaks the subsequent text font.

+    number of consecutive \xreplaced[id=DB]{3}{objects of the ulitmate

Spelling error.

- THREADS} environment and an array with shared-qualified elements
+ THREADS} environment and \xreplaced[id=DB]{3}{a shared array}{an array
+ with shared-qualified elements}

This change touches wording already being changed by PendingApproval issue 94/95, and
will need to be merged "carefully".

Reported by danbonachea on 2013-02-25 05:23:17

[Former user]

> Steve - Can you verify the 17-line program in comment #52 should be fully permitted
by the rules we're specifying? Does the updated Cray compiler accept that program?

It should be.  I'll verify that our compiler accepts it tomorrow.

> Possible rewording:
> "For an array type ``array of T'', the ultimate element type of T."

That's much better!  I'll change it tomorrow.

> Given the new definitions above, wouldn't it be equivalent and cleaner to replace
this entire definition with "an array with shared type"?

Probably.

> How about this replacement text:
>  The {\tt upc\_elemsizeof} operator returns the size, in bytes, of the ultimate element
type of its operand.
>  For non-array operands, {\tt upc\_elemsizeof} returns the same value as sizeof.

I assume that should be {\tt sizeof} at the end, but yes, that would work.

> ...
> Could we perhaps merge all three into a single, clear statement? Something like:
>   
>  \np When the unary {\tt \&} operator is applied to an expression with shared
>      array type, the result is a pointer-to-shared with zero phase that points to
the
>      address of the first byte of the initial array element. The referenced type
of the 
>      pointer matches the operand type.

The result is NOT guaranteed to have zero phase.  6.5.2.1 applies to taking the address
of a variable whose type is a shared array, not expressions with whose type is a shared
array type in general.  If we always force the resulting pointer-to-shared to have
zero phase, then pointer arithmetic (and thus array indexing!) does not work for multi-dimensional
shared arrays.  See the (updated) test program in comment 70 for an example of why
this is necessary.

> The replacement text is a bit problematic when ST and LT are permitted to differ
(eg consider ST=(int[10]) and LT=(int))
> In that case the expression: (LT*) S1  is at least "fishy", since it casts (shared
int (*)[10]) directly to (int *), which technically has an incompatible referenced
type. I believe this is still permitted by C99 (although the alignment restrictions
in 6.3.2.3 might technically be problematic?), but utilizing that corner case for one
of the axioms that defines our pointer arithmetic seems a bit "sketchy". More importantly,
*P1 and *S1 no longer have the same unqualified type, so it is incorrect to say that
"S1 and P1 shall point to the same object", as one might actually point to a sub-object
of the other.

No, casting (shared int (*)[10]) to (int (*)[10]) is exactly what we MUST forbid! 
Because of the block cyclic distribution, there may not be 10 ints on the local thread
to point at!  The ONLY thing that you can safely locally cast to is a pointer to the
ultimate element type, and therefore ST and LT MUST differ if ST is an array type.
 Keep in mind that *S1 could span multiple threads, while *P1 is limited to the (initial)
portion of *S1 that is local to the casting thread.

> Is it necessary to change the variable definitions at all? Couldn't we just leave
them unchanged and simply change the final expression to:
> ((P2 - P1) * upc_elemsizeof(*S1))

I think your replacement equation works, but, as explained above, we do need to change
the variable definitions.

> This is the first paragraph in a new section, and refers to "the array", a seemingly
prior entity which is not defined. Further, the proposed change removes the qualification
that this paragraph applies to (and only to) shared arrays. Perhaps add an introductory
sentence: "The elements of a shared array are laid out as follows."

Yes, I see I cut out too much from the existing wording.  I'll put the introductory
stuff back in. 

> Also while we're changing that paragraph, I suggest we change the equation:
> ...

Sounds good to me.

> We're not currently displaying change authors, but please use your own initials in
> the annotation id ;-)
> You might need to add your name to common/upc-spec-preamble.tex.

So that's what that field is for.  I just blindly copied it from the first other use
of \xreplaced I saw.

Reported by [e-mail] on 2013-02-25 07:23:51

[Former user]

> No, casting (shared int (*)[10]) to (int (*)[10]) is exactly what we MUST forbid!


I'm aware of that, and I agree the cast you mention needs to be prohibited, but that
restriction is already specified elsewhere (6.4.3-1). I think you misread the second
type in my statement "it casts (shared int (*)[10]) directly to (int *)". 

I still believe changing the final expression is sufficient to define this aspect of
pointer arithmetic, without the need to change the variable definitions, as that introduces
the other problems I mentioned. In particular, when the C99 spec says a pointer "points
to an object", the type of the object in question always matches the referenced type
of the pointer. Therefore with the proposed definition changes, the statement "S1 and
P1 shall point to the same object" is a type error when *P1 is a non-array type and
*S1 is an array type (they point to the "same address in memory", not to be confused
with "the same object").

> The result is NOT guaranteed to have zero phase.  6.5.2.1 applies to taking the address
of a variable whose 
> type is a shared array, not expressions with whose type is a shared array type in
general.

OK, I now see your reasoning for the distinction between the two closely-related paragraphs
in your proposal, and I agree that a PTS to a sub-array in the middle of an object
might have non-zero phase. However I'd still like to see if we can somehow improve
the wording to make it more approachable. 

In particular, I dislike the statement "A non-null PTS of type X always points to Y",
because this may be false in a program that uses prior casts to point their PTS at
garbage. I think it's generally preferable to describe the semantics of a language
construct directly (ie the result of &-operator and PTS arithmetic on shared array
types), instead of indirectly as a property of all programs using the construct in
the way we expect. Simply changing "A non-null PTS.." to "A valid, non-null PTS" would
be a step in the right direction.

As written the first paragraph also suffers from the subtle type error mentioned above,
ie a "pointer-to-shared whose reference type is an array type points.." to an ARRAY,
not to "an object of the ultimate element type". The second proposed paragraph avoids
this problem by stating it points to the BEGINNING of an array object, so perhaps we
can apply that wording here as well? This would leave us with something like:

  A valid, non-null pointer-to-shared whose reference type is an array type points
to
  the beginning of the pointed-to shared array object.

However this is starting to sound suspiciously like "a pointer points to what it points
to". 
What restriction are we really trying to specify here? Can you describe an implementation
scenario we are trying to prohibit with this paragraph?

Reported by danbonachea on 2013-02-25 13:10:15

[Former user]

> I still believe changing the final expression is sufficient to define this aspect
of pointer arithmetic, without the need to change the variable definitions, as that
introduces the other problems I mentioned. In particular, when the C99 spec says a
pointer "points to an object", the type of the object in question always matches the
referenced type of the pointer. Therefore with the proposed definition changes, the
statement "S1 and P1 shall point to the same object" is a type error when *P1 is a
non-array type and *S1 is an array type (they point to the "same address in memory",
not to be confused with "the same object").

Changing the final expression should be fine, if you think that's more clear.  As far
as the "points to the same object" question, what if we instead clearly distinguish
the array and non-array cases:

  If ST is not an array type, then LT is equivalent to ST, and S1 and P1 shall point
to the same object.  IF ST is an array type, then P1 shall point to the first object
(with ultimate element type of ST) in the shared array object that S1 points to.

> In that case the expression: (LT*) S1  is at least "fishy", since it casts (shared
int (*)[10]) directly to (int *), which technically has an incompatible referenced
type.

Yes, but there does not exist a pointer-to-local with a compatible referenced type
if ST is an array type, so (int *) is the best we can do.  The fundamental problem
here is that, unlike objects of all other complete types in UPC, objects of shared
array types are permitted to span threads.

> I believe this is still permitted by C99 (although the alignment restrictions in
6.3.2.3 might technically be problematic?), but utilizing that corner case for one
of the axioms that defines our pointer arithmetic seems a bit "sketchy".

Since a pointer to an array has the same address as the first element of that array,
explicitly casting down from a pointer to the array type to a pointer to the element
type is always safe and well-defined.

As to using a "corner case" of C99 to define pointer-to-shared arithmetic being "sketchy",
feel free to offer a replacement.

Reported by [e-mail] on 2013-02-25 15:08:43

[Former user]

I suppose another way to deal with this situation would be to make casting to a pointer-to-local
have defined behavior only if the ENTIRE object that the pointer-to-shared points to
has affinity to the local thread.  Then we can simply use the existing wording for
paragraphs 4 and 5.  Would that be better?

Reported by [e-mail] on 2013-02-25 15:59:59

[Former user]

Attached is an updated draft that takes Dan's recent comments into account.  Notable
changes:

1. Fixed "Editorial stuff" from comment 73.

2. Reworded definitions of "ultimate element type" and "shared array" based on  comment
73.

3. Reworded semantics of upc_elemsizeof based on comment 73.

4. As proposed in comment 77, made casting a pointer-to-shared that points to a shared
array object that spans threads to a pointer-to-local result in undefined behavior.
 This greatly simplifies a lot of the contentious language discussed in comments 73-76.

Reported by [e-mail] on 2013-02-25 16:55:03

<hr> * Attachment: upc-lang-spec-draft.pdf * Attachment: upc-terms-and-defs.dif * Attachment: upc-spec-preamble.dif * Attachment: upc-lib-core.dif * Attachment: upc-language.dif

[Former user]

> I suppose another way to deal with this situation would be to make casting to a 
> pointer-to-local have defined behavior only if the ENTIRE object that the pointer-
> to-shared points to has affinity to the local thread.  Then we can simply use the

> existing wording for paragraphs 4 and 5.  Would that be better?

At first glance this seems like a reasonable resolution to me - it certainly simplifies
the spec. Would it prohibit any important usage cases?

One potential drawback is this type of user error probably can't be statically diagnosed
(although that's OK as far as compliance since it falls under undefined behavior).
It's also "harder" for a high-quality implementation to generate a runtime warning
for this case, in contrast to the "cast a pointer with remote affinity to local" which
can be easily caught at runtime, with no type information. However since it only applies
to pointer-to-shared-array type (an "advanced" user feature) then perhaps that's not
a concern.

 \np The {\tt upc\_threadof} function returns the index of the
- thread that has affinity to the shared object pointed to by {\tt ptr}.\footnote{%
+ thread that has affinity to the shared object pointed to by {\tt ptr}.
+ \xadded[id=SV]{3}{If ptr points to a shared array object, then the
+ index of the thread that has affinity to the first element of data
+ storage that comprises the shared object is returned.}\footnote{%

I understand the purpose of this new wording is to handle "oddball" cases where the
referenced object may have multiple affinity, like:
  shared [5] int A[10][10];
  upc_threadof(&A);
  upc_threadof(&A[1]);
However I'm concerned that a casual reader might misread the new sentence to imply
the function always returns the affinity of the start of the array when passed a pointer
to a shared array element:
  upc_threadof(&A[1][0])
Can we somehow clarify this new sentence is only concerned with the "oddball" case?

Minor nitpick: 
In the constraint sections for upc_*sizeof(), can we use the singular tense instead
of the plural when discussing the operand?
Ie change:
   The upc_elemsizeof operator shall apply only to shared types or expressions with
shared type.
to:
   The upc_elemsizeof operator shall apply only to a shared type or an expression with
shared type.
I know this style was inherited from 1.2, but after reading it ten times I've realized
that referring to a unary operand using a plural tense is unnecessarily confusing.
The singular tense is also more stylistically consistent with similar phrases in C99.

Similarly in upc_elemsizeof semantics:
"For non-array operands" -> "For a non-array operand"

Reported by danbonachea on 2013-02-25 19:53:03

[Former user]

> Does the updated Cray compiler accept that program?

Yes, it does.

Reported by [e-mail] on 2013-02-26 18:34:58

[Former user]

> At first glance this seems like a reasonable resolution to me - it certainly simplifies
the spec. Would it prohibit any important usage cases?

Not that I can think of.  Any case that it prohibits can be trivially worked around
by using a pointer to the ultimate element type rather than a pointer to the array
type.

> One potential drawback is this type of user error probably can't be statically diagnosed
(although that's OK as far as compliance since it falls under undefined behavior).
It's also "harder" for a high-quality implementation to generate a runtime warning
for this case, in contrast to the "cast a pointer with remote affinity to local" which
can be easily caught at runtime, with no type information. However since it only applies
to pointer-to-shared-array type (an "advanced" user feature) then perhaps that's not
a concern.

True, but generating a run-time check that covers most cases is really easy:

Given a pointer-to-local 'LPTR' with type 'T1 *' and a pointer-to-shared 'SPTR' with
type 'shared [BS] T2 *', the run-time check merely needs to confirm that 'sizeof(T1)
<= (BS - upc_phaseof(SPTR)) * sizeof(T2)'.

The only case this doesn't catch is where SPTR points to a partially full final block
of a shared array object, and the cast would require a "more full" block.

Reported by [e-mail] on 2013-02-26 19:32:11

[Former user]

Attached is another update of the proposed draft.  I think we're really close with this
one.  Changes from that in comment 78:

1. Removed inserted text from upc_threadof().  Instead, modified the definition of
the term 'affinity' (section 3.5) to define that shared objects have affinity to exactly
one thread, that of the element of data storage containing the beginning of the shared
object.  A footnote clarifies that for shared array objects, the affinity of the object
itself may not be the same as that of all the elements of data storage containing it.

2. Small language changes based on Dan's "nitpicks" in comment 79.

Reported by [e-mail] on 2013-02-26 20:14:34

<hr> * Attachment: upc-lang-spec-draft.pdf * Attachment: upc-terms-and-defs.dif * Attachment: upc-spec-preamble.dif * Attachment: upc-lib-core.dif * Attachment: upc-language.dif

[Former user]

New diff is looking very good! Minor comments:

+ {Objects with ultimate element type making up shared arrays are
+ distributed in a round robin fashion, by chunks of block-size objects,
+ such that the i-th object}

Suggested rewording:
The objects with ultimate element type that comprise a shared array are ...

 \sterm{affinity}%
logical association between shared
objects and threads. Each element of data storage that contains
shared objects has affinity to exactly one thread.
+ \xadded[id=SV]{3}{All shared objects also have affinity to exactly one
+ thread, which is the same as that of the element of data storage
+ containing the beginning of the shared object.\footnote{For non-array
+ shared objects, all elements of data storage containing the object
+ have the same affinity as the object itself. This is not necessarily
+ true for shared array objects, which may span multiple threads.}}

I like the new text. I'm curious if it's also clear to readers who've spent fewer hours
thinking about these issues?

Latex issue: you'll need to change \footnote to \truefootnote to ensure it's correctly
rendered by the changes package (yes it's a filthy hack -- don't ask).

I also noticed there are two "spurious" changes:

- shared T *S1, *S2;
+ shared T *S1, *S2;

-\item The expression (({\tt (ptrdiff\_t) upc\_addrfield} (S2) - {\tt (ptrdiff\_t)
upc\_addrfield(S1))} shall
+\item The expression ({\tt (ptrdiff\_t) upc\_addrfield} (S2) - {\tt (ptrdiff\_t) upc\_addrfield}
(S1)) shall 

The first is just whitespace. The second one breaks the Latex margins and fonts for
the equation.

Reported by danbonachea on 2013-02-26 20:52:32

[Former user]

> The first is just whitespace. The second one breaks the Latex margins and fonts for
the equation.

I changed the second one for consistency--'(S2)' was not in the {\tt ...} block while
'(S1)' was.  Either both should be, or both shouldn't be.

Reported by [e-mail] on 2013-02-26 21:12:15

[Former user]

> I changed the second one for consistency--'(S2)' was not in the {\tt ...} block while
'(S1)' was.  Either both should be, or both shouldn't be.

If we're changing it for cosmetic reasons, please make sure the typeset result looks
reasonable :)  

Given the length of that equation it might be simpler to place it on a line by itself
and put the whole thing in {\tt}. The local equation should probably also be in {\tt}

While you're at it, this changed line:

+                         + (upc_phaseof(p) + elem_delta) div B) mod THREADS 

now also overflows into the right margin when typeset, and should probably be "scooted"
left.

Reported by danbonachea on 2013-02-26 21:26:28

[Former user]

Attached is another update of the proposed draft.  Changes from that in comment 82:

1. Reworded description of data distribution for definitely types shared arrays based
on comment 83.

2. Removed spurious whitespace changes from upc-language.tex.

3. Fixed (?) formatting issues noted in comment 85.

Reported by [e-mail] on 2013-02-26 21:59:29

<hr> * Attachment: upc-lang-spec-draft.pdf * Attachment: upc-terms-and-defs.dif * Attachment: upc-spec-preamble.dif * Attachment: upc-lib-core.dif * Attachment: upc-language.dif

[Former user]

s/definitely types/definitely blocked/

Same typo in two bugs now today.  I need more caffeine! 

Reported by [e-mail] on 2013-02-26 22:00:40

[Former user]

Attached is another update of the proposed draft with minor typographical changes from
that in comment 86:

1. Changed "objects of the ultimate element type" to "objects with ultimate element
type" in 6.5.1.1 11 to match similar phrasing elsewhere in the document.

2. Corrected type "bjects" to "objects" in 6.5.2.1 5.

Reported by [e-mail] on 2013-02-26 22:39:40

<hr> * Attachment: upc-lang-spec-draft.pdf * Attachment: upc-language.dif

[Former user]

+\xchangenote[id=SV]{3}{Changed formatting of last item.}

We've been following the convention of only annotating changes that affect actual wording
or technical content. Typesetting fixes need not be annotated.


Your latest diff retracts this change:

- P1 = (T*) S1; /* allowed if S1 has affinity to MYTHREAD */
- P2 = (T*) S2; /* allowed if S2 has affinity to MYTHREAD */
+ P1 = (T*) S1; /* allowed if *S1 has affinity only to MYTHREAD */
+ P2 = (T*) S2; /* allowed if *S2 has affinity only to MYTHREAD */ 

The new definition of affinity obviates the need for the word "only".
However I believe we still want the S1->*S1 and S2->*S2 change (the old text is "loose"
and technically incorrect - we're referring to the affinity of the array element, not
the pointer object).

Reported by danbonachea on 2013-02-26 22:46:25

[Former user]

Changes from comment 88:

1. Updated 6.4.3 6 to reflect that a shared object that span multiple threads cannot
be completely accessed by a local pointer on the thread which it has affinity to.

2. Removed change note for typesetting changes as noted in comment 89.

3. Updated comments in 6.4.2 4 to be a bit more precise.

Reported by [e-mail] on 2013-02-27 00:18:08

<hr> * Attachment: upc-lang-spec-draft.pdf * Attachment: upc-language.dif

[Former user]

Overall looks very good.  One general question occurred to me:

We repeatedly use the phrase "element of data storage" - mostly in the sections changed
by this proposal. This phrase is somewhat cumbersome, and unfortunately overloads the
word "element", which we also use to discuss array elements (a closely related but
distinct concept). 

C99 does not use this phrase at all, but it does use "unit of data storage", eg:

3.6 byte
addressable unit of data storage large enough to hold any member of the basic character
set of the execution environment

The phrase "data storage" appears in the exactly 3 places in C99 - the definitions
of bit, byte and object. The entire subsequent document then uses one of those three
defined terms, as appropriate.

I believe all five occurrences of the phrase "element of data storage" in the current
proposal actually mean "byte". I think it might improve clarity to simply say "byte".
Thoughts?

Reported by danbonachea on 2013-02-27 13:07:33

[Former user]

> I believe all five occurrences of the phrase "element of data storage" in the current
proposal actually mean "byte". I think it might improve clarity to simply say "byte".
Thoughts?

I don't have any objection to that.  I tried to follow the existing practice from the
1.2 spec, but "element of data storage" is definitely a bit obtuse.

Reported by [e-mail] on 2013-02-27 13:59:24

[Former user]

Do I need to be concerned with the following errors:

$ make
...
------------------------------------
latex-mk:   LaTeX references have changed, another latex run is needed
------------------------------------
------------------------------------
latex-mk:   Failed to get LaTeX to converge after 5 tries
latex-mk:   Please fix the document or try again if you think it
latex-mk:   should be ok
------------------------------------
make[1]: *** [upc-lang-spec-draft.pdf] Error 1
make[1]: Leaving directory `/ptmp/sdvormwa/upc-specification-read-only/lang'
make: *** [make-default] Error 2

They seem to go away if I simply run make a second time:

$ make
...
LaTeX-Mk: Checking for ./upc-lib-nb-mem-ops-spec-draft.toc ...
make[3]: Leaving directory `/ptmp/sdvormwa/upc-specification-read-only/lib/proposed/nb-mem-ops'
make[2]: Leaving directory `/ptmp/sdvormwa/upc-specification-read-only/lib/proposed'
make[1]: Leaving directory `/ptmp/sdvormwa/upc-specification-read-only/lib'
$

Reported by [e-mail] on 2013-02-27 17:01:25

[Former user]

> Do I need to be concerned with the following errors:

Probably harmless. The changes package output takes a long time to converge, especially
now that it's getting so large. I usually 'make clean' when I see this error to ensure
I'm starting with a clean slate.

Reported by danbonachea on 2013-02-27 17:06:55

[Former user]

Changes from comment 90:

1. Replace uses of "element[s] of data storage" with "byte[s]".

2. Slightly reword definition of "affinity" to (hopefully) be a bit less awkward.

Reported by [e-mail] on 2013-02-27 17:18:55

<hr> * Attachment: upc-lang-spec-draft.pdf * Attachment: upc-terms-and-defs.dif * Attachment: upc-language.dif

[Former user]

Dan, does the following change (included in attachments in comments 90 and 95) address
your concern in comment 89, or would you prefer to revert to the 'allowed if *S1 has
affinity to MYTHREAD' wording?

- P1 = (T*) S1; /* allowed if S1 has affinity to MYTHREAD */
- P2 = (T*) S2; /* allowed if S2 has affinity to MYTHREAD */
+ P1 = (T*) S1; /* allowed if upc_threadof(S1) == MYTHREAD */
+ P2 = (T*) S2; /* allowed if upc_threadof(S2) == MYTHREAD */ 

Reported by [e-mail] on 2013-02-27 17:24:03

[Former user]

Attached is a version of the test from comment 70 that has been updated to fix incorrect
assumptions about the behavior of upc_addrfield() (see issues 106 and 107).

Reported by [e-mail] on 2013-02-27 18:00:14

[Former user]

Revamped test to remove upc_addrfield() altogether, and rely solely on results of pointer
arithmetic.

Reported by [e-mail] on 2013-02-27 18:43:31

<hr> * Attachment: t_2darr.c

[Former user]

> does the following change (included in attachments in comments 90 and 95) address
your concern in comment 89

Yes, the new threadof() comment looks like a good improvement.

I'm satisfied with the latest draft in comment #95. I think we should mail the official
PendingApproval change announcement and start the 4-week comment period. We can of
course continue to tweak based on feedback from the committee as it arises.

Reported by danbonachea on 2013-02-27 19:15:45

[Former user]

Official proposal mailed 2013-02-27.

Reported by [e-mail] on 2013-02-27 19:24:13 - Status changed: PendingApproval

<hr> * Attachment: upc-lang-spec-draft.pdf * Attachment: issue-3-unified.dif

[Former user]

FYI, we have confirmed that Clang UPC passes the shared array test cited in comment
#98.  See https://github.com/Intrepid/clang-upc/wiki for details on building Clang
UPC.

Reported by gary.funck on 2013-03-13 16:02:27

[Former user]

Steve: I'm working on applying the change proposal for issue 94/95, which has reached
the end of its comment period. However, there is a "merge conflict" between that proposal
and this issue 3 proposal, which both "touch" the same sentence in 6.5.2.1 for different
reasons. 

I'd like to avoid "nested" change annotations (which would probably break the changes
package horribly), and since issue 3 only touches this sentence in a trivial way (terminology
fixup), it makes more sense for the final annotation on this sentence to reference
issue 95 which actually makes a semantic change.

Below is the relevant paragraph with my proposed resolution to the merge conflict:

+    When a UPC program is translated in the {\em dynamic THREADS} environment,
+  \xreplaced[id=DB]{95}{%
+    every {\em declarator} or {\em type-name} containing 
+    an array type which is a shared type with definite blocksize 
+    shall include the THREADS keyword
+  }{%
+    and an array with shared-qualified elements
+    is declared with definite blocksize, the THREADS expression shall occur 
+  }%
+    exactly once in one dimension of the array declarator
+    (including through typedefs).  

This is carefully worded to work for both shared array declarations and typedefs of
arrays with shared type (which is the purpose of issue 95). Because of the latter case
we cannot simply say "shared array", because there may be no object involved. It would
be less wordy to say "shared array type", but that's not a term defined by issue 3
(only "shared array" and "shared type") and might be considered ambiguous, so I opted
for the more explicit wording.

Does this resolution look acceptable? 

Reported by danbonachea on 2013-03-17 01:08:21

[Former user]

> This is carefully worded to work for both shared array declarations and typedefs of
arrays with shared type (which is the purpose of issue 95). Because of the latter case
we cannot simply say "shared array", because there may be no object involved. It would
be less wordy to say "shared array type", but that's not a term defined by issue 3
(only "shared array" and "shared type") and might be considered ambiguous, so I opted
for the more explicit wording.
>
> Does this resolution look acceptable? 

Don't we need to restrict this to shared array object declarators, so we can construct
pointers to dimensions lower than the THREADS-scaled dimension?

Consider the following example:

 static shared int array[THREADS][5];
 shared int (*pts)[5];
 pts = &array[MYTHREAD];

Wouldn't the declaration of 'pts' be illegal with this wording?

Reported by [e-mail] on 2013-03-17 17:06:50

[Former user]

Moving this discussion to issue 95.

Reported by danbonachea on 2013-03-18 04:03:16

[Former user]

committed as SVN r215

Reported by danbonachea on 2013-04-30 20:27:42 - Status changed: Fixed

[Former user]

Ratified in the 5/22 telecon.

Reported by danbonachea on 2013-08-03 03:55:36 - Status changed: Ratified