Write section "Proposed Additions and Extensions"

[Former user]

Originally reported on Google Code with ID 54

The language appendix "Proposed Additions and Extensions" is outdated and needs to be
re-written to reflect our new process with the optional and required library specs.

Reported by danbonachea on 2012-06-17 20:19:12

[Former user]

``` Jim seems to be the most interested party for specifying the process of spec change, so I nominate him to draft an updated version of Appendix A for spec 1.3. This is a "critical" priority, because the current wording makes no sense under the new document split arrangement.

Notable points to be updated:

• Appendix A no longer contains libraries, nor is the core library in the language document subject to extension.
• Extension libraries first appear in the optional library document, subject to the initial set of criteria.
• After satisfying the secondary set of criteria they can be promoted from the optional to required document.
• Extension libraries define not only a feature macro but also a dedicated header file.
• Need a new section documenting our procedure for localized changes to any of the documents (from the Wiki).

```

Reported by `danbonachea` on 2012-08-17 00:08:38 - Labels added: Priority-Critical - Labels removed: Priority-Medium

[Former user]

``` This appendix is probably also the best place to cross-reference the required and optional library documents, whose existence is not currently mentioned anywhere in the main language spec document. ```

Reported by `danbonachea` on 2012-08-17 00:14:59

[Former user]

``` Set default Consensus to "Low". ```

Reported by `gary.funck` on 2012-08-19 23:26:19 - Labels added: Consensus-Low

[Former user]

``` Change Status to New: Requires review. ```

Reported by `gary.funck` on 2012-08-19 23:37:41 - Status changed: `New`

[Former user]

``` I guess this is what I get for speaking up. :) Sure, I can lead the effort to update Appendix A. ```

Reported by `james.dinan` on 2012-08-20 20:34:59

[Former user]

``` Bumping this as a reminder - we discussed this in the call and agreed it needs to be rewritten. Jim are you working on replacement text? ```

Reported by `danbonachea` on 2012-09-26 20:16:21

[Former user]

``` Thanks for the reminder. I'll bump this up my to-do list. ```

Reported by `james.dinan` on 2012-09-28 18:52:37

[Former user]

``` Jim has indicated he won't have time available to handle UPC matters for the next month, so barring any other volunteers I'll take over this issue. ```

Reported by `danbonachea` on 2012-10-05 08:55:47 - Status changed: `Accepted`

[Former user]

``` Miscommunication, I can still handle this ticket. ```

Reported by `james.dinan` on 2012-10-05 18:08:35

[Former user]

``` Here is the proposed change to this section. With this change, we would include language changes in Appendix A and library changes in the optional libraries section:

Index: upc-lang-extensions.tex
===================================================================
--- upc-lang-extensions.tex	(revision 148)
+++ upc-lang-extensions.tex	(working copy)
@@ -3,14 +3,17 @@
 \index{proposed extensions}
 
 \np This section contains proposed additions and extensions to the UPC
-    specification.  Such proposals are included when stable enough for
+    language specification.  An optional library specifications section
+    is included, which contains proposed additions and extension to the
+    UPC library specification.  Such proposals are included when stable enough for
     developers to implement and for users to study and experiment with
     them.  However, their presence does not suggest long term support.
     When fully stable and tested, they will be moved to the main body of
     the specification.
 
 \np This section also describes the process used to add new items to the
-    specification, which starts with inclusion in this section.  Requirements
+    specification, which starts with inclusion in this section or the optional
+    libraries section.  Requirements
     for inclusion are:\footnote{These requirements ensure that most of
     the semantic issues that arise during initial implementation have
     been addressed and prevents the accumulation of interfaces that no
@@ -36,7 +39,7 @@
     specification are:
 
 \begin{enumerate}
-\item Six months residence in this section.
+\item Six months residence in this section or the optional libraries section.
 
 \item The existence of either one (or more) publicly available "reference" implementation

 written in standard UPC OR at least two independent implementations (possibly specific

```

Reported by `james.dinan` on 2012-10-05 19:09:25 - Status changed: `Started`

[Former user]

``` This proposed change is missing several of the points raised in comments 1 and 2. ```

Reported by `danbonachea` on 2012-10-05 19:29:24

[Former user]

``` From comments #1-#2:

• Appendix A no longer contains libraries

Should be covered with the change in comment 10.

• Nor is the core library in the language document subject to extension.

Can you explain what you mean by this?

• Extension libraries first appear in the optional library document, subject to the initial set of criteria.

This is covered.

• After satisfying the secondary set of criteria they can be promoted from the optional to required document.

This is covered.

• Extension libraries define not only a feature macro but also a dedicated header file.

This needs to be added.

• Need a new section documenting our procedure for localized changes to any of the documents (from the Wiki).

Can you explain what you mean by this, or provide a link? From reading Appendix A, it seems like changes to the main body of the spec need to be included in Appendix A itself before they can be moved into the main body of the text. This is horrifying, but it seems to be what is stated by the existing text.

• This appendix is probably also the best place to cross-reference the required and optional library documents, whose existence is not currently mentioned anywhere in the main language spec document.

This is pretty much the worst place to present the separation of libraries from the main document and the split into optional and required chapters. This should be done at the beginning, rather than at the end of the document. ```

Reported by `james.dinan` on 2012-10-05 19:39:49

[Former user]

``` Here is an updated change, which includes the statement about header files:

--snip--

Index: lang/upc-lang-extensions.tex

--- lang/upc-lang-extensions.tex (revision 148) +++ lang/upc-lang-extensions.tex (working copy) @@ -3,14 +3,17 @@ \index{proposed extensions}

\np This section contains proposed additions and extensions to the UPC - specification. Such proposals are included when stable enough for + language specification. An optional library specifications section + is included, which contains proposed additions and extension to the + UPC library specification. Such proposals are included when stable enough for developers to implement and for users to study and experiment with them. However, their presence does not suggest long term support. When fully stable and tested, they will be moved to the main body of the specification.

\np This section also describes the process used to add new items to the - specification, which starts with inclusion in this section. Requirements + specification, which starts with inclusion in this section or the optional + libraries section. Requirements for inclusion are:\footnote{These requirements ensure that most of the semantic issues that arise during initial implementation have been addressed and prevents the accumulation of interfaces that no @@ -36,7 +39,7 @@ specification are:

\begin{enumerate} -\item Six months residence in this section. +\item Six months residence in this section or the optional libraries section.

\item The existence of either one (or more) publicly available "reference" implementation

written in standard UPC OR at least two independent implementations (possibly specific

@@ -56,3 +59,7 @@ to be the interface version of the extension if it is supported, otherwise undefined.

+\index{header files} +\np For each library extension, a separate header file whose name begins with + {\tt upc\_} must be specified. This header file must be provided by an + implementation if the extension is supported. ```

Reported by `james.dinan` on 2012-10-05 19:48:10

[Former user]

``` "This section contains proposed additions and extensions to the UPC specification. ... An optional library specifications section is included ... starts with inclusion in this section"

Originally Appendix A was intended to actually contain subsections of proposed/optional libraries. In 1.2 it contained the UPC-IO library. This is no longer the purpose of Appendix A - it is now purely a process documentation, not a container for actual specs. Actual specs are contained in the optional and required library documents.

" When fully stable and tested, they will be moved to the main body of the specification."

Our current lifecycle for libraries is proposed to optional to required, but it stops there. The "core" library in the language document is effectively "closed" to major new library additions, so that text is outdated.

"* Need a new section documenting our procedure for localized changes to any of the documents (from the Wiki). Can you explain what you mean by this, or provide a link? "

My point here is we should augment appendix A with the process we've worked out on the wiki for changes to the non-library sections of the language document.

"From reading Appendix A, it seems like changes to the main body of the spec need to be included in Appendix A itself before they can be moved into the main body of the text. This is horrifying, but it seems to be what is stated by the existing text."

Right - this is part of what needs to be updated :) Appendix A was originally intended only to define process for defining new library functionality. The process for language changes was not considered when writing it. ```

Reported by `danbonachea` on 2012-10-05 21:27:34

[Former user]

``` "* This appendix is probably also the best place to cross-reference the required and optional library documents, whose existence is not currently mentioned anywhere in the main language spec document.

This is pretty much the worst place to present the separation of libraries from the main document and the split into optional and required chapters. This should be done at the beginning, rather than at the end of the document."

Actually I think it's not a bad place. We can certainly define the reference symbols to the other library documents at the beginning (probably in section 2), but it's kind of putting the cart before the horse to talk about the optional libraries before section 7 which defines the core libraries. C99 uses Appendixes to summarize the libraries and reference several other specification documents which are relevant to the C spec (notably the IEC floating point standard). I guess I was hoping we could include whatever cross-references we want to add as part of the change proposal for this issue.

```

Reported by `danbonachea` on 2012-10-05 21:33:39

[Former user]

Dan, others--

Thanks for the feedback, and sorry about the high latency.  I've drafted a new proposed
additions and extension section, and attached the patch.

I think that we have been using the term "libraries" for this section, because it reflects
how we are currently using it.  However, this section is really meant for any "additions
and extensions" -- one could also imagine language-level extensions (e.g. looping constructs,
etc) that should follow this same process.  With that in mind, I think I've put together
a draft that should be workable for this broader usage model.

I doesn't seem like this section was providing rules for making updates to existing
parts of the spec (language, required/optional extensions).  If we want to define that
in the spec, then I think we will be adding new rules.  What's the feeling on this?
 Is it something that should be in the spec, and to what extent?

Cheers,
 ~Jim.

Reported by james.dinan on 2012-10-17 15:35:53

<hr> * Attachment: add_ext_10-17-2012.patch

[Former user]

> + The proposed extensions
> +    specifications section contains proposed additions and extensions to the
> +    UPC specification.  Such proposals are included when stable enough for
> +    developers to implement and for users to study and experiment with them.
> +    However, their presence does not suggest long term support.  When fully
> +    stable and tested, proposed additions and extensions will be moved to the
> +    required or optional section of the specification.
...
> -    Otherwise, the requirements for inclusion of an extension in the main body of
the
> -    specification are:
> +    Otherwise, the requirements for inclusion of an extension in the required or
optional
> +    extension section of the specification are:
> 
> -\item Six months residence in this section.
> +\item Six months residence in the proposed extensions section.

Perhaps I misunderstood our overall plan for the 1.3 release, but I thought the idea
was to release three official documents (language, required libs and optional libs),
not a fourth "proposed" document as the new text seems to imply. My understanding was
the library proposal documents we've been passing around the committee are solely internal
working copies to iron out details and presentation, not something intended for general
release (I apologize if my comment 14 was misleading - re-reading it I can see how
it was ambiguous). I believe the committee agreed early on that the three major proposals
slated for 1.3 (non-blocking, castability and AMO) would appear directly in the optional
library document for 1.3 ratification. 

Appendix A previously specified a 6 month delay between libraries "in this section"
(what has become the optional library) and "the main body of the specification" (which
has been divided into the core library embedded in the language document and the required
library document). I believe we want that same effect despite the cosmetic reorganization
- ie 6 months in the ratified optional document for evaluation before consideration
for movement to the required library. I don't think we want an additional 6 month delay
preceding optional - libraries in that section are already optional (allowing implementations
to completely ignore them if they choose) and they remain there until there is broad
consensus to accept them as a requirement for conformance.

"one could also imagine language-level extensions (e.g. looping constructs, etc) that
should follow this same process."

The overall goal of 1.3 was to provide some long-overdue library extensions and minor
language clarifications, but not to introduce major semantic changes to the language
itself - so I'm not sure we need to solidify a process for those kinds of changes just
yet. I had originally suggested copying our wikified spec clarification procedures
into the appendix, but upon further reflection that doesn't really seem a worthwhile
inclusion; we are presenting the reader with a ratified language specification, and
the process used to tweak and ratify that document is irrelevant to them, as it is
already done. I'm sure the ISO C committee has mountains of process documentation,
but none of it appears in the actual specification product. What they DO include is
sections like "Future language directions" and "Future library directions" that summarize
deprecations and other changes likely to appear in a future spec.

I see the purpose of the new Appendix A is to:
1. Introduce the two library documents, and explain what each implies for implementation
conformance
2. Explain that "optional" library features may eventually become "required" (ie future
directions)
3. Sketch the broad process for libraries to appear in the optional section and eventually
move to required.

However the more I think about it the more #3 seems out of place in a formal specification.
The primary readers of our ratified document are end users and implementers, both of
whom care about syntax and semantics, not specification process. It's important that
our community have concrete, documented processes in place for evolving specifications,
I'm just not sure the description of that process belongs in the spec itself. What
do others think?

Reported by danbonachea on 2012-10-18 06:49:55

[Former user]

Dan,

Sorry, the misunderstanding was on my end.  I looked in upc-specification/lib, saw
opt, req, and proposed, and thought we had moved to three extensions sections.  See
attached patch.

 ~Jim.

Reported by james.dinan on 2012-10-18 14:57:06

<hr> * Attachment: add_ext_10-18-2012-2.patch

[Former user]

Thanks for the corrections, minor nitpick:

+ Otherwise, the requirements for inclusion of an extension in the required or optional
+ extension section of the specification are:
...
+\item Six months residence in the optional extensions section.

Delete phrase "or optional".

I also think we need to include explicit bibliographical references, to clarify that
"optional extensions specifications section" is referring to the "UPC Optional Library
Specifications" document (and similarly for required). I just submitted issue 100 to
define these references in the introduction, and Appendix A should use them.

Reported by danbonachea on 2012-10-18 21:16:52

[Former user]

Whoops, thanks for catching that typo.  I also changed a couple instances of "must"
to "shall" to follow the existing style.

I agree on adding refs.  Should we just wait for a solution in issue 100?

 ~Jim.

Attached: updated patch and pdf.

Reported by james.dinan on 2012-10-18 21:46:20

<hr> * Attachment: add_ext_10-18-2012-3.patch * Attachment: upc-lang-spec-draft-issue54.pdf

[Former user]

Thanks for the updates - the latest version looks good to me.

"I agree on adding refs.  Should we just wait for a solution in issue 100?"

Unless someone objects to the spelling of the reference tags, we might as well insert
them now. The patch for this issue and issue 100 can both go into the working draft
"together".

Reported by danbonachea on 2012-10-18 22:02:48

[Former user]

This issue was discussed in the 10/19 telecon, including a long discussion about whether
process belongs in the spec at all.

A few post-call thoughts:

1. The version of Appendix A in the current working draft (without Jim's changes),
is currently nonsense due to the document split - it refers to sections that don't
exist and fails to describe the new library docs. As a matter of expediency I would
like to see this fixed in time for the SC12 draft, so we don't distribute a document
containing a section that we know makes no sense.

2. Jim's changes are a unilateral improvement to #1. With his patch, the Appendix once
again "makes sense", and provides essentially the same information and content as spec
1.2 Appendix A. Ie it's no WORSE than what's currently there, and definitely a significant
improvement.

3. The questions of whether or not the spec should include process documentation at
all and what process updates are required may end up affecting the same text, but is
an orthogonal issue and may require protracted discussion to resolve. 

Based on this I think we should proceed immediately with the 4 week comment period
and merge a patch for this issue (with the minor presentation update to the references),
and then open a new separate issue to discuss matters of process documentation.

Reported by danbonachea on 2012-10-19 21:29:35

[Former user]

Hi All,

I updated the spec with the [UPC-EXT-REQ] and [UPC-EXT-OPT] pointers.  I think we should
use "EXT" rather than "LIB" since extensions need not be limited to linkable libraries
(e.g. a new parallel loop could be defined, which is a language extension).  Updated
patch and PDF are attached.  If all looks good, let's send this our for review.

Cheers,
 ~Jim.

Reported by james.dinan on 2012-10-22 17:49:12

<hr> * Attachment: upc-lang-spec-draft-issue54.pdf * Attachment: issue_54_10-22-12.patch

[Former user]

"I think we should use "EXT" rather than "LIB" since extensions need not be limited
to linkable libraries (e.g. a new parallel loop could be defined, which is a language
extension)."

I strongly disagree with this. Currently our 2 auxiliary documents are titled "UPC
Optional/Required Library Specifications", and are very strictly library extensions
only. Each section defines a header file, a feature macro, and associated library content.
Changing them to contain anything else (language changes) would be a major format change
to these documents, and a significant departure from anything discussed or agreed upon
by the committee. We currently have no plans to introduce "optional" language features
in 1.3, and no discussion about if or how we'd do that in future revisions. Calling
them anything but library documents for 1.3 just introduces confusion.

Also, I think the Appendix section should use the word "specification" when referring
to these documents, not "section" or "section of the specification" - as they are independent
documents and not sections within the current document.

Reported by danbonachea on 2012-10-22 18:35:18

[Former user]

As I read it, the intent of the 1.2 spec is that these sections be used for "additions
and extensions".  Restricting this to libraries seems like an unnecessary constraint
that will limit its utility in the future.  I think it's important to maintain the
flexibility we have in the 1.2 spec, and to preserve the process that was laid out
for additions and extensions, which the 1.2 spec does not restrict to libraries.

In terms of the header text, we can state that it must be provided if it is needed.

I can rename "section" -> "specification".  Before I do, I should get this into SVN.
 Is there an existing branch, or should I make one?

 ~Jim.

Reported by james.dinan on 2012-10-22 20:30:23

[Former user]

"As I read it, the intent of the 1.2 spec is that these sections be used for "additions
and extensions".  Restricting this to libraries seems like an unnecessary constraint
that will limit its utility in the future.  I think it's important to maintain the
flexibility we have in the 1.2 spec, and to preserve the process that was laid out
for additions and extensions, which the 1.2 spec does not restrict to libraries."

The Appendix can still refer to the process for "extensions", but when you refer to
the actual optional and required *documents*, it should be "library", because that's
current title and organization of those documents. Calling them something else is inconsistent
and invites confusion. If 1.4 or 2.0 adds optional language extensions, we can add
a new document with a new ref or update the appendix text references accordingly.

> I can rename "section" -> "specification".  Before I do, I should get this into 
> SVN.  Is there an existing branch, or should I make one?

No textual changes may be committed to the trunk until after the 4 week comment period.
You're welcome to create a private branch for this issue, or keep updating the patch
file, whichever you find more convenient.

Reported by danbonachea on 2012-10-22 20:41:43

[Former user]

At some point we renamed "additions and extensions" (the original title of Appendix
A) to "libraries".  My assertion is that this change was not made deliberately and
should be undone.  Am I incorrect?

Reported by james.dinan on 2012-10-22 20:59:46

[Former user]

Hi All,

Dan and I have assembled a draft of the ticket 54 change.  This ticket has been moved
to pending approval, please review the proposed change and send any feedback.  If we
have consensus in time for the SC draft, we would like to include this change.

 ~Jim.

Reported by james.dinan on 2012-10-26 01:44:29 - Status changed: PendingApproval - Labels added: Consensus-High - Labels removed: Consensus-Low

<hr> * Attachment: upc-lang-spec-draft-issue54-10-25-12.pdf * Attachment: issue_54_10-25-12.patch

[Former user]

This PendingApproval change appeared in the SC12 Draft 3 release.
It was officially ratified at the 11/29 telecon.

Reported by danbonachea on 2012-11-29 20:03:22 - Status changed: Ratified