Commits

mikem  committed 1c04710

2nd part of: Overhauled llvm/clang docs builds.

  • Participants
  • Parent commits 1c9e3ba

Comments (0)

Files changed (46)

File docs/AnalyzerRegions.html

-<html>
-<head>
-<title>Static Analyzer Design Document: Memory Regions</title>
-</head>
-<body>
-  
-<h1>Static Analyzer Design Document: Memory Regions</h1>
-
-<h3>Authors</h3>
-
-<p>Ted Kremenek, <tt>kremenek at apple</tt><br>
-Zhongxing Xu, <tt>xuzhongzhing at gmail</tt></p>
-
-<h2 id="intro">Introduction</h2>
-
-<p>The path-sensitive analysis engine in libAnalysis employs an extensible API
-for abstractly modeling the memory of an analyzed program. This API employs the
-concept of "memory regions" to abstractly model chunks of program memory such as
-program variables and dynamically allocated memory such as those returned from
-'malloc' and 'alloca'. Regions are hierarchical, with subregions modeling
-subtyping relationships, field and array offsets into larger chunks of memory,
-and so on.</p>
-
-<p>The region API consists of two components:</p>
-
-<ul> <li>A taxonomy and representation of regions themselves within the analyzer
-engine. The primary definitions and interfaces are described in <tt><a
-href="http://clang.llvm.org/doxygen/MemRegion_8h-source.html">MemRegion.h</a></tt>.
-At the root of the region hierarchy is the class <tt>MemRegion</tt> with
-specific subclasses refining the region concept for variables, heap allocated
-memory, and so forth.</li> <li>The modeling of binding of values to regions. For
-example, modeling the value stored to a local variable <tt>x</tt> consists of
-recording the binding between the region for <tt>x</tt> (which represents the
-raw memory associated with <tt>x</tt>) and the value stored to <tt>x</tt>. This
-binding relationship is captured with the notion of &quot;symbolic
-stores.&quot;</li> </ul>
-
-<p>Symbolic stores, which can be thought of as representing the relation
-<tt>regions -> values</tt>, are implemented by subclasses of the
-<tt>StoreManager</tt> class (<tt><a
-href="http://clang.llvm.org/doxygen/Store_8h-source.html">Store.h</a></tt>). A
-particular StoreManager implementation has complete flexibility concerning the
-following:
-
-<ul>
-<li><em>How</em> to model the binding between regions and values</li>
-<li><em>What</em> bindings are recorded
-</ul>
-
-<p>Together, both points allow different StoreManagers to tradeoff between
-different levels of analysis precision and scalability concerning the reasoning
-of program memory. Meanwhile, the core path-sensitive engine makes no
-assumptions about either points, and queries a StoreManager about the bindings
-to a memory region through a generic interface that all StoreManagers share. If
-a particular StoreManager cannot reason about the potential bindings of a given
-memory region (e.g., '<tt>BasicStoreManager</tt>' does not reason about fields
-of structures) then the StoreManager can simply return 'unknown' (represented by
-'<tt>UnknownVal</tt>') for a particular region-binding. This separation of
-concerns not only isolates the core analysis engine from the details of
-reasoning about program memory but also facilities the option of a client of the
-path-sensitive engine to easily swap in different StoreManager implementations
-that internally reason about program memory in very different ways.</pp>
-
-<p>The rest of this document is divided into two parts. We first discuss region
-taxonomy and the semantics of regions. We then discuss the StoreManager
-interface, and details of how the currently available StoreManager classes
-implement region bindings.</p>
-
-<h2 id="regions">Memory Regions and Region Taxonomy</h2>
-
-<h3>Pointers</h3>
-
-<p>Before talking about the memory regions, we would talk about the pointers
-since memory regions are essentially used to represent pointer values.</p>
-
-<p>The pointer is a type of values. Pointer values have two semantic aspects.
-One is its physical value, which is an address or location. The other is the
-type of the memory object residing in the address.</p>
-
-<p>Memory regions are designed to abstract these two properties of the pointer.
-The physical value of a pointer is represented by MemRegion pointers. The rvalue
-type of the region corresponds to the type of the pointee object.</p>
-
-<p>One complication is that we could have different view regions on the same
-memory chunk. They represent the same memory location, but have different
-abstract location, i.e., MemRegion pointers. Thus we need to canonicalize the
-abstract locations to get a unique abstract location for one physical
-location.</p>
-
-<p>Furthermore, these different view regions may or may not represent memory
-objects of different types. Some different types are semantically the same,
-for example, 'struct s' and 'my_type' are the same type.</p>
-
-<pre>
-struct s;
-typedef struct s my_type;
-</pre>
-
-<p>But <tt>char</tt> and <tt>int</tt> are not the same type in the code below:</p>
-
-<pre>
-void *p;
-int *q = (int*) p;
-char *r = (char*) p;
-</pre
-
-<p>Thus we need to canonicalize the MemRegion which is used in binding and
-retrieving.</p>
-
-<h3>Regions</h3>
-<p>Region is the entity used to model pointer values. A Region has the following
-properties:</p>
-
-<ul>
-<li>Kind</li>
-
-<li>ObjectType: the type of the object residing on the region.</li>
-
-<li>LocationType: the type of the pointer value that the region corresponds to.
-  Usually this is the pointer to the ObjectType. But sometimes we want to cache
-  this type explicitly, for example, for a CodeTextRegion.</li>
-
-<li>StartLocation</li>
-
-<li>EndLocation</li>
-</ul>
-
-<h3>Symbolic Regions</h3>
-
-<p>A symbolic region is a map of the concept of symbolic values into the domain
-of regions. It is the way that we represent symbolic pointers. Whenever a
-symbolic pointer value is needed, a symbolic region is created to represent
-it.</p>
-
-<p>A symbolic region has no type. It wraps a SymbolData. But sometimes we have
-type information associated with a symbolic region. For this case, a
-TypedViewRegion is created to layer the type information on top of the symbolic
-region. The reason we do not carry type information with the symbolic region is
-that the symbolic regions can have no type. To be consistent, we don't let them
-to carry type information.</p>
-
-<p>Like a symbolic pointer, a symbolic region may be NULL, has unknown extent,
-and represents a generic chunk of memory.</p>
-
-<p><em><b>NOTE</b>: We plan not to use loc::SymbolVal in RegionStore and remove it
-  gradually.</em></p>
-
-<p>Symbolic regions get their rvalue types through the following ways:</p>
-
-<ul>
-<li>Through the parameter or global variable that points to it, e.g.:
-<pre>
-void f(struct s* p) {
-  ...
-}
-</pre>
-
-<p>The symbolic region pointed to by <tt>p</tt> has type <tt>struct
-s</tt>.</p></li>
-
-<li>Through explicit or implicit casts, e.g.:
-<pre>
-void f(void* p) {
-  struct s* q = (struct s*) p;
-  ...
-}
-</pre>
-</li>
-</ul>
-
-<p>We attach the type information to the symbolic region lazily. For the first
-case above, we create the <tt>TypedViewRegion</tt> only when the pointer is
-actually used to access the pointee memory object, that is when the element or
-field region is created. For the cast case, the <tt>TypedViewRegion</tt> is
-created when visiting the <tt>CastExpr</tt>.</p>
-
-<p>The reason for doing lazy typing is that symbolic regions are sometimes only
-used to do location comparison.</p>
-
-<h3>Pointer Casts</h3>
-
-<p>Pointer casts allow people to impose different 'views' onto a chunk of
-memory.</p>
-
-<p>Usually we have two kinds of casts. One kind of casts cast down with in the
-type hierarchy. It imposes more specific views onto more generic memory regions.
-The other kind of casts cast up with in the type hierarchy. It strips away more
-specific views on top of the more generic memory regions.</p>
-
-<p>We simulate the down casts by layering another <tt>TypedViewRegion</tt> on
-top of the original region. We simulate the up casts by striping away the top
-<tt>TypedViewRegion</tt>. Down casts is usually simple. For up casts, if the
-there is no <tt>TypedViewRegion</tt> to be stripped, we return the original
-region. If the underlying region is of the different type than the cast-to type,
-we flag an error state.</p>
-
-<p>For toll-free bridging casts, we return the original region.</p>
-
-<p>We can set up a partial order for pointer types, with the most general type
-<tt>void*</tt> at the top. The partial order forms a tree with <tt>void*</tt> as
-its root node.</p>
-
-<p>Every <tt>MemRegion</tt> has a root position in the type tree. For example,
-the pointee region of <tt>void *p</tt> has its root position at the root node of
-the tree. <tt>VarRegion</tt> of <tt>int x</tt> has its root position at the 'int
-type' node.</p>
-
-<p><tt>TypedViewRegion</tt> is used to move the region down or up in the tree.
-Moving down in the tree adds a <tt>TypedViewRegion</tt>. Moving up in the tree
-removes a <Tt>TypedViewRegion</tt>.</p>
-
-<p>Do we want to allow moving up beyond the root position? This happens
-when:</p> <pre> int x; void *p = &amp;x; </pre>
-
-<p>The region of <tt>x</tt> has its root position at 'int*' node. the cast to
-void* moves that region up to the 'void*' node. I propose to not allow such
-casts, and assign the region of <tt>x</tt> for <tt>p</tt>.</p>
-
-<p>Another non-ideal case is that people might cast to a non-generic pointer
-from another non-generic pointer instead of first casting it back to the generic
-pointer. Direct handling of this case would result in multiple layers of
-TypedViewRegions. This enforces an incorrect semantic view to the region,
-because we can only have one typed view on a region at a time. To avoid this
-inconsistency, before casting the region, we strip the TypedViewRegion, then do
-the cast. In summary, we only allow one layer of TypedViewRegion.</p>
-
-<h3>Region Bindings</h3>
-
-<p>The following region kinds are boundable: VarRegion, CompoundLiteralRegion,
-StringRegion, ElementRegion, FieldRegion, and ObjCIvarRegion.</p>
-
-<p>When binding regions, we perform canonicalization on element regions and field
-regions. This is because we can have different views on the same region, some
-of which are essentially the same view with different sugar type names.</p>
-
-<p>To canonicalize a region, we get the canonical types for all TypedViewRegions
-along the way up to the root region, and make new TypedViewRegions with those
-canonical types.</p>
-
-<p>For Objective-C and C++, perhaps another canonicalization rule should be
-added: for FieldRegion, the least derived class that has the field is used as
-the type of the super region of the FieldRegion.</p>
-
-<p>All bindings and retrievings are done on the canonicalized regions.</p>
-
-<p>Canonicalization is transparent outside the region store manager, and more
-specifically, unaware outside the Bind() and Retrieve() method. We don't need to
-consider region canonicalization when doing pointer cast.</p>
-
-<h3>Constraint Manager</h3>
-
-<p>The constraint manager reasons about the abstract location of memory objects.
-We can have different views on a region, but none of these views changes the
-location of that object. Thus we should get the same abstract location for those
-regions.</p>
-
-</body>
-</html>

File docs/Block-ABI-Apple.txt

-Block Implementation Specification
-
-Copyright 2008-2010 Apple, Inc.
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the "Software"), to deal
-in the Software without restriction, including without limitation the rights
-to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
-
-The above copyright notice and this permission notice shall be included in
-all copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
-THE SOFTWARE.
-
-0. History
-
-2008/7/14  - created
-2008/8/21  - revised, C++
-2008/9/24  - add NULL isa field to __block storage
-2008/10/1  - revise block layout to use a static descriptor structure
-2008/10/6  - revise block layout to use an unsigned long int flags
-2008/10/28 - specify use of _Block_object_assign/dispose for all "Object" types in helper functions
-2008/10/30 - revise new layout to have invoke function in same place
-2008/10/30 - add __weak support
-
-2010/3/16  - rev for stret return, signature field
-2010/4/6   - improved wording
-
-This document describes the Apple ABI implementation specification of Blocks.
-
-The first shipping version of this ABI is found in Mac OS X 10.6, and shall be referred to as 10.6.ABI. As of 2010/3/16, the following describes the ABI contract with the runtime and the compiler, and, as necessary, will be referred to as ABI.2010.3.16.
-
-Since the Apple ABI references symbols from other elements of the system, any attempt to use this ABI on systems prior to SnowLeopard is undefined.
-
-1. High Level
-
-The ABI of blocks consist of their layout and the runtime functions required by the compiler.
-A Block consists of a structure of the following form:
-
-struct Block_literal_1 {
-    void *isa; // initialized to &_NSConcreteStackBlock or &_NSConcreteGlobalBlock
-    int flags;
-    int reserved; 
-    void (*invoke)(void *, ...);
-    struct Block_descriptor_1 {
-	unsigned long int reserved;	// NULL
-    	unsigned long int size;         // sizeof(struct Block_literal_1)
-	// optional helper functions
-    	void (*copy_helper)(void *dst, void *src);     // IFF (1<<25)
-    	void (*dispose_helper)(void *src);             // IFF (1<<25)
-        // required ABI.2010.3.16
-        const char *signature;                         // IFF (1<<30)
-    } *descriptor;
-    // imported variables
-};
-
-The following flags bits are in use thusly for a possible ABI.2010.3.16:
-
-enum {
-    BLOCK_HAS_COPY_DISPOSE =  (1 << 25),
-    BLOCK_HAS_CTOR =          (1 << 26), // helpers have C++ code
-    BLOCK_IS_GLOBAL =         (1 << 28),
-    BLOCK_HAS_STRET =         (1 << 29),
-    BLOCK_HAS_SIGNATURE =     (1 << 30), 
-};
-
-In 10.6.ABI the (1<<29) was unconditionally set and ignored by the runtime - it was a transitional marker that did not get deleted after the transition. This bit is now paired with (1<<30), and represented as the pair (3<<30), for the following combinations of valid bit settings, and their meanings.
-
-switch (flags & (3<<29)) {
-  case (0<<29):  <unused>    , error
-  case (1<<29):      10.6.ABI, no signature field available
-  case (2<<29): ABI.2010.3.16, regular calling convention, presence of signature field
-  case (3<<29): ABI.2010.3.16, stret calling convention, presence of signature field,
-}
-
-The following discussions are presented as 10.6.ABI otherwise.
-
-Block literals may occur within functions where the structure is created in stack local memory.  They may also appear as initialization expressions for Block variables of global or static local variables.
-
-When a Block literal expression is evaluated the stack based structure is initialized as follows:
-
-1) static descriptor structure is declared and initialized as follows:
-1a) the invoke function pointer is set to a function that takes the Block structure as its first argument and the rest of the arguments (if any) to the Block and executes the Block compound statement.
-1b) the size field is set to the size of the following Block literal structure.
-1c) the copy_helper and dispose_helper function pointers are set to respective helper functions if they are required by the Block literal
-2) a stack (or global) Block literal data structure is created and initialized as follows:
-2a) the isa field is set to the address of the external _NSConcreteStackBlock, which is a block of uninitialized memory supplied in libSystem, or _NSConcreteGlobalBlock if this is a static or file level block literal.
-2) The flags field is set to zero unless there are variables imported into the block that need helper functions for program level Block_copy() and Block_release() operations, in which case the (1<<25) flags bit is set.
-
-
-As an example, the Block literal expression
-   ^ { printf("hello world\n"); }
-would cause to be created on a 32-bit system:
-
-struct __block_literal_1 {
-    void *isa;
-    int flags;
-    int reserved; 
-    void (*invoke)(struct __block_literal_1 *);
-    struct __block_descriptor_1 *descriptor;
-};
-
-void __block_invoke_1(struct __block_literal_1 *_block) {
-    printf("hello world\n");
-}
-
-static struct __block_descriptor_1 {
-    unsigned long int reserved;
-    unsigned long int Block_size;
-} __block_descriptor_1 = { 0, sizeof(struct __block_literal_1), __block_invoke_1 };
-
-and where the block literal appeared
-
-  struct __block_literal_1 _block_literal = {
-	&_NSConcreteStackBlock,
-	(1<<29), <uninitialized>,
-	__block_invoke_1,
-	&__block_descriptor_1
-   };
-
-Blocks import other Block references, const copies of other variables, and variables marked __block.  In Objective-C variables may additionally be objects.
-
-When a Block literal expression used as the initial value of a global or static local variable it is initialized as follows:
-  struct __block_literal_1 __block_literal_1 = {
-	&_NSConcreteGlobalBlock,
-	(1<<28)|(1<<29), <uninitialized>,
-	__block_invoke_1,
-	&__block_descriptor_1
-   };
-that is, a different address is provided as the first value and a particular (1<<28) bit is set in the flags field, and otherwise it is the same as for stack based Block literals.  This is an optimization that can be used for any Block literal that imports no const or __block storage variables.
-
-
-2. Imported Variables
-
-Variables of "auto" storage class are imported as const copies.  Variables of "__block" storage class are imported as a pointer to an enclosing data structure.  Global variables are simply referenced and not considered as imported.
-
-2.1 Imported const copy variables
-
-Automatic storage variables not marked with __block are imported as const copies.
-
-The simplest example is that of importing a variable of type int.
-
-   int x = 10;
-   void (^vv)(void) = ^{ printf("x is %d\n", x); }
-   x = 11;
-   vv();
-
-would be compiled
-
-struct __block_literal_2 {
-    void *isa;
-    int flags;
-    int reserved; 
-    void (*invoke)(struct __block_literal_2 *);
-    struct __block_descriptor_2 *descriptor;
-    const int x;
-};
-
-void __block_invoke_2(struct __block_literal_2 *_block) {
-    printf("x is %d\n", _block->x);
-}
-
-static struct __block_descriptor_2 {
-    unsigned long int reserved;
-    unsigned long int Block_size;
-} __block_descriptor_2 = { 0, sizeof(struct __block_literal_2) };
-
-and
-
-  struct __block_literal_2 __block_literal_2 = {
-	&_NSConcreteStackBlock,
-	(1<<29), <uninitialized>,
-	__block_invoke_2,
-	&__block_descriptor_2,
-        x
-   };
-
-In summary, scalars, structures, unions, and function pointers are generally imported as const copies with no need for helper functions.
-
-2.2 Imported const copy of Block reference
-
-The first case where copy and dispose helper functions are required is for the case of when a block itself is imported.  In this case both a copy_helper function and a dispose_helper function are needed.  The copy_helper function is passed both the existing stack based pointer and the pointer to the new heap version and should call back into the runtime to actually do the copy operation on the imported fields within the block.  The runtime functions are all described in Section 5.0 Runtime Helper Functions.
-
-An example:
-
-   void (^existingBlock)(void) = ...;
-   void (^vv)(void) = ^{ existingBlock(); }
-   vv();
-
-struct __block_literal_3 {
-   ...; // existing block
-};
-
-struct __block_literal_4 {
-    void *isa;
-    int flags;
-    int reserved; 
-    void (*invoke)(struct __block_literal_4 *);
-    struct __block_literal_3 *const existingBlock;
-};
-
-void __block_invoke_4(struct __block_literal_2 *_block) {
-   __block->existingBlock->invoke(__block->existingBlock);
-}
-
-void __block_copy_4(struct __block_literal_4 *dst, struct __block_literal_4 *src) {
-     //_Block_copy_assign(&dst->existingBlock, src->existingBlock, 0);
-     _Block_object_assign(&dst->existingBlock, src->existingBlock, BLOCK_FIELD_IS_BLOCK);
-}
-
-void __block_dispose_4(struct __block_literal_4 *src) {
-     // was _Block_destroy
-     _Block_object_dispose(src->existingBlock, BLOCK_FIELD_IS_BLOCK);
-}
-
-static struct __block_descriptor_4 {
-    unsigned long int reserved;
-    unsigned long int Block_size;
-    void (*copy_helper)(struct __block_literal_4 *dst, struct __block_literal_4 *src);
-    void (*dispose_helper)(struct __block_literal_4 *);
-} __block_descriptor_4 = {
-	0,
-	sizeof(struct __block_literal_4),
-	__block_copy_4,
-	__block_dispose_4,
-};
-
-and where it is used
-
-  struct __block_literal_4 _block_literal = {
-	&_NSConcreteStackBlock,
-	(1<<25)|(1<<29), <uninitialized>
-	__block_invoke_4,
-	& __block_descriptor_4
-        existingBlock,
-   };
-
-2.2.1 Importing __attribute__((NSObject)) variables.
-
-GCC introduces __attribute__((NSObject)) on structure pointers to mean "this is an object".  This is useful because many low level data structures are declared as opaque structure pointers, e.g. CFStringRef, CFArrayRef, etc.  When used from C, however, these are still really objects and are the second case where that requires copy and dispose helper functions to be generated.  The copy helper functions generated by the compiler should use the _Block_object_assign runtime helper function and in the dispose helper the _Block_object_dispose runtime helper function should be called.
-
-For example, block xyzzy in the following
-
-    struct Opaque *__attribute__((NSObject)) objectPointer = ...;
-    ...
-    void (^xyzzy)(void) = ^{  CFPrint(objectPointer); };
-
-would have helper functions
-
-void __block_copy_xyzzy(struct __block_literal_5 *dst, struct __block_literal_5 *src) {
-     _Block_object_assign(&dst->objectPointer, src-> objectPointer, BLOCK_FIELD_IS_OBJECT);
-}
-
-void __block_dispose_xyzzy(struct __block_literal_5 *src) {
-     _Block_object_dispose(src->objectPointer, BLOCK_FIELD_IS_OBJECT);
-}
-
-generated.
-
-
-2.3 Imported __block marked variables.
-
-2.3.1 Layout of __block marked variables
-
-The compiler must embed variables that are marked __block in a specialized structure of the form:
-
-struct _block_byref_xxxx {
-    void *isa;
-    struct Block_byref *forwarding;
-    int flags;   //refcount;
-    int size;
-    typeof(marked_variable) marked_variable;
-};
-
-Variables of certain types require helper functions for when Block_copy() and Block_release() are performed upon a referencing Block.  At the "C" level only variables that are of type Block or ones that have __attribute__((NSObject)) marked require helper functions.  In Objective-C objects require helper functions and in C++ stack based objects require helper functions. Variables that require helper functions use the form:
-
-struct _block_byref_xxxx {
-    void *isa;
-    struct _block_byref_xxxx *forwarding;
-    int flags;   //refcount;
-    int size;
-    // helper functions called via Block_copy() and Block_release()
-    void (*byref_keep)(void  *dst, void *src);
-    void (*byref_dispose)(void *);
-    typeof(marked_variable) marked_variable;
-};
-
-The structure is initialized such that
- a) the forwarding pointer is set to the beginning of its enclosing structure,
- b) the size field is initialized to the total size of the enclosing structure,
- c) the flags field is set to either 0 if no helper functions are needed or (1<<25) if they are,
- d) the helper functions are initialized (if present)
- e) the variable itself is set to its initial value.
- f) the isa field is set to NULL
-
-2.3.2 Access to __block variables from within its lexical scope.
-
-In order to "move" the variable to the heap upon a copy_helper operation the compiler must rewrite access to such a variable to be indirect through the structures forwarding pointer.  For example:
-
-  int __block i = 10;
-  i = 11;
-
-would be rewritten to be:
-
-  struct _block_byref_i {
-    void *isa;
-    struct _block_byref_i *forwarding;
-    int flags;   //refcount;
-    int size;
-    int captured_i;
-  } i = { NULL, &i, 0, sizeof(struct _block_byref_i), 11 };
-
-  i.forwarding->captured_i = 11;
-
-In the case of a Block reference variable being marked __block the helper code generated must use the _Block_object_assign and _Block_object_dispose routines supplied by the runtime to make the copies.  For example:
-
-   __block void (voidBlock)(void) = blockA;
-   voidBlock = blockB;
-
-would translate into
-
-struct _block_byref_voidBlock {
-    void *isa;
-    struct _block_byref_voidBlock *forwarding;
-    int flags;   //refcount;
-    int size;
-    void (*byref_keep)(struct _block_byref_voidBlock *dst, struct _block_byref_voidBlock *src);
-    void (*byref_dispose)(struct _block_byref_voidBlock *);
-    void (^captured_voidBlock)(void);
-};
-
-void _block_byref_keep_helper(struct _block_byref_voidBlock *dst, struct _block_byref_voidBlock *src) {
-    //_Block_copy_assign(&dst->captured_voidBlock, src->captured_voidBlock, 0);
-    _Block_object_assign(&dst->captured_voidBlock, src->captured_voidBlock, BLOCK_FIELD_IS_BLOCK | BLOCK_BYREF_CALLER);
-}
-
-void _block_byref_dispose_helper(struct _block_byref_voidBlock *param) {
-    //_Block_destroy(param->captured_voidBlock, 0);
-    _Block_object_dispose(param->captured_voidBlock, BLOCK_FIELD_IS_BLOCK | BLOCK_BYREF_CALLER)}
-
-and
-  struct _block_byref_voidBlock voidBlock = {( .forwarding=&voidBlock, .flags=(1<<25), .size=sizeof(struct _block_byref_voidBlock *),
-      .byref_keep=_block_byref_keep_helper, .byref_dispose=_block_byref_dispose_helper,
-      .captured_voidBlock=blockA };
-
-  voidBlock.forwarding->captured_voidBlock = blockB;
-  
-
-2.3.3 Importing __block variables into Blocks
-
-A Block that uses a __block variable in its compound statement body must import the variable and emit copy_helper and dispose_helper helper functions that, in turn, call back into the runtime to actually copy or release the byref data block using the functions _Block_object_assign and _Block_object_dispose.
-
-For example:
-
-   int __block i = 2;
-   functioncall(^{ i = 10; });
-
-would translate to
-
-struct _block_byref_i {
-    void *isa;  // set to NULL
-    struct _block_byref_voidBlock *forwarding;
-    int flags;   //refcount;
-    int size;
-    void (*byref_keep)(struct _block_byref_i *dst, struct _block_byref_i *src);
-    void (*byref_dispose)(struct _block_byref_i *);
-    int captured_i;
-};
-
-
-struct __block_literal_5 {
-    void *isa;
-    int flags;
-    int reserved; 
-    void (*invoke)(struct __block_literal_5 *);
-    struct __block_descriptor_5 *descriptor;
-    struct _block_byref_i *i_holder;
-};
-
-void __block_invoke_5(struct __block_literal_5 *_block) {
-   _block->forwarding->captured_i = 10;
-}
-
-void __block_copy_5(struct __block_literal_5 *dst, struct __block_literal_5 *src) {
-     //_Block_byref_assign_copy(&dst->captured_i, src->captured_i);
-     _Block_object_assign(&dst->captured_i, src->captured_i, BLOCK_FIELD_IS_BYREF | BLOCK_BYREF_CALLER);
-}
-
-void __block_dispose_5(struct __block_literal_5 *src) {
-     //_Block_byref_release(src->captured_i);
-     _Block_object_dispose(src->captured_i, BLOCK_FIELD_IS_BYREF | BLOCK_BYREF_CALLER);
-}
-
-static struct __block_descriptor_5 {
-    unsigned long int reserved;
-    unsigned long int Block_size;
-    void (*copy_helper)(struct __block_literal_5 *dst, struct __block_literal_5 *src);
-    void (*dispose_helper)(struct __block_literal_5 *);
-} __block_descriptor_5 = { 0, sizeof(struct __block_literal_5) __block_copy_5, __block_dispose_5 };
-
-and
-
-  struct _block_byref_i i = {( .forwarding=&i, .flags=0, .size=sizeof(struct _block_byref_i) )};
-  struct __block_literal_5 _block_literal = {
-	&_NSConcreteStackBlock,
-	(1<<25)|(1<<29), <uninitialized>,
-	__block_invoke_5,
-	&__block_descriptor_5,
-        2,
-   };
-
-2.3.4 Importing __attribute__((NSObject)) __block variables
-
-A __block variable that is also marked __attribute__((NSObject)) should have byref_keep and byref_dispose helper functions that use _Block_object_assign and _Block_object_dispose.
-
-2.3.5 __block escapes
-
-Because Blocks referencing __block variables may have Block_copy() performed upon them the underlying storage for the variables may move to the heap.  In Objective-C Garbage Collection Only compilation environments the heap used is the garbage collected one and no further action is required.  Otherwise the compiler must issue a call to potentially release any heap storage for __block variables at all escapes or terminations of their scope.
-
-
-2.3.6 Nesting
-
-Blocks may contain Block literal expressions.  Any variables used within inner blocks are imported into all enclosing Block scopes even if the variables are not used.  This includes const imports as well as __block variables.
-
-3. Objective C Extensions to Blocks
-
-3.1 Importing Objects
-
-Objects should be treated as __attribute__((NSObject)) variables; all copy_helper, dispose_helper, byref_keep, and byref_dispose helper functions should use _Block_object_assign and _Block_object_dispose.  There should be no code generated that uses -retain or -release methods.
-
-
-3.2 Blocks as Objects
-
-The compiler will treat Blocks as objects when synthesizing property setters and getters, will characterize them as objects when generating garbage collection strong and weak layout information in the same manner as objects, and will issue strong and weak write-barrier assignments in the same manner as objects.
-
-3.3 __weak __block Support
-
-Objective-C (and Objective-C++) support the __weak attribute on __block variables.  Under normal circumstances the compiler uses the Objective-C runtime helper support functions objc_assign_weak and objc_read_weak.  Both should continue to be used for all reads and writes of __weak __block variables:
-	objc_read_weak(&block->byref_i->forwarding->i)
-
-The __weak variable is stored in a _block_byref_xxxx structure and the Block has copy and dispose helpers for this structure that call:
-	_Block_object_assign(&dest->_block_byref_i, src-> _block_byref_i, BLOCK_FIELD_IS_WEAK | BLOCK_FIELD_IS_BYREF);
-and
-	_Block_object_dispose(src->_block_byref_i, BLOCK_FIELD_IS_WEAK | BLOCK_FIELD_IS_BYREF);
-
-
-In turn, the block_byref copy support helpers distinguish between whether the __block variable is a Block or not and should either call:
-	_Block_object_assign(&dest->_block_byref_i, src->_block_byref_i, BLOCK_FIELD_IS_WEAK | BLOCK_FIELD_IS_OBJECT | BLOCK_BYREF_CALLER);
-for something declared as an object or
-	_Block_object_assign(&dest->_block_byref_i, src->_block_byref_i, BLOCK_FIELD_IS_WEAK | BLOCK_FIELD_IS_BLOCK | BLOCK_BYREF_CALLER);
-for something declared as a Block.
-
-A full example follows:
-
-
-   __block __weak id obj = <initialization expression>;
-   functioncall(^{ [obj somemessage]; });
-
-would translate to
-
-struct _block_byref_obj {
-    void *isa;  // uninitialized
-    struct _block_byref_obj *forwarding;
-    int flags;   //refcount;
-    int size;
-    void (*byref_keep)(struct _block_byref_i *dst, struct _block_byref_i *src);
-    void (*byref_dispose)(struct _block_byref_i *);
-    int captured_obj;
-};
-
-void _block_byref_obj_keep(struct _block_byref_voidBlock *dst, struct _block_byref_voidBlock *src) {
-    //_Block_copy_assign(&dst->captured_obj, src->captured_obj, 0);
-    _Block_object_assign(&dst->captured_obj, src->captured_obj, BLOCK_FIELD_IS_OBJECT | BLOCK_FIELD_IS_WEAK | BLOCK_BYREF_CALLER);
-}
-
-void _block_byref_obj_dispose(struct _block_byref_voidBlock *param) {
-    //_Block_destroy(param->captured_obj, 0);
-    _Block_object_dispose(param->captured_obj, BLOCK_FIELD_IS_OBJECT | BLOCK_FIELD_IS_WEAK | BLOCK_BYREF_CALLER);
-};
-
-for the block byref part and
-
-struct __block_literal_5 {
-    void *isa;
-    int flags;
-    int reserved; 
-    void (*invoke)(struct __block_literal_5 *);
-    struct __block_descriptor_5 *descriptor;
-    struct _block_byref_obj *byref_obj;
-};
-
-void __block_invoke_5(struct __block_literal_5 *_block) {
-   [objc_read_weak(&_block->byref_obj->forwarding->captured_obj) somemessage];
-}
-
-void __block_copy_5(struct __block_literal_5 *dst, struct __block_literal_5 *src) {
-     //_Block_byref_assign_copy(&dst->byref_obj, src->byref_obj);
-     _Block_object_assign(&dst->byref_obj, src->byref_obj, BLOCK_FIELD_IS_BYREF | BLOCK_FIELD_IS_WEAK);
-}
-
-void __block_dispose_5(struct __block_literal_5 *src) {
-     //_Block_byref_release(src->byref_obj);
-     _Block_object_dispose(src->byref_obj, BLOCK_FIELD_IS_BYREF | BLOCK_FIELD_IS_WEAK);
-}
-
-static struct __block_descriptor_5 {
-    unsigned long int reserved;
-    unsigned long int Block_size;
-    void (*copy_helper)(struct __block_literal_5 *dst, struct __block_literal_5 *src);
-    void (*dispose_helper)(struct __block_literal_5 *);
-} __block_descriptor_5 = { 0, sizeof(struct __block_literal_5), __block_copy_5, __block_dispose_5 };
-
-and within the compound statement:
-
-  struct _block_byref_obj obj = {( .forwarding=&obj, .flags=(1<<25), .size=sizeof(struct _block_byref_obj),
-				.byref_keep=_block_byref_obj_keep, .byref_dispose=_block_byref_obj_dispose,
-				.captured_obj = <initialization expression> )};
-
-  struct __block_literal_5 _block_literal = {
-	&_NSConcreteStackBlock,
-	(1<<25)|(1<<29), <uninitialized>,
-	__block_invoke_5,
-	&__block_descriptor_5,
-        &obj,		// a reference to the on-stack structure containing "captured_obj"
-   };
-
-
-   functioncall(_block_literal->invoke(&_block_literal));
-
-
-4.0 C++ Support
-
-Within a block stack based C++ objects are copied as const copies using the const copy constructor.  It is an error if a stack based C++ object is used within a block if it does not have a const copy constructor.  In addition both copy and destroy helper routines must be synthesized for the block to support the Block_copy() operation, and the flags work marked with the (1<<26) bit in addition to the (1<<25) bit.  The copy helper should call the constructor using appropriate offsets of the variable within the supplied stack based block source and heap based destination for all const constructed copies, and similarly should call the destructor in the destroy routine.
-
-As an example, suppose a C++ class FOO existed with a const copy constructor.  Within a code block a stack version of a FOO object is declared and used within a Block literal expression:
-
-{
-    FOO foo;
-    void (^block)(void) = ^{ printf("%d\n", foo.value()); };
-}
-
-The compiler would synthesize
-
-struct __block_literal_10 {
-    void *isa;
-    int flags;
-    int reserved; 
-    void (*invoke)(struct __block_literal_10 *);
-    struct __block_descriptor_10 *descriptor;
-    const FOO foo;
-};
-
-void __block_invoke_10(struct __block_literal_10 *_block) {
-   printf("%d\n", _block->foo.value());
-}
-
-void __block_literal_10(struct __block_literal_10 *dst, struct __block_literal_10 *src) {
-     comp_ctor(&dst->foo, &src->foo);
-}
-
-void __block_dispose_10(struct __block_literal_10 *src) {
-     comp_dtor(&src->foo);
-}
-
-static struct __block_descriptor_10 {
-    unsigned long int reserved;
-    unsigned long int Block_size;
-    void (*copy_helper)(struct __block_literal_10 *dst, struct __block_literal_10 *src);
-    void (*dispose_helper)(struct __block_literal_10 *);
-} __block_descriptor_10 = { 0, sizeof(struct __block_literal_10), __block_copy_10, __block_dispose_10 };
-
-and the code would be:
-{
-  FOO foo;
-  comp_ctor(&foo); // default constructor
-  struct __block_literal_10 _block_literal = {
-	&_NSConcreteStackBlock,
-	(1<<25)|(1<<26)|(1<<29), <uninitialized>,
-	__block_invoke_10,
-	&__block_descriptor_10,
-   };
-   comp_ctor(&_block_literal->foo, &foo);  // const copy into stack version
-   struct __block_literal_10 &block = &_block_literal;  // assign literal to block variable
-   block->invoke(block);	// invoke block
-   comp_dtor(&_block_literal->foo); // destroy stack version of const block copy
-   comp_dtor(&foo); // destroy original version
-}
-
-
-C++ objects stored in __block storage start out on the stack in a block_byref data structure as do other variables.  Such objects (if not const objects) must support a regular copy constructor.  The block_byref data structure will have copy and destroy helper routines synthesized by the compiler.  The copy helper will have code created to perform the copy constructor based on the initial stack block_byref data structure, and will also set the (1<<26) bit in addition to the (1<<25) bit.  The destroy helper will have code to do the destructor on the object stored within the supplied block_byref heap data structure.
-
-To support member variable and function access the compiler will synthesize a const pointer to a block version of the this pointer.
-
-5.0 Runtime Helper Functions
-
-The runtime helper functions are described in /usr/local/include/Block_private.h.  To summarize their use, a block requires copy/dispose helpers if it imports any block variables, __block storage variables, __attribute__((NSObject)) variables, or C++ const copied objects with constructor/destructors.  The (1<<26) bit is set and functions are generated.
-
-The block copy helper function should, for each of the variables of the type mentioned above, call
-     _Block_object_assign(&dst->target, src->target, BLOCK_FIELD_<appropo>);
-in the copy helper and
-    _Block_object_dispose(->target, BLOCK_FIELD_<appropo>);
-in the dispose helper where
-      <appropo> is
-
-enum {
-    BLOCK_FIELD_IS_OBJECT   =  3,  // id, NSObject, __attribute__((NSObject)), block, ...
-    BLOCK_FIELD_IS_BLOCK    =  7,  // a block variable
-    BLOCK_FIELD_IS_BYREF    =  8,  // the on stack structure holding the __block variable
-
-    BLOCK_FIELD_IS_WEAK     = 16,  // declared __weak
-
-    BLOCK_BYREF_CALLER      = 128, // called from byref copy/dispose helpers
-};
-
-and of course the CTORs/DTORs for const copied C++ objects.
-
-The block_byref data structure similarly requires copy/dispose helpers for block variables, __attribute__((NSObject)) variables, or C++ const copied objects with constructor/destructors, and again the (1<<26) bit is set and functions are generated in the same manner.
-
-Under ObjC we allow __weak as an attribute on __block variables, and this causes the addition of BLOCK_FIELD_IS_WEAK orred onto the BLOCK_FIELD_IS_BYREF flag when copying the block_byref structure in the block copy helper, and onto the BLOCK_FIELD_<appropo> field within the block_byref copy/dispose helper calls.
-
-The prototypes, and summary, of the helper functions are
-
-/* Certain field types require runtime assistance when being copied to the heap.  The following function is used
-   to copy fields of types: blocks, pointers to byref structures, and objects (including __attribute__((NSObject)) pointers.
-   BLOCK_FIELD_IS_WEAK is orthogonal to the other choices which are mutually exclusive.
-   Only in a Block copy helper will one see BLOCK_FIELD_IS_BYREF.
- */
-void _Block_object_assign(void *destAddr, const void *object, const int flags);
-
-/* Similarly a compiler generated dispose helper needs to call back for each field of the byref data structure.
-   (Currently the implementation only packs one field into the byref structure but in principle there could be more).
-   The same flags used in the copy helper should be used for each call generated to this function:
- */
-void _Block_object_dispose(const void *object, const int flags);
-
-The following functions have been used and will continue to be supported until new compiler support is complete.
-
-// Obsolete functions.
-// Copy helper callback for copying a block imported into a Block
-// Called by copy_helper helper functions synthesized by the compiler.
-// The address in the destination block of an imported Block is provided as the first argument
-// and the value of the existing imported Block is the second.
-// Use: _Block_object_assign(dest, src, BLOCK_FIELD_IS_BLOCK {| BLOCK_FIELD_IS_WEAK});
-void _Block_copy_assign(struct Block_basic **dest, const struct Block_basic *src, const int flags);
-
-// Destroy helper callback for releasing Blocks imported into a Block
-// Called by dispose_helper helper functions synthesized by the compiler.
-// The value of the imported Block variable is passed back.
-// Use: _Block_object_dispose(src, BLOCK_FIELD_IS_BLOCK {| BLOCK_FIELD_IS_WEAK});
-void _Block_destroy(const struct Block_basic *src, const int flags);
-
-// Byref data block copy helper callback
-// Called by block copy helpers when copying __block structures
-// Use: _Block_object_assign(dest, src, BLOCK_FIELD_IS_BYREF {| BLOCK_FIELD_IS_WEAK});
-void _Block_byref_assign_copy(struct Block_byref **destp, struct Block_byref *src);
-
-// Byref data block release helper callback
-// Called by block release helpers when releasing a Block 
-// Called at escape points in scope where __block variables live (under non-GC-only conditions) 
-// Use: _Block_object_dispose(src, BLOCK_FIELD_IS_BYREF {| BLOCK_FIELD_IS_WEAK});
-void �(struct Block_byref *shared_struct);
-
-

File docs/BlockLanguageSpec.txt

-Language Specification for Blocks
-
-2008/2/25 — created
-2008/7/28 — revised, __block syntax
-2008/8/13 — revised, Block globals
-2008/8/21 — revised, C++ elaboration
-2008/11/1 — revised, __weak support
-2009/1/12 — revised, explicit return types
-2009/2/10 — revised, __block objects need retain
-
-Copyright 2008-2009 Apple, Inc. Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
-
-The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR  IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,  FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE  AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER  LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
-
-The Block Type
-
-A new derived type is introduced to C and, by extension, Objective-C, C++, and Objective-C++. Like function types, the Block type is a pair consisting of a result value type and a list of parameter types very similar to a function type. Blocks are intended to be used much like functions with the key distinction being that in addition to executable code they also contain various variable bindings to automatic (stack) or managed (heap) memory.
-
-The abstract declarator int (^)(char, float) describes a reference to a Block that, when invoked, takes two parameters, the first of type char and the second of type float, and returns a value of type int.  The Block referenced is of opaque data that may reside in automatic (stack) memory, global memory, or heap memory.
-
-
-Block Variable Declarations
-
-A variable with Block type is declared using function pointer style notation substituting ^ for *. The following are valid Block variable declarations:
-    void (^blockReturningVoidWithVoidArgument)(void);
-    int (^blockReturningIntWithIntAndCharArguments)(int, char);
-    void (^arrayOfTenBlocksReturningVoidWithIntArgument[10])(int);
-
-Variadic ... arguments are supported. [variadic.c]  A Block that takes no arguments must specify void in the argument list [voidarg.c].  An empty parameter list does not represent, as K&R provide, an unspecified argument list.  Note: both gcc and clang support K&R style as a convenience.
-
-A Block reference may be cast to a pointer of arbitrary type and vice versa. [cast.c]  A Block reference may not be dereferenced via the pointer dereference operator *, and thus a Block's size may not be computed at compile time. [sizeof.c]
-
-
-Block Literal Expressions
-
-A Block literal expression produces a reference to a Block. It is introduced by the use of the ^ token as a unary operator.  
-    Block_literal_expression ::=   ^ block_decl compound_statement_body
-    block_decl ::= 
-    block_decl ::= parameter_list
-    block_decl ::= type_expression
-
-...where type expression is extended to allow ^ as a Block reference (pointer) where * is allowed as a function reference (pointer).
-
-The following Block literal:
-    ^ void (void) { printf("hello world\n"); }
-
-...produces a reference to a Block with no arguments with no return value.
-
-The return type is optional and is inferred from the return statements. If the return statements return a value, they all must return a value of the same type. If there is no value returned the inferred type of the Block is void; otherwise it is the type of the return statement value.
-
-If the return type is omitted and the argument list is ( void ), the ( void ) argument list may also be omitted.
-
-So:
-    ^ ( void ) { printf("hello world\n"); }
-
-...and:
-    ^ { printf("hello world\n"); }
-
-...are exactly equivalent constructs for the same expression.
-
-The type_expression extends C expression parsing to accommodate Block reference declarations as it accommodates function pointer declarations.
-
-Given:
-    typedef int (*pointerToFunctionThatReturnsIntWithCharArg)(char);
-    pointerToFunctionThatReturnsIntWithCharArg functionPointer;
-
-    ^ pointerToFunctionThatReturnsIntWithCharArg (float x) { return functionPointer; }
-
-...and:
-    ^ int ((*)(float x))(char) { return functionPointer; }
-
-...are equivalent expressions, as is:
-
-    ^(float x) { return functionPointer; }
-
-[returnfunctionptr.c]
-
-The compound statement body establishes a new lexical scope within that of its parent. Variables used within the scope of the compound statement are bound to the Block in the normal manner with the exception of those in automatic (stack) storage. Thus one may access functions and global variables as one would expect, as well as static local variables. [testme]
-
-Local automatic (stack) variables referenced within the compound statement of a Block are imported and captured by the Block as const copies. The capture (binding) is performed at the time of the Block literal expression evaluation.
-
-The lifetime of variables declared in a Block is that of a function; each activation frame contains a new copy of variables declared within the local scope of the Block. Such variable declarations should be allowed anywhere [testme] rather than only when C99 parsing is requested, including for statements. [testme]
-
-Block literal expressions may occur within Block literal expressions (nest) and all variables captured by any nested blocks are implicitly also captured in the scopes of their enclosing Blocks.
-
-A Block literal expression may be used as the initialization value for Block variables at global or local static scope.
-
-
-The Invoke Operator
-
-Blocks are invoked using function call syntax with a list of expression parameters of types corresponding to the declaration and returning a result type also according to the declaration. Given:
-    int (^x)(char);
-    void (^z)(void);
-    int (^(*y))(char) = &x;
-
-...the following are all legal Block invocations:
-    x('a');
-    (*y)('a');
-    (true ? x : *y)('a')
-
-
-The Copy and Release Operations
-
-The compiler and runtime provide copy and release operations for Block references that create and, in matched use, release allocated storage for referenced Blocks.
-
-The copy operation Block_copy() is styled as a function that takes an arbitrary Block reference and returns a Block reference of the same type. The release operation, Block_release(), is styled as a function that takes an arbitrary Block reference and, if dynamically matched to a Block copy operation, allows recovery of the referenced allocated memory.
-
-
-The __block Storage Qualifier
-
-In addition to the new Block type we also introduce a new storage qualifier, __block, for local variables. [testme: a __block declaration within a block literal]  The __block storage qualifier is mutually exclusive to the existing local storage qualifiers auto, register, and static.[testme]  Variables qualified by __block act as if they were in allocated storage and this storage is automatically recovered after last use of said variable.  An implementation may choose an optimization where the storage is initially automatic and only "moved" to allocated (heap) storage upon a Block_copy of a referencing Block.  Such variables may be mutated as normal variables are.
-
-In the case where a __block variable is a Block one must assume that the __block variable resides in allocated storage and as such is assumed to reference a Block that is also in allocated storage (that it is the result of a Block_copy operation).  Despite this there is no provision to do a Block_copy or a Block_release if an implementation provides initial automatic storage for Blocks.  This is due to the inherent race condition of potentially several threads trying to update the shared variable and the need for synchronization around disposing of older values and copying new ones.  Such synchronization is beyond the scope of this language specification.
-
-
-Control Flow
-
-The compound statement of a Block is treated much like a function body with respect to control flow in that goto, break, and continue do not escape the Block.  Exceptions are treated "normally" in that when thrown they pop stack frames until a catch clause is found.
-
-
-Objective-C Extensions
-
-Objective-C extends the definition of a Block reference type to be that also of id.  A variable or expression of Block type may be messaged or used as a parameter wherever an id may be. The converse is also true. Block references may thus appear as properties and are subject to the assign, retain, and copy attribute logic that is reserved for objects.
-
-All Blocks are constructed to be Objective-C objects regardless of whether the Objective-C runtime is operational in the program or not. Blocks using automatic (stack) memory are objects and may be messaged, although they may not be assigned into __weak locations if garbage collection is enabled.
-
-Within a Block literal expression within a method definition references to instance variables are also imported into the lexical scope of the compound statement. These variables are implicitly qualified as references from self, and so self is imported as a const copy. The net effect is that  instance variables can be mutated.
-
-The Block_copy operator retains all objects held in variables of automatic storage referenced within the Block expression (or form strong references if running under garbage collection).  Object variables of __block storage type are assumed to hold normal pointers with no provision for retain and release messages.
-
-Foundation defines (and supplies) -copy and -release methods for Blocks.
-
-In the Objective-C and Objective-C++ languages, we allow the __weak specifier for __block variables of object type.  If garbage collection is not enabled, this qualifier causes these variables to be kept without retain messages being sent. This knowingly leads to dangling pointers if the Block (or a copy) outlives the lifetime of this object.
-
-In garbage collected environments, the __weak variable is set to nil when the object it references is collected, as long as the __block variable resides in the heap (either by default or via Block_copy()).  The initial Apple implementation does in fact start __block variables on the stack and migrate them to the heap only as a result of a Block_copy() operation.
-
-It is a runtime error to attempt to assign a reference to a stack-based Block into any storage marked __weak, including __weak __block variables.
-
-
-C++ Extensions
-
-Block literal expressions within functions are extended to allow const use of C++ objects, pointers, or references held in automatic storage.
-
-For example, given class Foo with member function fighter(void):
-      Foo foo;
-      Foo &fooRef = foo;
-      Foo *fooPtr = &foo;
-
-...a Block that used foo would import the variables as const variations:
-       const Foo block_foo = foo;	// const copy constructor
-       const Foo &block_fooRef = fooRef;
-       const Foo *block_fooPtr = fooPtr;
-
-Stack-local objects are copied into a Block via a copy const constructor.  If no such constructor exists, it is considered an error to reference such objects from within the Block compound statements. A destructor is run as control leaves the compound statement that contains the Block literal expression.
-
-If a Block originates on the stack, a const copy constructor of the stack-based Block const copy is performed when a Block_copy operation is called; when the last Block_release (or subsequently GC) occurs, a destructor is run on the heap copy.
-
-Variables declared as residing in __block storage may be initially allocated in the heap or may first appear on the stack and be copied to the heap as a result of a Block_copy() operation. When copied from the stack, a normal copy constructor is used to initialize the heap-based version from the original stack version. The destructor for a const copied object is run at the normal end of scope. The destructor for any initial stack based version is also called at normal end of scope.
-
-Within a member function, access to member functions and variables is done via an implicit const copy of a this pointer.
-
-Member variables that are Blocks may not be overloaded by the types of their arguments.
-

File docs/DriverArchitecture.png

Removed
Old image

File docs/DriverInternals.html

-<html>
-  <head>
-    <title>Clang Driver Manual</title>
-    <link type="text/css" rel="stylesheet" href="../menu.css" />
-    <link type="text/css" rel="stylesheet" href="../content.css" />
-    <style type="text/css">
-      td {
-      vertical-align: top;
-      }
-    </style>
-  </head>
-  <body>
-
-    <!--#include virtual="../menu.html.incl"-->
-
-    <div id="content">
-
-      <h1>Driver Design &amp; Internals</h1>
-
-      <ul>
-        <li><a href="#intro">Introduction</a></li>
-        <li><a href="#features">Features and Goals</a></li>
-        <ul>
-          <li><a href="#gcccompat">GCC Compatibility</a></li>
-          <li><a href="#components">Flexible</a></li>
-          <li><a href="#performance">Low Overhead</a></li>
-          <li><a href="#simple">Simple</a></li>
-        </ul>
-        <li><a href="#design">Design</a></li>
-        <ul>
-          <li><a href="#int_intro">Internals Introduction</a></li>
-          <li><a href="#int_overview">Design Overview</a></li>
-          <li><a href="#int_notes">Additional Notes</a></li>
-          <ul>
-            <li><a href="#int_compilation">The Compilation Object</a></li>
-            <li><a href="#int_unified_parsing">Unified Parsing &amp; Pipelining</a></li>
-            <li><a href="#int_toolchain_translation">ToolChain Argument Translation</a></li>
-            <li><a href="#int_unused_warnings">Unused Argument Warnings</a></li>
-          </ul>
-          <li><a href="#int_gcc_concepts">Relation to GCC Driver Concepts</a></li>
-        </ul>
-      </ul>
-
-
-      <!-- ======================================================================= -->
-      <h2 id="intro">Introduction</h2>
-      <!-- ======================================================================= -->
-
-      <p>This document describes the Clang driver. The purpose of this
-        document is to describe both the motivation and design goals
-        for the driver, as well as details of the internal
-        implementation.</p>
-
-      <!-- ======================================================================= -->
-      <h2 id="features">Features and Goals</h2>
-      <!-- ======================================================================= -->
-
-      <p>The Clang driver is intended to be a production quality
-        compiler driver providing access to the Clang compiler and
-        tools, with a command line interface which is compatible with
-        the gcc driver.</p>
-
-      <p>Although the driver is part of and driven by the Clang
-        project, it is logically a separate tool which shares many of
-        the same goals as Clang:</p>
-
-      <p><b>Features</b>:</p>
-      <ul>
-        <li><a href="#gcccompat">GCC Compatibility</a></li>
-        <li><a href="#components">Flexible</a></li>
-        <li><a href="#performance">Low Overhead</a></li>
-        <li><a href="#simple">Simple</a></li>
-      </ul>
-
-      <!--=======================================================================-->
-      <h3 id="gcccompat">GCC Compatibility</h3>
-      <!--=======================================================================-->
-
-      <p>The number one goal of the driver is to ease the adoption of
-        Clang by allowing users to drop Clang into a build system
-        which was designed to call GCC. Although this makes the driver
-        much more complicated than might otherwise be necessary, we
-        decided that being very compatible with the gcc command line
-        interface was worth it in order to allow users to quickly test
-        clang on their projects.</p>
-
-      <!--=======================================================================-->
-      <h3 id="components">Flexible</h3>
-      <!--=======================================================================-->
-
-      <p>The driver was designed to be flexible and easily accomodate
-        new uses as we grow the clang and LLVM infrastructure. As one
-        example, the driver can easily support the introduction of
-        tools which have an integrated assembler; something we hope to
-        add to LLVM in the future.</p>
-
-      <p>Similarly, most of the driver functionality is kept in a
-        library which can be used to build other tools which want to
-        implement or accept a gcc like interface. </p>
-
-      <!--=======================================================================-->
-      <h3 id="performance">Low Overhead</h3>
-      <!--=======================================================================-->
-
-      <p>The driver should have as little overhead as possible. In
-        practice, we found that the gcc driver by itself incurred a
-        small but meaningful overhead when compiling many small
-        files. The driver doesn't do much work compared to a
-        compilation, but we have tried to keep it as efficient as
-        possible by following a few simple principles:</p>
-      <ul>
-        <li>Avoid memory allocation and string copying when
-          possible.</li>
-
-        <li>Don't parse arguments more than once.</li>
-
-        <li>Provide a few simple interfaces for efficiently searching
-          arguments.</li>
-      </ul>
-
-      <!--=======================================================================-->
-      <h3 id="simple">Simple</h3>
-      <!--=======================================================================-->
-
-      <p>Finally, the driver was designed to be "as simple as
-        possible", given the other goals. Notably, trying to be
-        completely compatible with the gcc driver adds a significant
-        amount of complexity. However, the design of the driver
-        attempts to mitigate this complexity by dividing the process
-        into a number of independent stages instead of a single
-        monolithic task.</p>
-
-      <!-- ======================================================================= -->
-      <h2 id="design">Internal Design and Implementation</h2>
-      <!-- ======================================================================= -->
-
-      <ul>
-        <li><a href="#int_intro">Internals Introduction</a></li>
-        <li><a href="#int_overview">Design Overview</a></li>
-        <li><a href="#int_notes">Additional Notes</a></li>
-        <li><a href="#int_gcc_concepts">Relation to GCC Driver Concepts</a></li>
-      </ul>
-
-      <!--=======================================================================-->
-      <h3><a name="int_intro">Internals Introduction</a></h3>
-      <!--=======================================================================-->
-
-      <p>In order to satisfy the stated goals, the driver was designed
-        to completely subsume the functionality of the gcc executable;
-        that is, the driver should not need to delegate to gcc to
-        perform subtasks. On Darwin, this implies that the Clang
-        driver also subsumes the gcc driver-driver, which is used to
-        implement support for building universal images (binaries and
-        object files). This also implies that the driver should be
-        able to call the language specific compilers (e.g. cc1)
-        directly, which means that it must have enough information to
-        forward command line arguments to child processes
-        correctly.</p>
-
-      <!--=======================================================================-->
-      <h3><a name="int_overview">Design Overview</a></h3>
-      <!--=======================================================================-->
-
-      <p>The diagram below shows the significant components of the
-        driver architecture and how they relate to one another. The
-        orange components represent concrete data structures built by
-        the driver, the green components indicate conceptually
-        distinct stages which manipulate these data structures, and
-        the blue components are important helper classes. </p>
-
-      <center>
-        <a href="DriverArchitecture.png" alt="Driver Architecture Diagram">
-          <img width=400 src="DriverArchitecture.png">
-        </a>
-      </center>
-
-      <!--=======================================================================-->
-      <h3><a name="int_stages">Driver Stages</a></h3>
-      <!--=======================================================================-->
-
-      <p>The driver functionality is conceptually divided into five stages:</p>
-
-      <ol>
-        <li>
-          <b>Parse: Option Parsing</b>
-
-          <p>The command line argument strings are decomposed into
-            arguments (<tt>Arg</tt> instances). The driver expects to
-            understand all available options, although there is some
-            facility for just passing certain classes of options
-            through (like <tt>-Wl,</tt>).</p>
-
-          <p>Each argument corresponds to exactly one
-            abstract <tt>Option</tt> definition, which describes how
-            the option is parsed along with some additional
-            metadata. The Arg instances themselves are lightweight and
-            merely contain enough information for clients to determine
-            which option they correspond to and their values (if they
-            have additional parameters).</p>
-
-          <p>For example, a command line like "-Ifoo -I foo" would
-            parse to two Arg instances (a JoinedArg and a SeparateArg
-            instance), but each would refer to the same Option.</p>
-
-          <p>Options are lazily created in order to avoid populating
-            all Option classes when the driver is loaded. Most of the
-            driver code only needs to deal with options by their
-            unique ID (e.g., <tt>options::OPT_I</tt>),</p>
-
-          <p>Arg instances themselves do not generally store the
-            values of parameters. In many cases, this would
-            simply result in creating unnecessary string
-            copies. Instead, Arg instances are always embedded inside
-            an ArgList structure, which contains the original vector
-            of argument strings. Each Arg itself only needs to contain
-            an index into this vector instead of storing its values
-            directly.</p>
-
-          <p>The clang driver can dump the results of this
-            stage using the <tt>-ccc-print-options</tt> flag (which
-            must preceed any actual command line arguments). For
-            example:</p>
-          <pre>
-            $ <b>clang -ccc-print-options -Xarch_i386 -fomit-frame-pointer -Wa,-fast -Ifoo -I foo t.c</b>
-            Option 0 - Name: "-Xarch_", Values: {"i386", "-fomit-frame-pointer"}
-            Option 1 - Name: "-Wa,", Values: {"-fast"}
-            Option 2 - Name: "-I", Values: {"foo"}
-            Option 3 - Name: "-I", Values: {"foo"}
-            Option 4 - Name: "&lt;input&gt;", Values: {"t.c"}
-          </pre>
-
-          <p>After this stage is complete the command line should be
-            broken down into well defined option objects with their
-            appropriate parameters.  Subsequent stages should rarely,
-            if ever, need to do any string processing.</p>
-        </li>
-
-        <li>
-          <b>Pipeline: Compilation Job Construction</b>
-
-          <p>Once the arguments are parsed, the tree of subprocess
-            jobs needed for the desired compilation sequence are
-            constructed. This involves determining the input files and
-            their types, what work is to be done on them (preprocess,
-            compile, assemble, link, etc.), and constructing a list of
-            Action instances for each task. The result is a list of
-            one or more top-level actions, each of which generally
-            corresponds to a single output (for example, an object or
-            linked executable).</p>
-
-          <p>The majority of Actions correspond to actual tasks,
-            however there are two special Actions. The first is
-            InputAction, which simply serves to adapt an input
-            argument for use as an input to other Actions. The second
-            is BindArchAction, which conceptually alters the
-            architecture to be used for all of its input Actions.</p>
-
-          <p>The clang driver can dump the results of this
-            stage using the <tt>-ccc-print-phases</tt> flag. For
-            example:</p>
-          <pre>
-            $ <b>clang -ccc-print-phases -x c t.c -x assembler t.s</b>
-            0: input, "t.c", c
-            1: preprocessor, {0}, cpp-output
-            2: compiler, {1}, assembler
-            3: assembler, {2}, object
-            4: input, "t.s", assembler
-            5: assembler, {4}, object
-            6: linker, {3, 5}, image
-          </pre>
-          <p>Here the driver is constructing seven distinct actions,
-            four to compile the "t.c" input into an object file, two to
-            assemble the "t.s" input, and one to link them together.</p>
-
-          <p>A rather different compilation pipeline is shown here; in
-            this example there are two top level actions to compile
-            the input files into two separate object files, where each
-            object file is built using <tt>lipo</tt> to merge results
-            built for two separate architectures.</p>
-          <pre>
-            $ <b>clang -ccc-print-phases -c -arch i386 -arch x86_64 t0.c t1.c</b>
-            0: input, "t0.c", c
-            1: preprocessor, {0}, cpp-output
-            2: compiler, {1}, assembler
-            3: assembler, {2}, object
-            4: bind-arch, "i386", {3}, object
-            5: bind-arch, "x86_64", {3}, object
-            6: lipo, {4, 5}, object
-            7: input, "t1.c", c
-            8: preprocessor, {7}, cpp-output
-            9: compiler, {8}, assembler
-            10: assembler, {9}, object
-            11: bind-arch, "i386", {10}, object
-            12: bind-arch, "x86_64", {10}, object
-            13: lipo, {11, 12}, object
-          </pre>
-
-          <p>After this stage is complete the compilation process is
-            divided into a simple set of actions which need to be
-            performed to produce intermediate or final outputs (in
-            some cases, like <tt>-fsyntax-only</tt>, there is no
-            "real" final output). Phases are well known compilation
-            steps, such as "preprocess", "compile", "assemble",
-            "link", etc.</p>
-        </li>
-
-        <li>
-          <b>Bind: Tool &amp; Filename Selection</b>
-
-          <p>This stage (in conjunction with the Translate stage)
-            turns the tree of Actions into a list of actual subprocess
-            to run. Conceptually, the driver performs a top down
-            matching to assign Action(s) to Tools. The ToolChain is
-            responsible for selecting the tool to perform a particular
-            action; once selected the driver interacts with the tool
-            to see if it can match additional actions (for example, by
-            having an integrated preprocessor).
-
-          <p>Once Tools have been selected for all actions, the driver
-            determines how the tools should be connected (for example,
-            using an inprocess module, pipes, temporary files, or user
-            provided filenames). If an output file is required, the
-            driver also computes the appropriate file name (the suffix
-            and file location depend on the input types and options
-            such as <tt>-save-temps</tt>).
-
-          <p>The driver interacts with a ToolChain to perform the Tool
-            bindings. Each ToolChain contains information about all
-            the tools needed for compilation for a particular
-            architecture, platform, and operating system. A single
-            driver invocation may query multiple ToolChains during one
-            compilation in order to interact with tools for separate
-            architectures.</p>
-
-          <p>The results of this stage are not computed directly, but
-            the driver can print the results via
-            the <tt>-ccc-print-bindings</tt> option. For example:</p>
-          <pre>
-            $ <b>clang -ccc-print-bindings -arch i386 -arch ppc t0.c</b>
-            # "i386-apple-darwin9" - "clang", inputs: ["t0.c"], output: "/tmp/cc-Sn4RKF.s"
-            # "i386-apple-darwin9" - "darwin::Assemble", inputs: ["/tmp/cc-Sn4RKF.s"], output: "/tmp/cc-gvSnbS.o"
-            # "i386-apple-darwin9" - "darwin::Link", inputs: ["/tmp/cc-gvSnbS.o"], output: "/tmp/cc-jgHQxi.out"
-            # "ppc-apple-darwin9" - "gcc::Compile", inputs: ["t0.c"], output: "/tmp/cc-Q0bTox.s"
-            # "ppc-apple-darwin9" - "gcc::Assemble", inputs: ["/tmp/cc-Q0bTox.s"], output: "/tmp/cc-WCdicw.o"
-            # "ppc-apple-darwin9" - "gcc::Link", inputs: ["/tmp/cc-WCdicw.o"], output: "/tmp/cc-HHBEBh.out"
-            # "i386-apple-darwin9" - "darwin::Lipo", inputs: ["/tmp/cc-jgHQxi.out", "/tmp/cc-HHBEBh.out"], output: "a.out"
-          </pre>
-
-          <p>This shows the tool chain, tool, inputs and outputs which
-            have been bound for this compilation sequence. Here clang
-            is being used to compile t0.c on the i386 architecture and
-            darwin specific versions of the tools are being used to
-            assemble and link the result, but generic gcc versions of
-            the tools are being used on PowerPC.</p>
-        </li>
-
-        <li>
-          <b>Translate: Tool Specific Argument Translation</b>
-
-          <p>Once a Tool has been selected to perform a particular
-            Action, the Tool must construct concrete Jobs which will be
-            executed during compilation. The main work is in translating
-            from the gcc style command line options to whatever options
-            the subprocess expects.</p>
-
-          <p>Some tools, such as the assembler, only interact with a
-            handful of arguments and just determine the path of the
-            executable to call and pass on their input and output
-            arguments. Others, like the compiler or the linker, may
-            translate a large number of arguments in addition.</p>
-
-          <p>The ArgList class provides a number of simple helper
-            methods to assist with translating arguments; for example,
-            to pass on only the last of arguments corresponding to some
-            option, or all arguments for an option.</p>
-
-          <p>The result of this stage is a list of Jobs (executable
-            paths and argument strings) to execute.</p>
-        </li>
-
-        <li>
-          <b>Execute</b>
-          <p>Finally, the compilation pipeline is executed. This is
-            mostly straightforward, although there is some interaction
-            with options
-            like <tt>-pipe</tt>, <tt>-pass-exit-codes</tt>
-            and <tt>-time</tt>.</p>
-        </li>
-
-      </ol>
-
-      <!--=======================================================================-->
-      <h3><a name="int_notes">Additional Notes</a></h3>
-      <!--=======================================================================-->
-
-      <h4 id="int_compilation">The Compilation Object</h4>
-
-      <p>The driver constructs a Compilation object for each set of
-        command line arguments. The Driver itself is intended to be
-        invariant during construction of a Compilation; an IDE should be
-        able to construct a single long lived driver instance to use
-        for an entire build, for example.</p>
-
-      <p>The Compilation object holds information that is particular
-        to each compilation sequence. For example, the list of used
-        temporary files (which must be removed once compilation is
-        finished) and result files (which should be removed if
-        compilation files).</p>
-
-      <h4 id="int_unified_parsing">Unified Parsing &amp; Pipelining</h4>
-
-      <p>Parsing and pipelining both occur without reference to a
-        Compilation instance. This is by design; the driver expects that
-        both of these phases are platform neutral, with a few very well
-        defined exceptions such as whether the platform uses a driver
-        driver.</p>
-
-      <h4 id="int_toolchain_translation">ToolChain Argument Translation</h4>
-
-      <p>In order to match gcc very closely, the clang driver
-        currently allows tool chains to perform their own translation of
-        the argument list (into a new ArgList data structure). Although
-        this allows the clang driver to match gcc easily, it also makes
-        the driver operation much harder to understand (since the Tools
-        stop seeing some arguments the user provided, and see new ones
-        instead).</p>
-
-      <p>For example, on Darwin <tt>-gfull</tt> gets translated into two
-        separate arguments, <tt>-g</tt>
-        and <tt>-fno-eliminate-unused-debug-symbols</tt>. Trying to write Tool
-        logic to do something with <tt>-gfull</tt> will not work, because Tool
-        argument translation is done after the arguments have been
-        translated.</p>
-
-      <p>A long term goal is to remove this tool chain specific
-        translation, and instead force each tool to change its own logic
-        to do the right thing on the untranslated original arguments.</p>
-
-      <h4 id="int_unused_warnings">Unused Argument Warnings</h4>
-      <p>The driver operates by parsing all arguments but giving Tools
-        the opportunity to choose which arguments to pass on. One
-        downside of this infrastructure is that if the user misspells
-        some option, or is confused about which options to use, some
-        command line arguments the user really cared about may go
-        unused. This problem is particularly important when using
-        clang as a compiler, since the clang compiler does not support
-        anywhere near all the options that gcc does, and we want to make
-        sure users know which ones are being used.</p>
-
-      <p>To support this, the driver maintains a bit associated with
-        each argument of whether it has been used (at all) during the
-        compilation. This bit usually doesn't need to be set by hand,
-        as the key ArgList accessors will set it automatically.</p>
-
-      <p>When a compilation is successful (there are no errors), the
-        driver checks the bit and emits an "unused argument" warning for
-        any arguments which were never accessed. This is conservative
-        (the argument may not have been used to do what the user wanted)
-        but still catches the most obvious cases.</p>
-
-      <!--=======================================================================-->
-      <h3><a name="int_gcc_concepts">Relation to GCC Driver Concepts</a></h3>
-      <!--=======================================================================-->
-
-      <p>For those familiar with the gcc driver, this section provides
-        a brief overview of how things from the gcc driver map to the
-        clang driver.</p>
-
-      <ul>
-        <li>
-          <b>Driver Driver</b>
-          <p>The driver driver is fully integrated into the clang
-            driver. The driver simply constructs additional Actions to
-            bind the architecture during the <i>Pipeline</i>
-            phase. The tool chain specific argument translation is
-            responsible for handling <tt>-Xarch_</tt>.</p>
-
-          <p>The one caveat is that this approach
-            requires <tt>-Xarch_</tt> not be used to alter the
-            compilation itself (for example, one cannot
-            provide <tt>-S</tt> as an <tt>-Xarch_</tt> argument). The
-            driver attempts to reject such invocations, and overall
-            there isn't a good reason to abuse <tt>-Xarch_</tt> to
-            that end in practice.</p>
-
-          <p>The upside is that the clang driver is more efficient and
-            does little extra work to support universal builds. It also
-            provides better error reporting and UI consistency.</p>
-        </li>
-
-        <li>
-          <b>Specs</b>
-          <p>The clang driver has no direct correspondant for
-            "specs". The majority of the functionality that is
-            embedded in specs is in the Tool specific argument
-            translation routines. The parts of specs which control the
-            compilation pipeline are generally part of
-            the <ii>Pipeline</ii> stage.</p>
-        </li>
-
-        <li>
-          <b>Toolchains</b>
-          <p>The gcc driver has no direct understanding of tool
-            chains. Each gcc binary roughly corresponds to the
-            information which is embedded inside a single
-            ToolChain.</p>
-
-          <p>The clang driver is intended to be portable and support
-            complex compilation environments. All platform and tool
-            chain specific code should be protected behind either
-            abstract or well defined interfaces (such as whether the
-            platform supports use as a driver driver).</p>
-        </li>
-      </ul>
-    </div>
-  </body>
-</html>

File docs/InternalsManual.html

-<html>
-<head>
-<title>"Clang" CFE Internals Manual</title>
-<link type="text/css" rel="stylesheet" href="../menu.css" />
-<link type="text/css" rel="stylesheet" href="../content.css" />
-<style type="text/css">
-td {
-	vertical-align: top;
-}
-</style>
-</head>
-<body>
-
-<!--#include virtual="../menu.html.incl"-->
-
-<div id="content">
-
-<h1>"Clang" CFE Internals Manual</h1>
-
-<ul>
-<li><a href="#intro">Introduction</a></li>
-<li><a href="#libsystem">LLVM System and Support Libraries</a></li>
-<li><a href="#libbasic">The Clang 'Basic' Library</a>
-  <ul>
-  <li><a href="#Diagnostics">The Diagnostics Subsystem</a></li>
-  <li><a href="#SourceLocation">The SourceLocation and SourceManager
-      classes</a></li>
-  </ul>
-</li>
-<li><a href="#libdriver">The Driver Library</a>
-  <ul>
-  </ul>
-</li>
-<li><a href="#pch">Precompiled Headers</a>
-<li><a href="#libfrontend">The Frontend Library</a>
-  <ul>
-  </ul>
-</li>
-<li><a href="#liblex">The Lexer and Preprocessor Library</a>
-  <ul>
-  <li><a href="#Token">The Token class</a></li>
-  <li><a href="#Lexer">The Lexer class</a></li>
-  <li><a href="#AnnotationToken">Annotation Tokens</a></li>
-  <li><a href="#TokenLexer">The TokenLexer class</a></li>
-  <li><a href="#MultipleIncludeOpt">The MultipleIncludeOpt class</a></li>
-  </ul>
-</li>
-<li><a href="#libparse">The Parser Library</a>
-  <ul>
-  </ul>
-</li>
-<li><a href="#libast">The AST Library</a>
-  <ul>
-  <li><a href="#Type">The Type class and its subclasses</a></li>
-  <li><a href="#QualType">The QualType class</a></li>
-  <li><a href="#DeclarationName">Declaration names</a></li>
-  <li><a href="#DeclContext">Declaration contexts</a>
-    <ul>
-      <li><a href="#Redeclarations">Redeclarations and Overloads</a></li>
-      <li><a href="#LexicalAndSemanticContexts">Lexical and Semantic
-      Contexts</a></li>
-      <li><a href="#TransparentContexts">Transparent Declaration Contexts</a></li>
-      <li><a href="#MultiDeclContext">Multiply-Defined Declaration Contexts</a></li>
-    </ul>
-  </li>
-  <li><a href="#CFG">The CFG class</a></li>
-  <li><a href="#Constants">Constant Folding in the Clang AST</a></li>
-  </ul>
-</li>
-<li><a href="libIndex.html">The Index Library</a></li>
-</ul>
-
-
-<!-- ======================================================================= -->
-<h2 id="intro">Introduction</h2>
-<!-- ======================================================================= -->
-
-<p>This document describes some of the more important APIs and internal design
-decisions made in the Clang C front-end.  The purpose of this document is to
-both capture some of this high level information and also describe some of the
-design decisions behind it.  This is meant for people interested in hacking on
-Clang, not for end-users.  The description below is categorized by
-libraries, and does not describe any of the clients of the libraries.</p>
-
-<!-- ======================================================================= -->
-<h2 id="libsystem">LLVM System and Support Libraries</h2>
-<!-- ======================================================================= -->
-
-<p>The LLVM libsystem library provides the basic Clang system abstraction layer,
-which is used for file system access.  The LLVM libsupport library provides many
-underlying libraries and <a 
-href="http://llvm.org/docs/ProgrammersManual.html">data-structures</a>,
- including command line option
-processing and various containers.</p>
-
-<!-- ======================================================================= -->
-<h2 id="libbasic">The Clang 'Basic' Library</h2>
-<!-- ======================================================================= -->
-
-<p>This library certainly needs a better name.  The 'basic' library contains a
-number of low-level utilities for tracking and manipulating source buffers,
-locations within the source buffers, diagnostics, tokens, target abstraction,
-and information about the subset of the language being compiled for.</p>
-
-<p>Part of this infrastructure is specific to C (such as the TargetInfo class),
-other parts could be reused for other non-C-based languages (SourceLocation,
-SourceManager, Diagnostics, FileManager).  When and if there is future demand
-we can figure out if it makes sense to introduce a new library, move the general
-classes somewhere else, or introduce some other solution.</p>
-
-<p>We describe the roles of these classes in order of their dependencies.</p>
-
-
-<!-- ======================================================================= -->
-<h3 id="Diagnostics">The Diagnostics Subsystem</h3>
-<!-- ======================================================================= -->
-
-<p>The Clang Diagnostics subsystem is an important part of how the compiler
-communicates with the human.  Diagnostics are the warnings and errors produced
-when the code is incorrect or dubious.  In Clang, each diagnostic produced has
-(at the minimum) a unique ID, a <a href="#SourceLocation">SourceLocation</a> to
-"put the caret", an English translation associated with it, and a severity (e.g.
-<tt>WARNING</tt> or <tt>ERROR</tt>).  They can also optionally include a number
-of arguments to the dianostic (which fill in "%0"'s in the string) as well as a
-number of source ranges that related to the diagnostic.</p>
-
-<p>In this section, we'll be giving examples produced by the Clang command line
-driver, but diagnostics can be <a href="#DiagnosticClient">rendered in many
-different ways</a> depending on how the DiagnosticClient interface is
-implemented.  A representative example of a diagonstic is:</p>
-
-<pre>
-t.c:38:15: error: invalid operands to binary expression ('int *' and '_Complex float')
-   <font color="darkgreen">P = (P-42) + Gamma*4;</font>
-       <font color="blue">~~~~~~ ^ ~~~~~~~</font>
-</pre>
-
-<p>In this example, you can see the English translation, the severity (error),
-you can see the source location (the caret ("^") and file/line/column info),
-the source ranges "~~~~", arguments to the diagnostic ("int*" and "_Complex
-float").  You'll have to believe me that there is a unique ID backing the
-diagnostic :).</p>
-
-<p>Getting all of this to happen has several steps and involves many moving
-pieces, this section describes them and talks about best practices when adding
-a new diagnostic.</p>
-
-<!-- ============================== -->
-<h4>The Diagnostic*Kinds.def files</h4>
-<!-- ============================== -->
-
-<p>Diagnostics are created by adding an entry to one of the <tt>
-clang/Basic/Diagnostic*Kinds.def</tt> files, depending on what library will
-be using it.  This file encodes the unique ID of the 
-diagnostic (as an enum, the first argument), the severity of the diagnostic
-(second argument) and the English translation + format string.</p>
-
-<p>There is little sanity with the naming of the unique ID's right now.  Some
-start with err_, warn_, ext_ to encode the severity into the name.  Since the
-enum is referenced in the C++ code that produces the diagnostic, it is somewhat
-useful for it to be reasonably short.</p>
-
-<p>The severity of the diagnostic comes from the set {<tt>NOTE</tt>,
-<tt>WARNING</tt>, <tt>EXTENSION</tt>, <tt>EXTWARN</tt>, <tt>ERROR</tt>}.  The
-<tt>ERROR</tt> severity is used for diagnostics indicating the program is never
-acceptable under any circumstances.  When an error is emitted, the AST for the
-input code may not be fully built.  The <tt>EXTENSION</tt> and <tt>EXTWARN</tt>
-severities are used for extensions to the language that Clang accepts.  This
-means that Clang fully understands and can represent them in the AST, but we
-produce diagnostics to tell the user their code is non-portable.  The difference
-is that the former are ignored by default, and the later warn by default.  The
-<tt>WARNING</tt> severity is used for constructs that are valid in the currently
-selected source language but that are dubious in some way.  The <tt>NOTE</tt>
-level is used to staple more information onto previous diagnostics.</p>
-
-<p>These <em>severities</em> are mapped into a smaller set (the
-Diagnostic::Level enum, {<tt>Ignored</tt>, <tt>Note</tt>, <tt>Warning</tt>,
-<tt>Error</tt>, <tt>Fatal</tt> }) of output <em>levels</em> by the diagnostics
-subsystem based on various configuration options.  Clang internally supports a
-fully fine grained mapping mechanism that allows you to map almost any
-diagnostic to the output level that you want.  The only diagnostics that cannot
-be mapped are <tt>NOTE</tt>s, which always follow the severity of the previously
-emitted diagnostic and <tt>ERROR</tt>s, which can only be mapped to
-<tt>Fatal</tt> (it is not possible to turn an error into a warning,
-for example).</p>
-
-<p>Diagnostic mappings are used in many ways.  For example, if the user
-specifies <tt>-pedantic</tt>, <tt>EXTENSION</tt> maps to <tt>Warning</tt>, if
-they specify <tt>-pedantic-errors</tt>, it turns into <tt>Error</tt>.  This is
-used to implement options like <tt>-Wunused_macros</tt>, <tt>-Wundef</tt> etc.
-</p>
-
-<p>
-Mapping to <tt>Fatal</tt> should only be used for diagnostics that are
-considered so severe that error recovery won't be able to recover sensibly from
-them (thus spewing a ton of bogus errors).  One example of this class of error
-are failure to #include a file.
-</p>
-
-<!-- ================= -->
-<h4>The Format String</h4>
-<!-- ================= -->
-
-<p>The format string for the diagnostic is very simple, but it has some power.
-It takes the form of a string in English with markers that indicate where and
-how arguments to the diagnostic are inserted and formatted.  For example, here
-are some simple format strings:</p>
-
-<pre>
-  "binary integer literals are an extension"
-  "format string contains '\\0' within the string body"
-  "more '<b>%%</b>' conversions than data arguments"
-  "invalid operands to binary expression (<b>%0</b> and <b>%1</b>)"
-  "overloaded '<b>%0</b>' must be a <b>%select{unary|binary|unary or binary}2</b> operator"
-       " (has <b>%1</b> parameter<b>%s1</b>)"
-</pre>
-
-<p>These examples show some important points of format strings.  You can use any
-   plain ASCII character in the diagnostic string except "%" without a problem,
-   but these are C strings, so you have to use and be aware of all the C escape
-   sequences (as in the second example).  If you want to produce a "%" in the
-   output, use the "%%" escape sequence, like the third diagnostic.  Finally,
-   Clang uses the "%...[digit]" sequences to specify where and how arguments to
-   the diagnostic are formatted.</p>
-   
-<p>Arguments to the diagnostic are numbered according to how they are specified
-   by the C++ code that <a href="#producingdiag">produces them</a>, and are
-   referenced by <tt>%0</tt> .. <tt>%9</tt>.  If you have more than 10 arguments
-   to your diagnostic, you are doing something wrong :).  Unlike printf, there
-   is no requirement that arguments to the diagnostic end up in the output in
-   the same order as they are specified, you could have a format string with
-   <tt>"%1 %0"</tt> that swaps them, for example.  The text in between the
-   percent and digit are formatting instructions.  If there are no instructions,
-   the argument is just turned into a string and substituted in.</p>
-
-<p>Here are some "best practices" for writing the English format string:</p>
-
-<ul>
-<li>Keep the string short.  It should ideally fit in the 80 column limit of the
-    <tt>DiagnosticKinds.def</tt> file.  This avoids the diagnostic wrapping when
-    printed, and forces you to think about the important point you are conveying
-    with the diagnostic.</li>
-<li>Take advantage of location information.  The user will be able to see the
-    line and location of the caret, so you don't need to tell them that the
-    problem is with the 4th argument to the function: just point to it.</li>
-<li>Do not capitalize the diagnostic string, and do not end it with a
-    period.</li>
-<li>If you need to quote something in the diagnostic string, use single
-    quotes.</li>
-</ul>
-
-<p>Diagnostics should never take random English strings as arguments: you
-shouldn't use <tt>"you have a problem with %0"</tt> and pass in things like
-<tt>"your argument"</tt> or <tt>"your return value"</tt> as arguments. Doing
-this prevents <a href="translation">translating</a> the Clang diagnostics to
-other languages (because they'll get random English words in their otherwise
-localized diagnostic).  The exceptions to this are C/C++ language keywords
-(e.g. auto, const, mutable, etc) and C/C++ operators (<tt>/=</tt>).  Note
-that things like "pointer" and "reference" are not keywords.  On the other
-hand, you <em>can</em> include anything that comes from the user's source code,
-including variable names, types, labels, etc.  The 'select' format can be 
-used to achieve this sort of thing in a localizable way, see below.</p>
-
-<!-- ==================================== -->
-<h4>Formatting a Diagnostic Argument</a></h4>
-<!-- ==================================== -->
-
-<p>Arguments to diagnostics are fully typed internally, and come from a couple
-different classes: integers, types, names, and random strings.  Depending on
-the class of the argument, it can be optionally formatted in different ways.
-This gives the DiagnosticClient information about what the argument means
-without requiring it to use a specific presentation (consider this MVC for
-Clang :).</p>
-
-<p>Here are the different diagnostic argument formats currently supported by
-Clang:</p>
-
-<table>
-<tr><td colspan="2"><b>"s" format</b></td></tr>
-<tr><td>Example:</td><td><tt>"requires %1 parameter%s1"</tt></td></tr>
-<tr><td>Class:</td><td>Integers</td></tr>
-<tr><td>Description:</td><td>This is a simple formatter for integers that is
-    useful when producing English diagnostics.  When the integer is 1, it prints
-    as nothing.  When the integer is not 1, it prints as "s".  This allows some
-    simple grammatical forms to be to be handled correctly, and eliminates the
-    need to use gross things like <tt>"requires %1 parameter(s)"</tt>.</td></tr>
-
-<tr><td colspan="2"><b>"select" format</b></td></tr>
-<tr><td>Example:</td><td><tt>"must be a %select{unary|binary|unary or binary}2
-     operator"</tt></td></tr>
-<tr><td>Class:</td><td>Integers</td></tr>
-<tr><td>Description:</td><td><p>This format specifier is used to merge multiple
-    related diagnostics together into one common one, without requiring the
-    difference to be specified as an English string argument.  Instead of
-    specifying the string, the diagnostic gets an integer argument and the
-    format string selects the numbered option.  In this case, the "%2" value
-    must be an integer in the range [0..2].  If it is 0, it prints 'unary', if
-    it is 1 it prints 'binary' if it is 2, it prints 'unary or binary'.  This
-    allows other language translations to substitute reasonable words (or entire
-    phrases) based on the semantics of the diagnostic instead of having to do
-    things textually.</p>
-    <p>The selected string does undergo formatting.</p></td></tr>
-
-<tr><td colspan="2"><b>"plural" format</b></td></tr>
-<tr><td>Example:</td><td><tt>"you have %1 %plural{1:mouse|:mice}1 connected to
-    your computer"</tt></td></tr>
-<tr><td>Class:</td><td>Integers</td></tr>
-<tr><td>Description:</td><td><p>This is a formatter for complex plural forms.
-    It is designed to handle even the requirements of languages with very
-	complex plural forms, as many Baltic languages have. The argument consists
-	of a series of expression/form pairs, separated by ':', where the first form
-	whose expression evaluates to true is the result of the modifier.</p>
-	<p>An expression can be empty, in which case it is always true. See the
-	example at the top. Otherwise, it is a series of one or more numeric
-	conditions, separated by ','. If any condition matches, the expression
-	matches. Each numeric condition can take one of three forms.</p>
-	<ul>
-	    <li>number: A simple decimal number matches if the argument is the same
-		as the number. Example: <tt>"%plural{1:mouse|:mice}4"</tt></li>
-		<li>range: A range in square brackets matches if the argument is within
-		the range. Then range is inclusive on both ends. Example:
-		<tt>"%plural{0:none|1:one|[2,5]:some|:many}2"</tt></li>
-		<li>modulo: A modulo operator is followed by a number, and
-                equals sign and either a number or a range. The tests are the
-                same as for plain
-		numbers and ranges, but the argument is taken modulo the number first.
-		Example: <tt>"%plural{%100=0:even hundred|%100=[1,50]:lower half|:everything
-		else}1"</tt></li>
-	</ul>
-	<p>The parser is very unforgiving. A syntax error, even whitespace, will
-	abort, as will a failure to match the argument against any
-	expression.</p></td></tr>
-
-<tr><td colspan="2"><b>"ordinal" format</b></td></tr>
-<tr><td>Example:</td><td><tt>"ambiguity in %ordinal0 argument"</tt></td></tr>
-<tr><td>Class:</td><td>Integers</td></tr>
-<tr><td>Description:</td><td><p>This is a formatter which represents the
-    argument number as an ordinal:  the value <tt>1</tt> becomes <tt>1st</tt>,
-    <tt>3</tt> becomes <tt>3rd</tt>, and so on.  Values less than <tt>1</tt>
-    are not supported.</p>
-    <p>This formatter is currently hard-coded to use English ordinals.</p></td></tr>
-
-<tr><td colspan="2"><b>"objcclass" format</b></td></tr>
-<tr><td>Example:</td><td><tt>"method %objcclass0 not found"</tt></td></tr>
-<tr><td>Class:</td><td>DeclarationName</td></tr>
-<tr><td>Description:</td><td><p>This is a simple formatter that indicates the
-    DeclarationName corresponds to an Objective-C class method selector.  As
-    such, it prints the selector with a leading '+'.</p></td></tr>
-
-<tr><td colspan="2"><b>"objcinstance" format</b></td></tr>
-<tr><td>Example:</td><td><tt>"method %objcinstance0 not found"</tt></td></tr>
-<tr><td>Class:</td><td>DeclarationName</td></tr>
-<tr><td>Description:</td><td><p>This is a simple formatter that indicates the
-    DeclarationName corresponds to an Objective-C instance method selector.  As
-    such, it prints the selector with a leading '-'.</p></td></tr>
-
-<tr><td colspan="2"><b>"q" format</b></td></tr>
-<tr><td>Example:</td><td><tt>"candidate found by name lookup is %q0"</tt></td></tr>
-<tr><td>Class:</td><td>NamedDecl*</td></tr>
-<tr><td>Description</td><td><p>This formatter indicates that the fully-qualified name of the declaration should be printed, e.g., "std::vector" rather than "vector".</p></td></tr>
-    
-</table>
-
-<p>It is really easy to add format specifiers to the Clang diagnostics system,
-but they should be discussed before they are added.  If you are creating a lot
<