Commits

ken...@google.com@630680e5-0e50-0410-840e-4b1c322b438d  committed 9dc1961

Integrate recent changes from Google-internal code tree. See CHANGES.txt
for details.

  • Participants
  • Parent commits 6d17a8c

Comments (0)

Files changed (97)

-????-??-?? version 2.0.4:
+????-??-?? version 2.1.0:
 
   General
   * Repeated fields of primitive types (types other that string, group, and
   * Updated bundled Google Test to version 1.3.0.  Google Test is now bundled
     in its verbatim form as a nested autoconf package, so you can drop in any
     other version of Google Test if needed.
+  * optimize_for = SPEED is now the default, by popular demand.  Use
+    optimize_for = CODE_SIZE if code size is more important in your app.
+  * It is now an error to define a default value for a repeated field.
+    Previously, this was silently ignored (it had no effect on the generated
+    code).
+  * Fields can now be marked deprecated like:
+      optional int32 foo = 1 [deprecated = true];
+    Currently this does not have any actual effect, but in the future the code
+    generators may generate deprecation annotations in each language.
 
   protoc
   * --error_format=msvs option causes errors to be printed in Visual Studio
     format, which should allow them to be clicked on in the build log to go
-    directly to the error location. 
+    directly to the error location.
+  * The type name resolver will no longer resolve type names to fields.  For
+    example, this now works:
+      message Foo {}
+      message Bar {
+        optional int32 Foo = 1;
+        optional Foo baz = 2;
+      }
+    Previously, the type of "baz" would resolve to "Bar.Foo", and you'd get
+    an error because Bar.Foo is a field, not a type.  Now the type of "baz"
+    resolves to the message type Foo.  This change is unlikely to make a
+    difference to anyone who follows the Protocol Buffers style guide.
 
   C++
-  * UnknownFieldSet now supports STL-like iteration.
+  * Several optimizations, including but not limited to:
+    - Serialization, especially to flat arrays, is 10%-50% faster, possibly
+      more for small objects.
+    - Several descriptor operations which previously required locking no longer
+      do.
+    - Descriptors are now constructed lazily on first use, rather than at
+      process startup time.  This should save memory in programs which do not
+      use descriptors or reflection.
+    - UnknownFieldSet completely redesigned to be more efficient (especially in
+      terms of memory usage).
+    - Various optimizations to reduce code size (though the serialization speed
+      optimizations increased code size).
   * Message interface has method ParseFromBoundedZeroCopyStream() which parses
     a limited number of bytes from an input stream rather than parsing until
     EOF.
   * GzipInputStream and GzipOutputStream support reading/writing gzip- or
     zlib-compressed streams if zlib is available.
     (google/protobuf/io/gzip_stream.h)
-  * Generated constructors explicitly initialize all fields (to avoid warnings
-    with certain compiler settings).
+  * DescriptorPool::FindAllExtensions() and corresponding
+    DescriptorDatabase::FindAllExtensions() can be used to enumerate all
+    extensions of a given type.
+  * For each enum type Foo, protoc will generate functions:
+      const string& Foo_Name(Foo value);
+      bool Foo_Parse(const string& name, Foo* result);
+    The former returns the name of the enum constant corresponding to the given
+    value while the latter finds the value corresponding to a name.
+  * RepeatedField and RepeatedPtrField now have back-insertion iterators.
+  * String fields now have setters that take a char* and a size, in addition
+    to the existing ones that took char* or const string&.
+  * DescriptorPool::AllowUnknownDependencies() may be used to tell
+    DescriptorPool to create placeholder descriptors for unknown entities
+    referenced in a FileDescriptorProto.  This can allow you to parse a .proto
+    file without having access to other .proto files that it imports, for
+    example.
+  * Updated gtest to latest version.  The gtest package is now included as a
+    nested autoconf package, so it should be able to drop new versions into the
+    "gtest" subdirectory without modification.
 
   Java
   * Fixed bug where Message.mergeFrom(Message) failed to merge extensions.
     regex implementation (which unfortunately uses recursive backtracking
     rather than building an NFA).  Worked around by making use of possesive
     quantifiers.
+  * Generated service classes now also generate pure interfaces.  For a service
+    Foo, Foo.Interface is a pure interface containing all of the service's
+    defined methods.  Foo.newReflectiveService() can be called to wrap an
+    instance of this interface in a class that implements the generic
+    RpcService interface, which provides reflection support that is usually
+    needed by RPC server implementations.
+  * RPC interfaces now support blocking operation in addition to non-blocking.
+    The protocol compiler generates separate blocking and non-blocking stubs
+    which operate against separate blocking and non-blocking RPC interfaces.
+    RPC implementations will have to implement the new interfaces in order to
+    support blocking mode.
+  * New I/O methods parseDelimitedFrom(), mergeDelimitedFrom(), and
+    writeDelimitedTo() read and write "delemited" messages from/to a stream,
+    meaning that the message size precedes the data.  This way, you can write
+    multiple messages to a stream without having to worry about delimiting
+    them yourself.
+  * Throw a more descriptive exception when build() is double-called.
+  * Add a method to query whether CodedInputStream is at the end of the input
+    stream.
+  * Add a method to reset a CodedInputStream's size counter; useful when
+    reading many messages with the same stream.
+  * equals() and hashCode() now account for unknown fields.
 
   Python
   * Added slicing support for repeated scalar fields. Added slice retrieval and
     object will be returned directly to the caller.  This interface change
     cannot be used in practice until RPC implementations are updated to
     implement it.
+  * Changes to input_stream.py should make protobuf compatible with appengine.
 
 2008-11-25 version 2.0.3:
 

File CONTRIBUTORS.txt

   Petar Petrov <petar@google.com>
 
 Large code contributions:
+  Jason Hsueh <jasonh@google.com>
   Joseph Schorr <jschorr@google.com>
   Wenbo Zhu <wenboz@google.com>
 
   Kevin Ko <kevin.s.ko@gmail.com>
     * Small patch to handle trailing slashes in --proto_path flag.
   Johan Euphrosine <proppy@aminche.com>
-    * Small patch to fix Pyhton CallMethod().
+    * Small patch to fix Python CallMethod().
   Ulrich Kunitz <kune@deine-taler.de>
     * Small optimizations to Python serialization.
   Leandro Lucarella <llucax@gmail.com>
   examples/add_person.py                                                     \
   examples/list_people.py                                                    \
   java/src/main/java/com/google/protobuf/AbstractMessage.java                \
+  java/src/main/java/com/google/protobuf/BlockingRpcChannel.java             \
+  java/src/main/java/com/google/protobuf/BlockingService.java                \
   java/src/main/java/com/google/protobuf/ByteString.java                     \
   java/src/main/java/com/google/protobuf/CodedInputStream.java               \
   java/src/main/java/com/google/protobuf/CodedOutputStream.java              \
   java/src/main/java/com/google/protobuf/RpcController.java                  \
   java/src/main/java/com/google/protobuf/RpcUtil.java                        \
   java/src/main/java/com/google/protobuf/Service.java                        \
+  java/src/main/java/com/google/protobuf/ServiceException.java               \
   java/src/main/java/com/google/protobuf/TextFormat.java                     \
   java/src/main/java/com/google/protobuf/UninitializedMessageException.java  \
   java/src/main/java/com/google/protobuf/UnknownFieldSet.java                \

File java/src/main/java/com/google/protobuf/AbstractMessage.java

 
 import com.google.protobuf.Descriptors.FieldDescriptor;
 
+import java.io.FilterInputStream;
 import java.io.InputStream;
 import java.io.IOException;
 import java.io.OutputStream;
     codedOutput.flush();
   }
 
+  public void writeDelimitedTo(OutputStream output) throws IOException {
+    CodedOutputStream codedOutput = CodedOutputStream.newInstance(output);
+    codedOutput.writeRawVarint32(getSerializedSize());
+    writeTo(codedOutput);
+    codedOutput.flush();
+  }
+
   private int memoizedSize = -1;
 
   public int getSerializedSize() {
     if (getDescriptorForType() != otherMessage.getDescriptorForType()) {
       return false;
     }
-    return getAllFields().equals(otherMessage.getAllFields());
+    return getAllFields().equals(otherMessage.getAllFields()) &&
+        getUnknownFields().equals(otherMessage.getUnknownFields());
   }
 
   @Override
     int hash = 41;
     hash = (19 * hash) + getDescriptorForType().hashCode();
     hash = (53 * hash) + getAllFields().hashCode();
+    hash = (29 * hash) + getUnknownFields().hashCode();
     return hash;
   }
 
       codedInput.checkLastTagWas(0);
       return (BuilderType) this;
     }
+
+    public BuilderType mergeDelimitedFrom(InputStream input,
+                                          ExtensionRegistry extensionRegistry)
+                                          throws IOException {
+      final int size = CodedInputStream.readRawVarint32(input);
+
+      // A stream which will not read more than |size| bytes.
+      InputStream limitedInput = new FilterInputStream(input) {
+        int limit = size;
+
+        @Override
+        public int available() throws IOException {
+          return Math.min(super.available(), limit);
+        }
+
+        @Override
+        public int read() throws IOException {
+          if (limit <= 0) return -1;
+          int result = super.read();
+          if (result >= 0) --limit;
+          return result;
+        }
+
+        @Override
+        public int read(byte[] b, int off, int len) throws IOException {
+          if (limit <= 0) return -1;
+          len = Math.min(len, limit);
+          int result = super.read(b, off, len);
+          if (result >= 0) limit -= result;
+          return result;
+        }
+
+        @Override
+        public long skip(long n) throws IOException {
+          long result = super.skip(Math.min(n, limit));
+          if (result >= 0) limit -= result;
+          return result;
+        }
+      };
+      return mergeFrom(limitedInput, extensionRegistry);
+    }
+
+    public BuilderType mergeDelimitedFrom(InputStream input)
+        throws IOException {
+      return mergeDelimitedFrom(input, ExtensionRegistry.getEmptyRegistry());
+    }
   }
 }

File java/src/main/java/com/google/protobuf/BlockingRpcChannel.java

+// Protocol Buffers - Google's data interchange format
+// Copyright 2008 Google Inc.  All rights reserved.
+// http://code.google.com/p/protobuf/
+//
+// Redistribution and use in source and binary forms, with or without
+// modification, are permitted provided that the following conditions are
+// met:
+//
+//     * Redistributions of source code must retain the above copyright
+// notice, this list of conditions and the following disclaimer.
+//     * Redistributions in binary form must reproduce the above
+// copyright notice, this list of conditions and the following disclaimer
+// in the documentation and/or other materials provided with the
+// distribution.
+//     * Neither the name of Google Inc. nor the names of its
+// contributors may be used to endorse or promote products derived from
+// this software without specific prior written permission.
+//
+// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+package com.google.protobuf;
+
+/**
+ * <p>Abstract interface for a blocking RPC channel.  {@code BlockingRpcChannel}
+ * is the blocking equivalent to {@link RpcChannel}.
+ *
+ * @author kenton@google.com Kenton Varda
+ * @author cpovirk@google.com Chris Povirk
+ */
+public interface BlockingRpcChannel {
+  /**
+   * Call the given method of the remote service and blocks until it returns.
+   * {@code callBlockingMethod()} is the blocking equivalent to
+   * {@link RpcChannel#callMethod}.
+   */
+  Message callBlockingMethod(
+      Descriptors.MethodDescriptor method,
+      RpcController controller,
+      Message request,
+      Message responsePrototype) throws ServiceException;
+}

File java/src/main/java/com/google/protobuf/BlockingService.java

+// Protocol Buffers - Google's data interchange format
+// Copyright 2008 Google Inc.  All rights reserved.
+// http://code.google.com/p/protobuf/
+//
+// Redistribution and use in source and binary forms, with or without
+// modification, are permitted provided that the following conditions are
+// met:
+//
+//     * Redistributions of source code must retain the above copyright
+// notice, this list of conditions and the following disclaimer.
+//     * Redistributions in binary form must reproduce the above
+// copyright notice, this list of conditions and the following disclaimer
+// in the documentation and/or other materials provided with the
+// distribution.
+//     * Neither the name of Google Inc. nor the names of its
+// contributors may be used to endorse or promote products derived from
+// this software without specific prior written permission.
+//
+// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+package com.google.protobuf;
+
+/**
+ * Blocking equivalent to {@link Service}.
+ *
+ * @author kenton@google.com Kenton Varda
+ * @author cpovirk@google.com Chris Povirk
+ */
+public interface BlockingService {
+  /**
+   * Equivalent to {@link Service#getDescriptorForType}.
+   */
+  Descriptors.ServiceDescriptor getDescriptorForType();
+
+  /**
+   * Equivalent to {@link Service#callMethod}, except that
+   * {@code callBlockingMethod()} returns the result of the RPC or throws a
+   * {@link ServiceException} if there is a failure, rather than passing the
+   * information to a callback.
+   */
+  Message callBlockingMethod(Descriptors.MethodDescriptor method,
+                             RpcController controller,
+                             Message request) throws ServiceException;
+
+  /**
+   * Equivalent to {@link Service#getRequestPrototype}.
+   */
+  Message getRequestPrototype(Descriptors.MethodDescriptor method);
+
+  /**
+   * Equivalent to {@link Service#getResponsePrototype}.
+   */
+  Message getResponsePrototype(Descriptors.MethodDescriptor method);
+}

File java/src/main/java/com/google/protobuf/CodedInputStream.java

    * may legally end wherever a tag occurs, and zero is not a valid tag number.
    */
   public int readTag() throws IOException {
-    if (bufferPos == bufferSize && !refillBuffer(false)) {
+    if (isAtEnd()) {
       lastTag = 0;
       return 0;
     }
     return result;
   }
 
+  /**
+   * Reads a varint from the input one byte at a time, so that it does not
+   * read any bytes after the end of the varint.  If you simply wrapped the
+   * stream in a CodedInputStream and used {@link #readRawVarint32(InputStream)}
+   * then you would probably end up reading past the end of the varint since
+   * CodedInputStream buffers its input.
+   */
+  static int readRawVarint32(InputStream input) throws IOException {
+    int result = 0;
+    int offset = 0;
+    for (; offset < 32; offset += 7) {
+      int b = input.read();
+      if (b == -1) {
+        throw InvalidProtocolBufferException.truncatedMessage();
+      }
+      result |= (b & 0x7f) << offset;
+      if ((b & 0x80) == 0) {
+        return result;
+      }
+    }
+    // Keep reading up to 64 bits.
+    for (; offset < 64; offset += 7) {
+      int b = input.read();
+      if (b == -1) {
+        throw InvalidProtocolBufferException.truncatedMessage();
+      }
+      if ((b & 0x80) == 0) {
+        return result;
+      }
+    }
+    throw InvalidProtocolBufferException.malformedVarint();
+  }
+
   /** Read a raw Varint from the stream. */
   public long readRawVarint64() throws IOException {
     int shift = 0;
    * size limits only apply when reading from an {@code InputStream}, not
    * when constructed around a raw byte array (nor with
    * {@link ByteString#newCodedInput}).
+   * <p>
+   * If you want to read several messages from a single CodedInputStream, you
+   * could call {@link #resetSizeCounter()} after each one to avoid hitting the
+   * size limit.
    *
    * @return the old limit.
    */
   }
 
   /**
+   * Resets the current size counter to zero (see {@link #setSizeLimit(int)}).
+   */
+  public void resetSizeCounter() {
+    totalBytesRetired = 0;
+  }
+
+  /**
    * Sets {@code currentLimit} to (current position) + {@code byteLimit}.  This
    * is called when descending into a length-delimited embedded message.
    *
   }
 
   /**
+   * Returns true if the stream has reached the end of the input.  This is the
+   * case if either the end of the underlying input source has been reached or
+   * if the stream has reached a limit created using {@link #pushLimit(int)}.
+   */
+  public boolean isAtEnd() throws IOException {
+    return bufferPos == bufferSize && !refillBuffer(false);
+  }
+
+  /**
    * Called with {@code this.buffer} is empty to read more bytes from the
    * input.  If {@code mustSucceed} is true, refillBuffer() gurantees that
    * either there will be at least one byte in the buffer when it returns
 
     bufferPos = 0;
     bufferSize = (input == null) ? -1 : input.read(buffer);
+    if (bufferSize == 0 || bufferSize < -1) {
+      throw new IllegalStateException(
+          "InputStream#read(byte[]) returned invalid result: " + bufferSize +
+          "\nThe InputStream implementation is buggy.");
+    }
     if (bufferSize == -1) {
       bufferSize = 0;
       if (mustSucceed) {
       throw InvalidProtocolBufferException.truncatedMessage();
     }
 
-    if (size < bufferSize - bufferPos) {
+    if (size <= bufferSize - bufferPos) {
       // We have all the bytes we need already.
       bufferPos += size;
     } else {

File java/src/main/java/com/google/protobuf/Descriptors.java

 
       GenericDescriptor old =
         descriptorsByName.put(fullName,
-          new PackageDescriptor(fullName, name, file));
+          new PackageDescriptor(name, fullName, file));
       if (old != null) {
         descriptorsByName.put(fullName, old);
         if (!(old instanceof PackageDescriptor)) {

File java/src/main/java/com/google/protobuf/DynamicMessage.java

     }
 
     public DynamicMessage build() {
-      if (!isInitialized()) {
+      // If fields == null, we'll throw an appropriate exception later.
+      if (fields != null && !isInitialized()) {
         throw new UninitializedMessageException(
           new DynamicMessage(type, fields, unknownFields));
       }
     }
 
     public DynamicMessage buildPartial() {
+      if (fields == null) {
+        throw new IllegalStateException(
+            "build() has already been called on this Builder.");
+      }
       fields.makeImmutable();
       DynamicMessage result =
         new DynamicMessage(type, fields, unknownFields);

File java/src/main/java/com/google/protobuf/Message.java

    * Serializes the message and writes it to {@code output}.  This is just a
    * trivial wrapper around {@link #writeTo(CodedOutputStream)}.  This does
    * not flush or close the stream.
+   * <p>
+   * NOTE:  Protocol Buffers are not self-delimiting.  Therefore, if you write
+   * any more data to the stream after the message, you must somehow ensure
+   * that the parser on the receiving end does not interpret this as being
+   * part of the protocol message.  This can be done e.g. by writing the size
+   * of the message before the data, then making sure to limit the input to
+   * that size on the receiving end (e.g. by wrapping the InputStream in one
+   * which limits the input).  Alternatively, just use
+   * {@link #writeDelimitedTo(OutputStream)}.
    */
   void writeTo(OutputStream output) throws IOException;
 
+  /**
+   * Like {@link #writeTo(OutputStream)}, but writes the size of the message
+   * as a varint before writing the data.  This allows more data to be written
+   * to the stream after the message without the need to delimit the message
+   * data yourself.  Use {@link Builder#mergeDelimitedFrom(InputStream)} (or
+   * the static method {@code YourMessageType.parseDelimitedFrom(InputStream)})
+   * to parse messages written by this method.
+   */
+  void writeDelimitedTo(OutputStream output) throws IOException;
+
   // =================================================================
   // Builders
 
      * {@link #mergeFrom(CodedInputStream)}.  Note that this method always
      * reads the <i>entire</i> input (unless it throws an exception).  If you
      * want it to stop earlier, you will need to wrap your input in some
-     * wrapper stream that limits reading.  Despite usually reading the entire
-     * input, this does not close the stream.
+     * wrapper stream that limits reading.  Or, use
+     * {@link Message#writeDelimitedTo(OutputStream)} to write your message and
+     * {@link #mergeDelimitedFrom(InputStream)} to read it.
+     * <p>
+     * Despite usually reading the entire input, this does not close the stream.
      */
     Builder mergeFrom(InputStream input) throws IOException;
 
     Builder mergeFrom(InputStream input,
                       ExtensionRegistry extensionRegistry)
                       throws IOException;
+
+    /**
+     * Like {@link #mergeFrom(InputStream)}, but does not read until EOF.
+     * Instead, the size of the message (encoded as a varint) is read first,
+     * then the message data.  Use
+     * {@link Message#writeDelimitedTo(OutputStream)} to write messages in this
+     * format.
+     */
+    Builder mergeDelimitedFrom(InputStream input)
+                               throws IOException;
+
+    /**
+     * Like {@link #mergeDelimitedFrom(InputStream)} but supporting extensions.
+     */
+    Builder mergeDelimitedFrom(InputStream input,
+                               ExtensionRegistry extensionRegistry)
+                               throws IOException;
   }
 }

File java/src/main/java/com/google/protobuf/RpcUtil.java

   private RpcUtil() {}
 
   /**
-   * Take an {@code RcpCallabck<Message>} and convert it to an
+   * Take an {@code RpcCallback<Message>} and convert it to an
    * {@code RpcCallback} accepting a specific message type.  This is always
    * type-safe (parameter type contravariance).
    */
   }
 
   /**
-   * Take an {@code RcpCallabck} accepting a specific message type and convert
-   * it to an {@code RcpCallabck<Message>}.  The generalized callback will
+   * Take an {@code RpcCallback} accepting a specific message type and convert
+   * it to an {@code RpcCallback<Message>}.  The generalized callback will
    * accept any message object which has the same descriptor, and will convert
    * it to the correct class before calling the original callback.  However,
    * if the generalized callback is given a message with a different descriptor,

File java/src/main/java/com/google/protobuf/ServiceException.java

+// Protocol Buffers - Google's data interchange format
+// Copyright 2008 Google Inc.  All rights reserved.
+// http://code.google.com/p/protobuf/
+//
+// Redistribution and use in source and binary forms, with or without
+// modification, are permitted provided that the following conditions are
+// met:
+//
+//     * Redistributions of source code must retain the above copyright
+// notice, this list of conditions and the following disclaimer.
+//     * Redistributions in binary form must reproduce the above
+// copyright notice, this list of conditions and the following disclaimer
+// in the documentation and/or other materials provided with the
+// distribution.
+//     * Neither the name of Google Inc. nor the names of its
+// contributors may be used to endorse or promote products derived from
+// this software without specific prior written permission.
+//
+// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+package com.google.protobuf;
+
+/**
+ * Thrown by blocking RPC methods when a failure occurs.
+ * 
+ * @author cpovirk@google.com (Chris Povirk)
+ */
+public final class ServiceException extends Exception {
+  public ServiceException(String message) {
+    super(message);
+  }
+}

File java/src/main/java/com/google/protobuf/UnknownFieldSet.java

 import java.io.IOException;
 import java.io.OutputStream;
 import java.util.ArrayList;
+import java.util.Arrays;
 import java.util.Collections;
 import java.util.TreeMap;
 import java.util.List;
   }
   private Map<Integer, Field> fields;
 
+  @Override
+  public boolean equals(Object other) {
+    if (this == other) {
+      return true;
+    }
+    return (other instanceof UnknownFieldSet) &&
+        this.fields.equals(((UnknownFieldSet) other).fields);
+  }
+
+  @Override
+  public int hashCode() {
+    return this.fields.hashCode();
+  }
+
   /** Get a map of fields in the set by number. */
   public Map<Integer, Field> asMap() {
     return fields;
      */
     public List<UnknownFieldSet> getGroupList()      { return group;           }
 
+    @Override
+    public boolean equals(Object other) {
+      if (this == other) {
+        return true;
+      }
+      if (!(other instanceof Field)) {
+        return false;
+      }
+      return Arrays.equals(this.getIdentityArray(),
+          ((Field) other).getIdentityArray());
+    }
+
+    @Override
+    public int hashCode() {
+      return Arrays.hashCode(getIdentityArray());
+    }
+
+    /**
+     * Returns the array of objects to be used to uniquely identify this
+     * {@link UnknownFieldSet.Field} instance.
+     */
+    private Object[] getIdentityArray() {
+      return new Object[] {
+          this.varint,
+          this.fixed32,
+          this.fixed64,
+          this.lengthDelimited,
+          this.group};
+    }
+
     /**
      * Serializes the field, including field number, and writes it to
      * {@code output}.

File java/src/test/java/com/google/protobuf/AbstractMessageTest.java

 
 package com.google.protobuf;
 
+import protobuf_unittest.UnittestOptimizeFor.TestOptimizedForSize;
 import protobuf_unittest.UnittestProto;
 import protobuf_unittest.UnittestProto.ForeignMessage;
+import protobuf_unittest.UnittestProto.TestAllExtensions;
 import protobuf_unittest.UnittestProto.TestAllTypes;
-import protobuf_unittest.UnittestProto.TestAllExtensions;
 import protobuf_unittest.UnittestProto.TestPackedTypes;
 import protobuf_unittest.UnittestProto.TestRequired;
 import protobuf_unittest.UnittestProto.TestRequiredForeign;
-import protobuf_unittest.UnittestOptimizeFor.TestOptimizedForSize;
 
 import junit.framework.TestCase;
 
   // -----------------------------------------------------------------
   // Tests for equals and hashCode
   
-  public void testEqualsAndHashCode() {
+  public void testEqualsAndHashCode() throws Exception {
     TestAllTypes a = TestUtil.getAllSet();
     TestAllTypes b = TestAllTypes.newBuilder().build();
     TestAllTypes c = TestAllTypes.newBuilder(b).addRepeatedString("x").build();
     TestAllExtensions e = TestUtil.getAllExtensionsSet();
     TestAllExtensions f = TestAllExtensions.newBuilder(e)
         .addExtension(UnittestProto.repeatedInt32Extension, 999).build();
-      
+
     checkEqualsIsConsistent(a);
     checkEqualsIsConsistent(b);
     checkEqualsIsConsistent(c);
     checkNotEqual(d, f);
 
     checkNotEqual(e, f);
+
+    // Deserializing into the TestEmptyMessage such that every field
+    // is an {@link UnknownFieldSet.Field}.
+    UnittestProto.TestEmptyMessage eUnknownFields =
+        UnittestProto.TestEmptyMessage.parseFrom(e.toByteArray());
+    UnittestProto.TestEmptyMessage fUnknownFields =
+        UnittestProto.TestEmptyMessage.parseFrom(f.toByteArray());
+    checkNotEqual(eUnknownFields, fUnknownFields);
+    checkEqualsIsConsistent(eUnknownFields);
+    checkEqualsIsConsistent(fUnknownFields);
+
+    // Subseqent reconstitutions should be identical
+    UnittestProto.TestEmptyMessage eUnknownFields2 =
+        UnittestProto.TestEmptyMessage.parseFrom(e.toByteArray());
+    checkEqualsIsConsistent(eUnknownFields, eUnknownFields2);
   }
   
   /**
-   * Asserts that the given protos are equal and have the same hash code.
+   * Asserts that the given proto has symetric equals and hashCode methods.
    */
   private void checkEqualsIsConsistent(Message message) {
     // Object should be equal to itself.
     
     // Object should be equal to a dynamic copy of itself.
     DynamicMessage dynamic = DynamicMessage.newBuilder(message).build();
-    assertEquals(message, dynamic);
-    assertEquals(dynamic, message);
-    assertEquals(dynamic.hashCode(), message.hashCode());
+    checkEqualsIsConsistent(message, dynamic);
+  }
+
+  /**
+   * Asserts that the given protos are equal and have the same hash code.
+   */
+  private void checkEqualsIsConsistent(Message message1, Message message2) {
+    assertEquals(message1, message2);
+    assertEquals(message2, message1);
+    assertEquals(message2.hashCode(), message1.hashCode());
   }
 
   /**

File java/src/test/java/com/google/protobuf/CodedInputStreamTest.java

 
     input = CodedInputStream.newInstance(data);
     assertEquals(value, input.readRawVarint64());
+    assertTrue(input.isAtEnd());
 
     // Try different block sizes.
     for (int blockSize = 1; blockSize <= 16; blockSize *= 2) {
       input = CodedInputStream.newInstance(
         new SmallBlockInputStream(data, blockSize));
       assertEquals(value, input.readRawVarint64());
+      assertTrue(input.isAtEnd());
     }
+
+    // Try reading direct from an InputStream.  We want to verify that it
+    // doesn't read past the end of the input, so we copy to a new, bigger
+    // array first.
+    byte[] longerData = new byte[data.length + 1];
+    System.arraycopy(data, 0, longerData, 0, data.length);
+    InputStream rawInput = new ByteArrayInputStream(longerData);
+    assertEquals((int)value, CodedInputStream.readRawVarint32(rawInput));
+    assertEquals(1, rawInput.available());
   }
 
   /**
     } catch (InvalidProtocolBufferException e) {
       assertEquals(expected.getMessage(), e.getMessage());
     }
+
+    // Make sure we get the same error when reading direct from an InputStream.
+    try {
+      CodedInputStream.readRawVarint32(new ByteArrayInputStream(data));
+      fail("Should have thrown an exception.");
+    } catch (InvalidProtocolBufferException e) {
+      assertEquals(expected.getMessage(), e.getMessage());
+    }
   }
 
   /** Tests readRawVarint32() and readRawVarint64(). */
                                         throws Exception {
     CodedInputStream input = CodedInputStream.newInstance(data);
     assertEquals(value, input.readRawLittleEndian32());
+    assertTrue(input.isAtEnd());
 
     // Try different block sizes.
     for (int blockSize = 1; blockSize <= 16; blockSize *= 2) {
       input = CodedInputStream.newInstance(
         new SmallBlockInputStream(data, blockSize));
       assertEquals(value, input.readRawLittleEndian32());
+      assertTrue(input.isAtEnd());
     }
   }
 
                                         throws Exception {
     CodedInputStream input = CodedInputStream.newInstance(data);
     assertEquals(value, input.readRawLittleEndian64());
+    assertTrue(input.isAtEnd());
 
     // Try different block sizes.
     for (int blockSize = 1; blockSize <= 16; blockSize *= 2) {
       input = CodedInputStream.newInstance(
         new SmallBlockInputStream(data, blockSize));
       assertEquals(value, input.readRawLittleEndian64());
+      assertTrue(input.isAtEnd());
     }
   }
 
     }
   }
 
+  /**
+   * Test that a bug in skipRawBytes() has been fixed:  if the skip skips
+   * exactly up to a limit, this should not break things.
+   */
+  public void testSkipRawBytesBug() throws Exception {
+    byte[] rawBytes = new byte[] { 1, 2 };
+    CodedInputStream input = CodedInputStream.newInstance(rawBytes);
+
+    int limit = input.pushLimit(1);
+    input.skipRawBytes(1);
+    input.popLimit(limit);
+    assertEquals(2, input.readRawByte());
+  }
+
   public void testReadHugeBlob() throws Exception {
     // Allocate and initialize a 1MB blob.
     byte[] blob = new byte[1 << 20];
     }
   }
 
+  public void testResetSizeCounter() throws Exception {
+    CodedInputStream input = CodedInputStream.newInstance(
+        new SmallBlockInputStream(new byte[256], 8));
+    input.setSizeLimit(16);
+    input.readRawBytes(16);
+
+    try {
+      input.readRawByte();
+      fail("Should have thrown an exception!");
+    } catch (InvalidProtocolBufferException e) {
+      // success.
+    }
+
+    input.resetSizeCounter();
+    input.readRawByte();  // No exception thrown.
+
+    try {
+      input.readRawBytes(16);  // Hits limit again.
+      fail("Should have thrown an exception!");
+    } catch (InvalidProtocolBufferException e) {
+      // success.
+    }
+  }
+
   /**
    * Tests that if we read an string that contains invalid UTF-8, no exception
    * is thrown.  Instead, the invalid bytes are replaced with the Unicode

File java/src/test/java/com/google/protobuf/DynamicMessageTest.java

     reflectionTester.assertAllFieldsSetViaReflection(message);
   }
 
+  public void testDoubleBuildError() throws Exception {
+    Message.Builder builder =
+      DynamicMessage.newBuilder(TestAllTypes.getDescriptor());
+    builder.build();
+    try {
+      builder.build();
+      fail("Should have thrown exception.");
+    } catch (IllegalStateException e) {
+      // Success.
+    }
+  }
+
   public void testDynamicMessageSettersRejectNull() throws Exception {
     Message.Builder builder =
       DynamicMessage.newBuilder(TestAllTypes.getDescriptor());

File java/src/test/java/com/google/protobuf/GeneratedMessageTest.java

     TestUtil.assertAllFieldsSet(message);
   }
 
+  public void testDoubleBuildError() throws Exception {
+    TestAllTypes.Builder builder = TestAllTypes.newBuilder();
+    builder.build();
+    try {
+      builder.build();
+      fail("Should have thrown exception.");
+    } catch (IllegalStateException e) {
+      // Success.
+    }
+  }
+
   public void testSettersRejectNull() throws Exception {
     TestAllTypes.Builder builder = TestAllTypes.newBuilder();
     try {

File java/src/test/java/com/google/protobuf/ServiceTest.java

 
 package com.google.protobuf;
 
+import com.google.protobuf.Descriptors.MethodDescriptor;
+import protobuf_unittest.MessageWithNoOuter;
+import protobuf_unittest.ServiceWithNoOuter;
+import protobuf_unittest.UnittestProto.TestAllTypes;
 import protobuf_unittest.UnittestProto.TestService;
 import protobuf_unittest.UnittestProto.FooRequest;
 import protobuf_unittest.UnittestProto.FooResponse;
   private final Descriptors.MethodDescriptor barDescriptor =
     TestService.getDescriptor().getMethods().get(1);
 
+  @Override
   protected void setUp() throws Exception {
     super.setUp();
     control = EasyMock.createStrictControl();
     control.verify();
   }
 
+  /** Tests generated blocking stubs. */
+  public void testBlockingStub() throws Exception {
+    FooRequest fooRequest = FooRequest.newBuilder().build();
+    BarRequest barRequest = BarRequest.newBuilder().build();
+    BlockingRpcChannel mockChannel =
+        control.createMock(BlockingRpcChannel.class);
+    TestService.BlockingInterface stub =
+        TestService.newBlockingStub(mockChannel);
+
+    FooResponse fooResponse = FooResponse.newBuilder().build();
+    BarResponse barResponse = BarResponse.newBuilder().build();
+
+    EasyMock.expect(mockChannel.callBlockingMethod(
+      EasyMock.same(fooDescriptor),
+      EasyMock.same(mockController),
+      EasyMock.same(fooRequest),
+      EasyMock.same(FooResponse.getDefaultInstance()))).andReturn(fooResponse);
+    EasyMock.expect(mockChannel.callBlockingMethod(
+      EasyMock.same(barDescriptor),
+      EasyMock.same(mockController),
+      EasyMock.same(barRequest),
+      EasyMock.same(BarResponse.getDefaultInstance()))).andReturn(barResponse);
+    control.replay();
+
+    assertSame(fooResponse, stub.foo(mockController, fooRequest));
+    assertSame(barResponse, stub.bar(mockController, barRequest));
+    control.verify();
+  }
+
+  public void testNewReflectiveService() {
+    ServiceWithNoOuter.Interface impl =
+        control.createMock(ServiceWithNoOuter.Interface.class);
+    RpcController controller = control.createMock(RpcController.class);
+    Service service = ServiceWithNoOuter.newReflectiveService(impl);
+
+    MethodDescriptor fooMethod =
+        ServiceWithNoOuter.getDescriptor().findMethodByName("Foo");
+    MessageWithNoOuter request = MessageWithNoOuter.getDefaultInstance();
+    RpcCallback<Message> callback = new RpcCallback<Message>() {
+      public void run(Message parameter) {
+        // No reason this should be run.
+        fail();
+      }
+    };
+    RpcCallback<TestAllTypes> specializedCallback =
+        RpcUtil.specializeCallback(callback);
+
+    impl.foo(EasyMock.same(controller), EasyMock.same(request),
+        EasyMock.same(specializedCallback));
+    EasyMock.expectLastCall();
+
+    control.replay();
+
+    service.callMethod(fooMethod, controller, request, callback);
+
+    control.verify();
+  }
+
+  public void testNewReflectiveBlockingService() throws ServiceException {
+    ServiceWithNoOuter.BlockingInterface impl =
+        control.createMock(ServiceWithNoOuter.BlockingInterface.class);
+    RpcController controller = control.createMock(RpcController.class);
+    BlockingService service =
+        ServiceWithNoOuter.newReflectiveBlockingService(impl);
+
+    MethodDescriptor fooMethod =
+        ServiceWithNoOuter.getDescriptor().findMethodByName("Foo");
+    MessageWithNoOuter request = MessageWithNoOuter.getDefaultInstance();
+    RpcCallback<Message> callback = new RpcCallback<Message>() {
+      public void run(Message parameter) {
+        // No reason this should be run.
+        fail();
+      }
+    };
+
+    TestAllTypes expectedResponse = TestAllTypes.getDefaultInstance();
+    EasyMock.expect(impl.foo(EasyMock.same(controller), EasyMock.same(request)))
+        .andReturn(expectedResponse);
+
+    control.replay();
+
+    Message response =
+        service.callBlockingMethod(fooMethod, controller, request);
+    assertEquals(expectedResponse, response);
+
+    control.verify();
+  }
+
   // =================================================================
 
   /**
    * In other words, c wraps the given callback.
    */
   private <Type extends Message> RpcCallback<Type> wrapsCallback(
-      MockCallback callback) {
+      MockCallback<?> callback) {
     EasyMock.reportMatcher(new WrapsCallback(callback));
     return null;
   }
 
   /** Implementation of the wrapsCallback() argument matcher. */
   private static class WrapsCallback implements IArgumentMatcher {
-    private MockCallback callback;
+    private MockCallback<?> callback;
 
-    public WrapsCallback(MockCallback callback) {
+    public WrapsCallback(MockCallback<?> callback) {
       this.callback = callback;
     }
 

File java/src/test/java/com/google/protobuf/UnknownFieldSetTest.java

 package com.google.protobuf;
 
 import protobuf_unittest.UnittestProto;
+import protobuf_unittest.UnittestProto.TestAllExtensions;
 import protobuf_unittest.UnittestProto.TestAllTypes;
-import protobuf_unittest.UnittestProto.TestAllExtensions;
 import protobuf_unittest.UnittestProto.TestEmptyMessage;
-import protobuf_unittest.UnittestProto.
-  TestEmptyMessageWithExtensions;
+import protobuf_unittest.UnittestProto.TestEmptyMessageWithExtensions;
 
 import junit.framework.TestCase;
+
 import java.util.Arrays;
 import java.util.Map;
 
     assertEquals(1, field.getVarintList().size());
     assertEquals(0x7FFFFFFFFFFFFFFFL, (long)field.getVarintList().get(0));
   }
+
+  public void testEqualsAndHashCode() {
+    UnknownFieldSet.Field fixed32Field =
+        UnknownFieldSet.Field.newBuilder()
+            .addFixed32(1)
+            .build();
+    UnknownFieldSet.Field fixed64Field =
+        UnknownFieldSet.Field.newBuilder()
+            .addFixed64(1)
+            .build();
+    UnknownFieldSet.Field varIntField =
+        UnknownFieldSet.Field.newBuilder()
+            .addVarint(1)
+            .build();
+    UnknownFieldSet.Field lengthDelimitedField =
+        UnknownFieldSet.Field.newBuilder()
+            .addLengthDelimited(ByteString.EMPTY)
+            .build();
+    UnknownFieldSet.Field groupField =
+        UnknownFieldSet.Field.newBuilder()
+            .addGroup(unknownFields)
+            .build();
+
+    UnknownFieldSet a =
+        UnknownFieldSet.newBuilder()
+            .addField(1, fixed32Field)
+            .build();
+    UnknownFieldSet b =
+        UnknownFieldSet.newBuilder()
+            .addField(1, fixed64Field)
+            .build();
+    UnknownFieldSet c =
+        UnknownFieldSet.newBuilder()
+            .addField(1, varIntField)
+            .build();
+    UnknownFieldSet d =
+        UnknownFieldSet.newBuilder()
+            .addField(1, lengthDelimitedField)
+            .build();
+    UnknownFieldSet e =
+        UnknownFieldSet.newBuilder()
+            .addField(1, groupField)
+            .build();
+
+    checkEqualsIsConsistent(a);
+    checkEqualsIsConsistent(b);
+    checkEqualsIsConsistent(c);
+    checkEqualsIsConsistent(d);
+    checkEqualsIsConsistent(e);
+
+    checkNotEqual(a, b);
+    checkNotEqual(a, c);
+    checkNotEqual(a, d);
+    checkNotEqual(a, e);
+    checkNotEqual(b, c);
+    checkNotEqual(b, d);
+    checkNotEqual(b, e);
+    checkNotEqual(c, d);
+    checkNotEqual(c, e);
+    checkNotEqual(d, e);
+  }
+
+  /**
+   * Asserts that the given field sets are not equal and have different
+   * hash codes.
+   *
+   * @warning It's valid for non-equal objects to have the same hash code, so
+   *   this test is stricter than it needs to be. However, this should happen
+   *   relatively rarely.
+   */
+  private void checkNotEqual(UnknownFieldSet s1, UnknownFieldSet s2) {
+    String equalsError = String.format("%s should not be equal to %s", s1, s2);
+    assertFalse(equalsError, s1.equals(s2));
+    assertFalse(equalsError, s2.equals(s1));
+
+    assertFalse(
+        String.format("%s should have a different hash code from %s", s1, s2),
+        s1.hashCode() == s2.hashCode());
+  }
+
+  /**
+   * Asserts that the given field sets are equal and have identical hash codes.
+   */
+  private void checkEqualsIsConsistent(UnknownFieldSet set) {
+    // Object should be equal to itself.
+    assertEquals(set, set);
+
+    // Object should be equal to a copy of itself.
+    UnknownFieldSet copy = UnknownFieldSet.newBuilder(set).build();
+    assertEquals(set, copy);
+    assertEquals(copy, set);
+    assertEquals(set.hashCode(), copy.hashCode());
+  }
 }

File java/src/test/java/com/google/protobuf/WireFormatTest.java

 package com.google.protobuf;
 
 import junit.framework.TestCase;
+
+import java.io.ByteArrayInputStream;
+import java.io.ByteArrayOutputStream;
+
 import protobuf_unittest.UnittestProto;
 import protobuf_unittest.UnittestProto.TestAllExtensions;
 import protobuf_unittest.UnittestProto.TestAllTypes;
                  TestUtil.getAllExtensionsSet().getSerializedSize());
   }
 
+  public void testSerializeDelimited() throws Exception {
+    ByteArrayOutputStream output = new ByteArrayOutputStream();
+    TestUtil.getAllSet().writeDelimitedTo(output);
+    output.write(12);
+    TestUtil.getPackedSet().writeDelimitedTo(output);
+    output.write(34);
+
+    ByteArrayInputStream input = new ByteArrayInputStream(output.toByteArray());
+
+    TestUtil.assertAllFieldsSet(TestAllTypes.parseDelimitedFrom(input));
+    assertEquals(12, input.read());
+    TestUtil.assertPackedFieldsSet(TestPackedTypes.parseDelimitedFrom(input));
+    assertEquals(34, input.read());
+    assertEquals(-1, input.read());
+  }
+
   private void assertFieldsInOrder(ByteString data) throws Exception {
     CodedInputStream input = data.newCodedInput();
     int previousTag = 0;

File python/google/protobuf/internal/input_stream.py

 
 __author__ = 'robinson@google.com (Will Robinson)'
 
+import array
 import struct
 from google.protobuf import message
 from google.protobuf.internal import wire_format
 # proto2 implementation.
 
 
-class InputStream(object):
+class InputStreamBuffer(object):
 
   """Contains all logic for reading bits, and dealing with stream position.
 
       shift += 7
       if not (b & 0x80):
         return result
+
+
+class InputStreamArray(object):
+
+  """Contains all logic for reading bits, and dealing with stream position.
+
+  If an InputStream method ever raises an exception, the stream is left
+  in an indeterminate state and is not safe for further use.
+
+  This alternative to InputStreamBuffer is used in environments where buffer()
+  is unavailble, such as Google App Engine.
+  """
+
+  def __init__(self, s):
+    self._buffer = array.array('B', s)
+    self._pos = 0
+
+  def EndOfStream(self):
+    return self._pos >= len(self._buffer)
+
+  def Position(self):
+    return self._pos
+
+  def GetSubBuffer(self, size=None):
+    if size is None:
+      return self._buffer[self._pos : ].tostring()
+    else:
+      if size < 0:
+        raise message.DecodeError('Negative size %d' % size)
+      return self._buffer[self._pos : self._pos + size].tostring()
+
+  def SkipBytes(self, num_bytes):
+    if num_bytes < 0:
+      raise message.DecodeError('Negative num_bytes %d' % num_bytes)
+    self._pos += num_bytes
+    self._pos = min(self._pos, len(self._buffer))
+
+  def ReadBytes(self, size):
+    if size < 0:
+      raise message.DecodeError('Negative size %d' % size)
+    s = self._buffer[self._pos : self._pos + size].tostring()
+    self._pos += len(s)  # Only advance by the number of bytes actually read.
+    return s
+
+  def ReadLittleEndian32(self):
+    try:
+      i = struct.unpack(wire_format.FORMAT_UINT32_LITTLE_ENDIAN,
+                        self._buffer[self._pos : self._pos + 4])
+      self._pos += 4
+      return i[0]  # unpack() result is a 1-element tuple.
+    except struct.error, e:
+      raise message.DecodeError(e)
+
+  def ReadLittleEndian64(self):
+    try:
+      i = struct.unpack(wire_format.FORMAT_UINT64_LITTLE_ENDIAN,
+                        self._buffer[self._pos : self._pos + 8])
+      self._pos += 8
+      return i[0]  # unpack() result is a 1-element tuple.
+    except struct.error, e:
+      raise message.DecodeError(e)
+
+  def ReadVarint32(self):
+    i = self.ReadVarint64()
+    if not wire_format.INT32_MIN <= i <= wire_format.INT32_MAX:
+      raise message.DecodeError('Value out of range for int32: %d' % i)
+    return int(i)
+
+  def ReadVarUInt32(self):
+    i = self.ReadVarUInt64()
+    if i > wire_format.UINT32_MAX:
+      raise message.DecodeError('Value out of range for uint32: %d' % i)
+    return i
+
+  def ReadVarint64(self):
+    i = self.ReadVarUInt64()
+    if i > wire_format.INT64_MAX:
+      i -= (1 << 64)
+    return i
+
+  def ReadVarUInt64(self):
+    i = self._ReadVarintHelper()
+    if not 0 <= i <= wire_format.UINT64_MAX:
+      raise message.DecodeError('Value out of range for uint64: %d' % i)
+    return i
+
+  def _ReadVarintHelper(self):
+    result = 0
+    shift = 0
+    while 1:
+      if shift >= 64:
+        raise message.DecodeError('Too many bytes when decoding varint.')
+      try:
+        b = self._buffer[self._pos]
+      except IndexError:
+        raise message.DecodeError('Truncated varint.')
+      self._pos += 1
+      result |= ((b & 0x7f) << shift)
+      shift += 7
+      if not (b & 0x80):
+        return result
+
+
+try:
+  buffer('')
+  InputStream = InputStreamBuffer
+except NotImplementedError:
+  # Google App Engine: dev_appserver.py
+  InputStream = InputStreamArray
+except RuntimeError:
+  # Google App Engine: production
+  InputStream = InputStreamArray

File python/google/protobuf/internal/input_stream_test.py

 from google.protobuf.internal import input_stream
 
 
-class InputStreamTest(unittest.TestCase):
+class InputStreamBufferTest(unittest.TestCase):
+
+  def setUp(self):
+    self.__original_input_stream = input_stream.InputStream
+    input_stream.InputStream = input_stream.InputStreamBuffer
+
+  def tearDown(self):
+    input_stream.InputStream = self.__original_input_stream
 
   def testEndOfStream(self):
     stream = input_stream.InputStream('abcd')
     stream = input_stream.InputStream(s)
     self.assertRaises(message.DecodeError, stream.ReadVarUInt64)
 
+
+class InputStreamArrayTest(InputStreamBufferTest):
+
+  def setUp(self):
+    # Test InputStreamArray against the same tests in InputStreamBuffer
+    self.__original_input_stream = input_stream.InputStream
+    input_stream.InputStream = input_stream.InputStreamArray
+
+  def tearDown(self):
+    input_stream.InputStream = self.__original_input_stream
+
+
 if __name__ == '__main__':
   unittest.main()

File python/google/protobuf/internal/reflection_test.py

     test_util.SetAllFields(self.first_proto)
     test_util.SetAllFields(self.second_proto)
 
+  def testNoneNotEqual(self):
+    self.assertNotEqual(self.first_proto, None)
+    self.assertNotEqual(None, self.second_proto)
+
+  def testNotEqualToOtherMessage(self):
+    third_proto = unittest_pb2.TestRequired()
+    self.assertNotEqual(self.first_proto, third_proto)
+    self.assertNotEqual(third_proto, self.second_proto)
+
   def testAllFieldsFilledEquality(self):
     self.assertEqual(self.first_proto, self.second_proto)
 

File python/google/protobuf/reflection.py

 def _AddEqualsMethod(message_descriptor, cls):
   """Helper for _AddMessageMethods()."""
   def __eq__(self, other):
+    if (not isinstance(other, message_mod.Message) or
+        other.DESCRIPTOR != self.DESCRIPTOR):
+      return False
+
     if self is other:
       return True
 

File src/Makefile.am

 
 nobase_include_HEADERS =                                       \
   google/protobuf/stubs/common.h                               \
+  google/protobuf/stubs/once.h                                 \
   google/protobuf/descriptor.h                                 \
   google/protobuf/descriptor.pb.h                              \
   google/protobuf/descriptor_database.h                        \
 libprotobuf_la_LDFLAGS = -version-info 3:0:0
 libprotobuf_la_SOURCES =                                       \
   google/protobuf/stubs/common.cc                              \
+  google/protobuf/stubs/once.cc                                \
   google/protobuf/stubs/hash.cc                                \
   google/protobuf/stubs/hash.h                                 \
   google/protobuf/stubs/map-util.cc                            \
 
 protoc_inputs =                                                \
   google/protobuf/unittest.proto                               \
+  google/protobuf/unittest_empty.proto                         \
   google/protobuf/unittest_import.proto                        \
   google/protobuf/unittest_mset.proto                          \
   google/protobuf/unittest_optimize_for.proto                  \
 protoc_outputs =                                               \
   google/protobuf/unittest.pb.cc                               \
   google/protobuf/unittest.pb.h                                \
+  google/protobuf/unittest_empty.pb.cc                         \
+  google/protobuf/unittest_empty.pb.h                          \
   google/protobuf/unittest_import.pb.cc                        \
   google/protobuf/unittest_import.pb.h                         \
   google/protobuf/unittest_mset.pb.cc                          \
                          -I$(top_builddir)/gtest/include
 protobuf_test_SOURCES =                                        \
   google/protobuf/stubs/common_unittest.cc                     \
+  google/protobuf/stubs/once_unittest.cc                       \
   google/protobuf/stubs/strutil_unittest.cc                    \
   google/protobuf/stubs/structurally_valid_unittest.cc         \
   google/protobuf/descriptor_database_unittest.cc              \

File src/google/protobuf/compiler/cpp/cpp_enum.cc

     "const $classname$ $prefix$$short_name$_MIN = $prefix$$min_name$;\n"
     "const $classname$ $prefix$$short_name$_MAX = $prefix$$max_name$;\n"
     "\n");
+
+  // The _Name and _Parse methods
+  printer->Print(vars,
+    "inline const ::std::string& $classname$_Name($classname$ value) {\n"
+    "  return ::google::protobuf::internal::NameOfEnum(\n"
+    "    $classname$_descriptor(), value);\n"
+    "}\n");
+  printer->Print(vars,
+    "inline bool $classname$_Parse(\n"
+    "    const ::std::string& name, $classname$* value) {\n"
+    "  return ::google::protobuf::internal::ParseNamedEnum<$classname$>(\n"
+    "    $classname$_descriptor(), name, value);\n"
+    "}\n");
 }
 
 void EnumGenerator::GenerateSymbolImports(io::Printer* printer) {
     "static inline bool $nested_name$_IsValid(int value) {\n"
     "  return $classname$_IsValid(value);\n"
     "}\n"
+    "static inline const ::std::string& $nested_name$_Name($nested_name$ value) {\n"
+    "  return $classname$_Name(value);\n"
+    "}\n"
+    "static inline bool $nested_name$_Parse(const ::std::string& name,\n"
+    "    $nested_name$* value) {\n"
+    "  return $classname$_Parse(name, value);\n"
+    "}\n"
     "static const $nested_name$ $nested_name$_MIN =\n"
     "  $classname$_$nested_name$_MIN;\n"
     "static const $nested_name$ $nested_name$_MAX =\n"
 void EnumGenerator::GenerateMethods(io::Printer* printer) {
   map<string, string> vars;
   vars["classname"] = classname_;
-  vars["builddescriptorsname"] =
-      GlobalBuildDescriptorsName(descriptor_->file()->name());
 
   printer->Print(vars,
     "const ::google::protobuf::EnumDescriptor* $classname$_descriptor() {\n"
-    "  if ($classname$_descriptor_ == NULL) $builddescriptorsname$();\n"
+    "  protobuf_AssignDescriptorsOnce();\n"
     "  return $classname$_descriptor_;\n"
     "}\n"
     "bool $classname$_IsValid(int value) {\n"

File src/google/protobuf/compiler/cpp/cpp_enum_field.cc

 }
 
 void EnumFieldGenerator::
-GenerateInitializer(io::Printer* printer) const {
-  printer->Print(variables_, ",\n$name$_($default$)");
+GenerateConstructorCode(io::Printer* printer) const {
+  printer->Print(variables_, "$name$_ = $default$;\n");
 }
 
 void EnumFieldGenerator::
     "if ($type$_IsValid(value)) {\n"
     "  set_$name$(static_cast< $type$ >(value));\n"
     "} else {\n"
-    "  mutable_unknown_fields()->AddField($number$)->add_varint(value);\n"
+    "  mutable_unknown_fields()->AddVarint($number$, value);\n"
     "}\n");
 }
 
 void EnumFieldGenerator::
 GenerateSerializeWithCachedSizes(io::Printer* printer) const {
   printer->Print(variables_,
-    "DO_(::google::protobuf::internal::WireFormat::WriteEnum("
-      "$number$, this->$name$(), output));\n");
+    "::google::protobuf::internal::WireFormat::WriteEnum("
+      "$number$, this->$name$(), output);\n");
+}
+
+void EnumFieldGenerator::
+GenerateSerializeWithCachedSizesToArray(io::Printer* printer) const {
+  printer->Print(variables_,
+    "target = ::google::protobuf::internal::WireFormat::WriteEnumToArray("
+      "$number$, this->$name$(), target);\n");
 }
 
 void EnumFieldGenerator::
 }
 
 void RepeatedEnumFieldGenerator::
-GenerateInitializer(io::Printer* printer) const {
-  printer->Print(variables_, ",\n$name$_()");
-  if (descriptor_->options().packed() &&
-      descriptor_->file()->options().optimize_for() == FileOptions::SPEED) {
-    printer->Print(variables_, ",\n_$name$_cached_byte_size_()");
-  }
+GenerateConstructorCode(io::Printer* printer) const {
+  // Not needed for repeated fields.
 }
 
 void RepeatedEnumFieldGenerator::
       "if ($type$_IsValid(value)) {\n"
       "  add_$name$(static_cast< $type$ >(value));\n"
       "} else {\n"
-      "  mutable_unknown_fields()->AddField($number$)->add_varint(value);\n"
+      "  mutable_unknown_fields()->AddVarint($number$, value);\n"
       "}\n");
   }
 }
     // Write the tag and the size.
     printer->Print(variables_,
       "if (this->$name$_size() > 0) {\n"
-      "  DO_(::google::protobuf::internal::WireFormat::WriteTag("
-          "$number$, ::google::protobuf::internal::WireFormat::WIRETYPE_LENGTH_DELIMITED,"
-          "output));\n"
-      "  DO_(output->WriteVarint32(_$name$_cached_byte_size_));\n"
+      "  ::google::protobuf::internal::WireFormat::WriteTag("
+          "$number$, "
+          "::google::protobuf::internal::WireFormat::WIRETYPE_LENGTH_DELIMITED, "
+          "output);\n"
+      "  output->WriteVarint32(_$name$_cached_byte_size_);\n"
       "}\n");
   }
   printer->Print(variables_,
       "for (int i = 0; i < this->$name$_size(); i++) {\n");
   if (descriptor_->options().packed()) {
     printer->Print(variables_,
-      "  DO_(::google::protobuf::internal::WireFormat::WriteEnumNoTag("
-          "this->$name$(i), output));\n");
+      "  ::google::protobuf::internal::WireFormat::WriteEnumNoTag("
+          "this->$name$(i), output);\n");
   } else {
     printer->Print(variables_,
-      "  DO_(::google::protobuf::internal::WireFormat::WriteEnum("
-          "$number$, this->$name$(i), output));\n");
+      "  ::google::protobuf::internal::WireFormat::WriteEnum("
+          "$number$, this->$name$(i), output);\n");
+  }
+  printer->Print("}\n");
+}
+
+void RepeatedEnumFieldGenerator::
+GenerateSerializeWithCachedSizesToArray(io::Printer* printer) const {
+  if (descriptor_->options().packed()) {
+    // Write the tag and the size.
+    printer->Print(variables_,
+      "if (this->$name$_size() > 0) {\n"
+      "  target = ::google::protobuf::internal::WireFormat::WriteTagToArray("
+          "$number$, "
+          "::google::protobuf::internal::WireFormat::WIRETYPE_LENGTH_DELIMITED, "
+          "target);\n"
+      "  target = ::google::protobuf::io::CodedOutputStream::WriteVarint32ToArray("
+          "_$name$_cached_byte_size_, target);\n"
+      "}\n");
+  }
+  printer->Print(variables_,
+      "for (int i = 0; i < this->$name$_size(); i++) {\n");
+  if (descriptor_->options().packed()) {
+    printer->Print(variables_,
+      "  target = ::google::protobuf::internal::WireFormat::WriteEnumNoTagToArray("
+          "this->$name$(i), target);\n");
+  } else {
+    printer->Print(variables_,
+      "  target = ::google::protobuf::internal::WireFormat::WriteEnumToArray("
+          "$number$, this->$name$(i), target);\n");
   }
   printer->Print("}\n");
 }

File src/google/protobuf/compiler/cpp/cpp_enum_field.h

   void GenerateClearingCode(io::Printer* printer) const;
   void GenerateMergingCode(io::Printer* printer) const;
   void GenerateSwappingCode(io::Printer* printer) const;
-  void GenerateInitializer(io::Printer* printer) const;
+  void GenerateConstructorCode(io::Printer* printer) const;
   void GenerateMergeFromCodedStream(io::Printer* printer) const;
   void GenerateSerializeWithCachedSizes(io::Printer* printer) const;
+  void GenerateSerializeWithCachedSizesToArray(io::Printer* printer) const;
   void GenerateByteSize(io::Printer* printer) const;
 
  private:
   void GenerateClearingCode(io::Printer* printer) const;
   void GenerateMergingCode(io::Printer* printer) const;
   void GenerateSwappingCode(io::Printer* printer) const;
-  void GenerateInitializer(io::Printer* printer) const;
+  void GenerateConstructorCode(io::Printer* printer) const;
   void GenerateMergeFromCodedStream(io::Printer* printer) const;
   void GenerateSerializeWithCachedSizes(io::Printer* printer) const;
+  void GenerateSerializeWithCachedSizesToArray(io::Printer* printer) const;
   void GenerateByteSize(io::Printer* printer) const;
 
  private:

File src/google/protobuf/compiler/cpp/cpp_extension.cc

 #include <google/protobuf/compiler/cpp/cpp_helpers.h>
 #include <google/protobuf/stubs/strutil.h>
 #include <google/protobuf/io/printer.h>
+#include <google/protobuf/descriptor.pb.h>
 
 namespace google {
 namespace protobuf {
     case FieldDescriptor::CPPTYPE_ENUM:
       type_traits_.append("EnumTypeTraits< ");
       type_traits_.append(ClassName(descriptor_->enum_type(), true));
-      type_traits_.append(" >");
+      type_traits_.append(", ");
+      type_traits_.append(ClassName(descriptor_->enum_type(), true));
+      type_traits_.append("_IsValid>");
       break;
     case FieldDescriptor::CPPTYPE_STRING:
       type_traits_.append("StringTypeTraits");