David Schneider avatar David Schneider committed 02a8668 Merge

merge default

Comments (0)

Files changed (195)

File contents unchanged.

lib-python/modified-2.7/test/test_set.py

             for meth in (s.union, s.intersection, s.difference, s.symmetric_difference, s.isdisjoint):
                 for g in (G, I, Ig, L, R):
                     expected = meth(data)
-                    actual = meth(G(data))
+                    actual = meth(g(data))
                     if isinstance(expected, bool):
                         self.assertEqual(actual, expected)
                     else:

File contents unchanged.

Add a comment to this file

lib_pypy/pyrepl/unix_console.py

File contents unchanged.

File contents unchanged.

File contents unchanged.

Add a comment to this file

pypy/annotation/builtin.py

File contents unchanged.

Add a comment to this file

pypy/annotation/model.py

File contents unchanged.

Add a comment to this file

pypy/doc/coding-guide.rst

File contents unchanged.

pypy/doc/discussion/win64_todo.txt

+2011-11-04
+ll_os.py has a problem with the file rwin32.py.
+Temporarily disabled for the win64_gborg branch. This needs to be
+investigated and re-enabled.
+Resolved, enabled.
+
+2011-11-05
+test_typed.py needs explicit tests to ensure that we
+handle word sizes right.
Add a comment to this file

pypy/doc/how-to-release.rst

File contents unchanged.

Add a comment to this file

pypy/doc/project-ideas.rst

File contents unchanged.

pypy/doc/windows.rst

 Edition.  Other configurations may work as well.
 
 The translation scripts will set up the appropriate environment variables
-for the compiler.  They will attempt to locate the same compiler version that
+for the compiler, so you do not need to run vcvars before translation.  
+They will attempt to locate the same compiler version that
 was used to build the Python interpreter doing the
 translation.  Failing that, they will pick the most recent Visual Studio
 compiler they can find.  In addition, the target architecture
 using a 32 bit Python and vice versa.
 
 **Note:** PyPy is currently not supported for 64 bit Windows, and translation
-will be aborted in this case.
+will fail in this case.
 
 The compiler is all you need to build pypy-c, but it will miss some
 modules that relies on third-party libraries.  See below how to get
 install third-party libraries.  We chose to install them in the parent
 directory of the pypy checkout.  For example, if you installed pypy in
 ``d:\pypy\trunk\`` (This directory contains a README file), the base
-directory is ``d:\pypy``.
+directory is ``d:\pypy``. You may choose different values by setting the
+INCLUDE, LIB and PATH (for DLLs)
 
 The Boehm garbage collector
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~
 ------------------------
 
 You can compile pypy with the mingw compiler, using the --cc=mingw32 option;
-mingw.exe must be on the PATH.
+gcc.exe must be on the PATH. If the -cc flag does not begin with "ming", it should be
+the name of a valid gcc-derivative compiler, i.e. x86_64-w64-mingw32-gcc for the 64 bit
+compiler creating a 64 bit target.
 
-libffi for the mingw32 compiler
+libffi for the mingw compiler
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-To enable the _rawffi (and ctypes) module, you need to compile a mingw32
-version of libffi.  I downloaded the `libffi source files`_, and extracted
-them in the base directory.  Then run::
+To enable the _rawffi (and ctypes) module, you need to compile a mingw
+version of libffi.  Here is one way to do this, wich should allow you to try
+to build for win64 or win32:
+
+#. Download and unzip a `mingw32 build`_ or `mingw64 build`_, say into c:\mingw
+#. If you do not use cygwin, you will need msys to provide make, 
+   autoconf tools and other goodies.
+
+    #. Download and unzip a `msys for mingw`_, say into c:\msys
+    #. Edit the c:\msys\etc\fstab file to mount c:\mingw
+
+#. Download and unzip the `libffi source files`_, and extract
+   them in the base directory.  
+#. Run c:\msys\msys.bat or a cygwin shell which should make you
+   feel better since it is a shell prompt with shell tools.
+#. From inside the shell, cd to the libffi directory and do::
 
     sh ./configure
     make
     cp .libs/libffi-5.dll <somewhere on the PATH>
 
+If you can't find the dll, and the libtool issued a warning about 
+"undefined symbols not allowed", you will need to edit the libffi
+Makefile in the toplevel directory. Add the flag -no-undefined to
+the definition of libffi_la_LDFLAGS
+
+If you wish to experiment with win64, you must run configure with flags::
+
+    sh ./configure --build=x86_64-w64-mingw32 --host=x86_64-w64-mingw32
+
+or such, depending on your mingw64 download.
+
+hacking on Pypy with the mingw compiler
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Since hacking on Pypy means running tests, you will need a way to specify
+the mingw compiler when hacking (as opposed to translating). As of
+March 2012, --cc is not a valid option for pytest.py. However if you set an
+environment variable CC it will allow you to choose a compiler.
+
+.. _'mingw32 build': http://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win32/Automated%20Builds
+.. _`mingw64 build`: http://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win64/Automated%20Builds
+.. _`msys for mingw`: http://sourceforge.net/projects/mingw-w64/files/External%20binary%20packages%20%28Win64%20hosted%29/MSYS%20%2832-bit%29   
 .. _`libffi source files`: http://sourceware.org/libffi/
 .. _`RPython translation toolchain`: translation.html

pypy/doc/you-want-to-help.rst

+
+You want to help with PyPy, now what?
+=====================================
+
+PyPy is a very large project that has a reputation of being hard to dive into.
+Some of this fame is warranted, some of it is purely accidental. There are three
+important lessons that everyone willing to contribute should learn:
+
+* PyPy has layers. There are many pieces of architecture that are very well
+  separated from each other. More about this below, but often the manifestation
+  of this is that things are at a different layer than you would expect them
+  to be. For example if you are looking for the JIT implementation, you will
+  not find it in the implementation of the Python programming language.
+
+* Because of the above, we are very serious about Test Driven Development.
+  It's not only what we believe in, but also that PyPy's architecture is
+  working very well with TDD in mind and not so well without it. Often
+  the development means progressing in an unrelated corner, one unittest
+  at a time; and then flipping a giant switch, bringing it all together.
+  (It generally works out of the box.  If it doesn't, then we didn't
+  write enough unit tests.)  It's worth repeating - PyPy
+  approach is great if you do TDD, not so great otherwise.
+
+* PyPy uses an entirely different set of tools - most of them included
+  in the PyPy repository. There is no Makefile, nor autoconf. More below
+
+Architecture
+============
+
+PyPy has layers. The 100 miles view:
+
+* `RPython`_ is the language in which we write interpreters. Not the entire
+  PyPy project is written in RPython, only the parts that are compiled in
+  the translation process. The interesting point is that RPython has no parser,
+  it's compiled from the live python objects, which make it possible to do
+  all kinds of metaprogramming during import time. In short, Python is a meta
+  programming language for RPython.
+
+  The RPython standard library is to be found in the ``rlib`` subdirectory.
+
+.. _`RPython`: coding-guide.html#RPython
+
+* The translation toolchain - this is the part that takes care about translating
+  RPython to flow graphs and then to C. There is more in the `architecture`_
+  document written about it.
+
+  It mostly lives in ``rpython``, ``annotator`` and ``objspace/flow``.
+
+.. _`architecture`: architecture.html 
+
+* Python Interpreter
+
+  xxx
+
+* Python modules
+
+  xxx
+
+* Just-in-Time Compiler (JIT): `we have a tracing JIT`_ that traces the
+  interpreter written in RPython, rather than the user program that it
+  interprets.  As a result it applies to any interpreter, i.e. any
+  language.  But getting it to work correctly is not trivial: it
+  requires a small number of precise "hints" and possibly some small
+  refactorings of the interpreter.  The JIT itself also has several
+  almost-independent parts: the tracer itself in ``jit/metainterp``, the
+  optimizer in ``jit/metainterp/optimizer`` that optimizes a list of
+  residual operations, and the backend in ``jit/backend/<machine-name>``
+  that turns it into machine code.  Writing a new backend is a
+  traditional way to get into the project.
+
+.. _`we have a tracing JIT`: jit/index.html
+
+* Garbage Collectors (GC): as you can notice if you are used to CPython's
+  C code, there are no ``Py_INCREF/Py_DECREF`` equivalents in RPython code.
+  `Garbage collection in PyPy`_ is inserted
+  during translation.  Moreover, this is not reference counting; it is a real
+  GC written as more RPython code.  The best one we have so far is in
+  ``rpython/memory/gc/minimark.py``.
+
+.. _`Garbage collection in PyPy`: garbage_collection.html
+
+
+Toolset
+=======
+
+xxx

pypy/interpreter/astcompiler/test/test_astbuilder.py

 from pypy.interpreter.astcompiler import ast, consts
 
 
-try:
-    all
-except NameError:
-    def all(iterable):
-        for x in iterable:
-            if not x:
-                return False
-        return True
-
-
 class TestAstBuilder:
 
     def setup_class(cls):
Add a comment to this file

pypy/interpreter/astcompiler/test/test_compiler.py

File contents unchanged.

Add a comment to this file

pypy/interpreter/baseobjspace.py

File contents unchanged.

Add a comment to this file

pypy/jit/backend/arm/test/test_ztranslation.py

File contents unchanged.

Add a comment to this file

pypy/jit/backend/conftest.py

File contents unchanged.

Add a comment to this file

pypy/jit/backend/llgraph/llimpl.py

File contents unchanged.

Add a comment to this file

pypy/jit/backend/llgraph/runner.py

File contents unchanged.

Add a comment to this file

pypy/jit/backend/llsupport/descr.py

File contents unchanged.

Add a comment to this file

pypy/jit/backend/llsupport/gc.py

File contents unchanged.

Add a comment to this file

pypy/jit/backend/llsupport/llmodel.py

File contents unchanged.

Add a comment to this file

pypy/jit/backend/llsupport/test/test_descr.py

File contents unchanged.

Add a comment to this file

pypy/jit/backend/llsupport/test/test_ffisupport.py

File contents unchanged.

Add a comment to this file

pypy/jit/backend/llsupport/test/test_gc.py

File contents unchanged.

Add a comment to this file

pypy/jit/backend/model.py

File contents unchanged.

Add a comment to this file

pypy/jit/backend/test/runner_test.py

File contents unchanged.

Add a comment to this file

pypy/jit/backend/test/support.py

File contents unchanged.

Add a comment to this file

pypy/jit/backend/test/test_random.py

File contents unchanged.

Add a comment to this file

pypy/jit/backend/x86/assembler.py

File contents unchanged.

Add a comment to this file

pypy/jit/backend/x86/codebuf.py

File contents unchanged.

pypy/jit/backend/x86/regalloc.py

             self.Perform(op, [loc0], loc1)
             self.xrm.possibly_free_var(op.getarg(0))
         else:
-            loc0 = self.xrm.loc(op.getarg(0))
-            loc1 = self.xrm.force_allocate_reg(op.result)
+            arg0 = op.getarg(0)
+            loc0 = self.xrm.loc(arg0)
+            loc1 = self.xrm.force_allocate_reg(op.result, forbidden_vars=[arg0])
             self.Perform(op, [loc0], loc1)
             self.xrm.possibly_free_var(op.getarg(0))
 
Add a comment to this file

pypy/jit/backend/x86/runner.py

File contents unchanged.

Add a comment to this file

pypy/jit/backend/x86/test/test_assembler.py

File contents unchanged.

Add a comment to this file

pypy/jit/backend/x86/test/test_ztranslation.py

File contents unchanged.

Add a comment to this file

pypy/jit/backend/x86/tool/viewcode.py

File contents unchanged.

Add a comment to this file

pypy/jit/codewriter/codewriter.py

File contents unchanged.

Add a comment to this file

pypy/jit/codewriter/effectinfo.py

File contents unchanged.

Add a comment to this file

pypy/jit/codewriter/jtransform.py

File contents unchanged.

Add a comment to this file

pypy/jit/codewriter/test/test_longlong.py

File contents unchanged.

Add a comment to this file

pypy/jit/metainterp/executor.py

File contents unchanged.

Add a comment to this file

pypy/jit/metainterp/history.py

File contents unchanged.

Add a comment to this file

pypy/jit/metainterp/optimizeopt/fficall.py

File contents unchanged.

Add a comment to this file

pypy/jit/metainterp/optimizeopt/intutils.py

File contents unchanged.

Add a comment to this file

pypy/jit/metainterp/optimizeopt/test/test_optimizeopt.py

File contents unchanged.

Add a comment to this file

pypy/jit/metainterp/optimizeopt/test/test_util.py

File contents unchanged.

Add a comment to this file

pypy/jit/metainterp/optimizeopt/vstring.py

File contents unchanged.

Add a comment to this file

pypy/jit/metainterp/resoperation.py

File contents unchanged.

Add a comment to this file

pypy/jit/metainterp/test/test_ajit.py

File contents unchanged.

Add a comment to this file

pypy/jit/metainterp/test/test_fficall.py

File contents unchanged.

Add a comment to this file

pypy/jit/metainterp/test/test_jitdriver.py

File contents unchanged.

Add a comment to this file

pypy/jit/metainterp/test/test_loop.py

File contents unchanged.

Add a comment to this file

pypy/jit/metainterp/test/test_recursive.py

File contents unchanged.

Add a comment to this file

pypy/jit/metainterp/test/test_warmspot.py

File contents unchanged.

Add a comment to this file

pypy/jit/metainterp/test/test_ztranslation.py

File contents unchanged.

Add a comment to this file

pypy/jit/metainterp/warmspot.py

File contents unchanged.

File contents unchanged.

pypy/module/__pypy__/__init__.py

     appleveldefs = {}
     interpleveldefs = {}
     if sys.platform.startswith("linux"):
+        from pypy.module.__pypy__ import interp_time
         interpleveldefs["clock_gettime"] = "interp_time.clock_gettime"
         interpleveldefs["clock_getres"] = "interp_time.clock_getres"
         for name in [
             "CLOCK_REALTIME", "CLOCK_MONOTONIC", "CLOCK_MONOTONIC_RAW",
             "CLOCK_PROCESS_CPUTIME_ID", "CLOCK_THREAD_CPUTIME_ID"
         ]:
-            interpleveldefs[name] = "space.wrap(interp_time.%s)" % name
+            if getattr(interp_time, name) is not None:
+                interpleveldefs[name] = "space.wrap(interp_time.%s)" % name
 
 
 class Module(MixedModule):

pypy/module/__pypy__/interp_time.py

+from __future__ import with_statement
 import sys
 
 from pypy.interpreter.error import exception_from_errno
Add a comment to this file

pypy/module/_ffi/test/test__ffi.py

File contents unchanged.

Add a comment to this file

pypy/module/_multiprocessing/test/test_semaphore.py

File contents unchanged.

Add a comment to this file

pypy/module/_ssl/test/test_ssl.py

File contents unchanged.

Add a comment to this file

pypy/module/cpyext/api.py

File contents unchanged.

Add a comment to this file

pypy/module/cpyext/include/Python.h

File contents unchanged.

Add a comment to this file

pypy/module/cpyext/include/modsupport.h

File contents unchanged.

Add a comment to this file

pypy/module/cpyext/presetup.py

File contents unchanged.

Add a comment to this file

pypy/module/cpyext/slotdefs.py

File contents unchanged.

Add a comment to this file

pypy/module/cpyext/test/test_typeobject.py

File contents unchanged.

Add a comment to this file

pypy/module/micronumpy/__init__.py

File contents unchanged.

pypy/module/micronumpy/app_numpy.py

         a[i][i] = 1
     return a
 
-def sum(a,axis=None):
+def sum(a,axis=None, out=None):
     '''sum(a, axis=None)
     Sum of array elements over a given axis.
 
     # TODO: add to doc (once it's implemented): cumsum : Cumulative sum of array elements.
     if not hasattr(a, "sum"):
         a = _numpypy.array(a)
-    return a.sum(axis)
+    return a.sum(axis=axis, out=out)
 
-def min(a, axis=None):
+def min(a, axis=None, out=None):
     if not hasattr(a, "min"):
         a = _numpypy.array(a)
-    return a.min(axis)
+    return a.min(axis=axis, out=out)
 
-def max(a, axis=None):
+def max(a, axis=None, out=None):
     if not hasattr(a, "max"):
         a = _numpypy.array(a)
-    return a.max(axis)
+    return a.max(axis=axis, out=out)
 
 def arange(start, stop=None, step=1, dtype=None):
     '''arange([start], stop[, step], dtype=None)
Add a comment to this file

pypy/module/micronumpy/bench/add.py

File contents unchanged.

Add a comment to this file

pypy/module/micronumpy/bench/iterate.py

File contents unchanged.

pypy/module/micronumpy/interp_boxes.py

         return space.wrap(dtype.itemtype.bool(self))
 
     def _binop_impl(ufunc_name):
-        def impl(self, space, w_other):
+        def impl(self, space, w_other, w_out=None):
             from pypy.module.micronumpy import interp_ufuncs
-            return getattr(interp_ufuncs.get(space), ufunc_name).call(space, [self, w_other])
+            return getattr(interp_ufuncs.get(space), ufunc_name).call(space,
+                                                            [self, w_other, w_out])
         return func_with_new_name(impl, "binop_%s_impl" % ufunc_name)
 
     def _binop_right_impl(ufunc_name):
-        def impl(self, space, w_other):
+        def impl(self, space, w_other, w_out=None):
             from pypy.module.micronumpy import interp_ufuncs
-            return getattr(interp_ufuncs.get(space), ufunc_name).call(space, [w_other, self])
+            return getattr(interp_ufuncs.get(space), ufunc_name).call(space, 
+                                                            [w_other, self, w_out])
         return func_with_new_name(impl, "binop_right_%s_impl" % ufunc_name)
 
     def _unaryop_impl(ufunc_name):
-        def impl(self, space):
+        def impl(self, space, w_out=None):
             from pypy.module.micronumpy import interp_ufuncs
-            return getattr(interp_ufuncs.get(space), ufunc_name).call(space, [self])
+            return getattr(interp_ufuncs.get(space), ufunc_name).call(space,
+                                                                    [self, w_out])
         return func_with_new_name(impl, "unaryop_%s_impl" % ufunc_name)
 
     descr_add = _binop_impl("add")

pypy/module/micronumpy/interp_iter.py

 
     def apply_transformations(self, arr, transformations):
         v = BaseIterator.apply_transformations(self, arr, transformations)
-        if len(arr.shape) == 1:
+        if len(arr.shape) == 1 and len(v.res_shape) == 1:
             return OneDimIterator(self.offset, self.strides[0],
                                   self.res_shape[0])
         return v

pypy/module/micronumpy/interp_numarray.py

         return space.wrap(W_NDimArray(shape[:], dtype=dtype))
 
     def _unaryop_impl(ufunc_name):
-        def impl(self, space):
-            return getattr(interp_ufuncs.get(space), ufunc_name).call(space, [self])
+        def impl(self, space, w_out=None):
+            return getattr(interp_ufuncs.get(space), ufunc_name).call(space,
+                                                                [self, w_out])
         return func_with_new_name(impl, "unaryop_%s_impl" % ufunc_name)
 
     descr_pos = _unaryop_impl("positive")
     descr_invert = _unaryop_impl("invert")
 
     def _binop_impl(ufunc_name):
-        def impl(self, space, w_other):
-            return getattr(interp_ufuncs.get(space), ufunc_name).call(space, [self, w_other])
+        def impl(self, space, w_other, w_out=None):
+            return getattr(interp_ufuncs.get(space), ufunc_name).call(space,
+                                                        [self, w_other, w_out])
         return func_with_new_name(impl, "binop_%s_impl" % ufunc_name)
 
     descr_add = _binop_impl("add")
         return space.newtuple([w_quotient, w_remainder])
 
     def _binop_right_impl(ufunc_name):
-        def impl(self, space, w_other):
+        def impl(self, space, w_other, w_out=None):
             w_other = scalar_w(space,
                 interp_ufuncs.find_dtype_for_scalar(space, w_other, self.find_dtype()),
                 w_other
             )
-            return getattr(interp_ufuncs.get(space), ufunc_name).call(space, [w_other, self])
+            return getattr(interp_ufuncs.get(space), ufunc_name).call(space, [w_other, self, w_out])
         return func_with_new_name(impl, "binop_right_%s_impl" % ufunc_name)
 
     descr_radd = _binop_right_impl("add")
         return space.newtuple([w_quotient, w_remainder])
 
     def _reduce_ufunc_impl(ufunc_name, promote_to_largest=False):
-        def impl(self, space, w_axis=None):
+        def impl(self, space, w_axis=None, w_out=None):
             if space.is_w(w_axis, space.w_None):
                 axis = -1
             else:
                 axis = space.int_w(w_axis)
+            if space.is_w(w_out, space.w_None) or not w_out:
+                out = None
+            elif not isinstance(w_out, BaseArray):
+                raise OperationError(space.w_TypeError, space.wrap( 
+                        'output must be an array'))
+            else:
+                out = w_out
             return getattr(interp_ufuncs.get(space), ufunc_name).reduce(space,
-                                        self, True, promote_to_largest, axis)
+                                        self, True, promote_to_largest, axis,
+                                                                   False, out)
         return func_with_new_name(impl, "reduce_%s_impl" % ufunc_name)
 
     descr_sum = _reduce_ufunc_impl("add")
     def descr_dot(self, space, w_other):
         other = convert_to_array(space, w_other)
         if isinstance(other, Scalar):
+            #Note: w_out is not modified, this is numpy compliant.
             return self.descr_mul(space, other)
         elif len(self.shape) < 2 and len(other.shape) < 2:
             w_res = self.descr_mul(space, other)
             )
         return w_result
 
-    def descr_mean(self, space, w_axis=None):
+    def descr_mean(self, space, w_axis=None, w_out=None):
         if space.is_w(w_axis, space.w_None):
             w_axis = space.wrap(-1)
             w_denom = space.wrap(support.product(self.shape))
         else:
             dim = space.int_w(w_axis)
             w_denom = space.wrap(self.shape[dim])
-        return space.div(self.descr_sum_promote(space, w_axis), w_denom)
+        return space.div(self.descr_sum_promote(space, w_axis, w_out), w_denom)
 
     def descr_var(self, space, w_axis=None):
         return get_appbridge_cache(space).call_method(space, '_var', self,
     """
     Class for representing virtual arrays, such as binary ops or ufuncs
     """
-    def __init__(self, name, shape, res_dtype):
+    def __init__(self, name, shape, res_dtype, out_arg=None):
         BaseArray.__init__(self, shape)
         self.forced_result = None
         self.res_dtype = res_dtype
         self.name = name
+        self.res = out_arg
         self.size = support.product(self.shape) * res_dtype.get_size()
 
     def _del_sources(self):
         raise NotImplementedError
 
     def compute(self):
-        ra = ResultArray(self, self.shape, self.res_dtype)
+        ra = ResultArray(self, self.shape, self.res_dtype, self.res)
         loop.compute(ra)
+        if self.res:
+            broadcast_dims = len(self.res.shape) - len(self.shape)
+            chunks = [Chunk(0,0,0,0)] * broadcast_dims + \
+                     [Chunk(0, i, 1, i) for i in self.shape]
+            return Chunks(chunks).apply(self.res)
         return ra.left
 
     def force_if_needed(self):
         if self.forced_result is None:
-            self.forced_result = self.compute()
+            self.forced_result = self.compute().get_concrete()
             self._del_sources()
 
     def get_concrete(self):
 
 
 class Call1(VirtualArray):
-    def __init__(self, ufunc, name, shape, calc_dtype, res_dtype, values):
-        VirtualArray.__init__(self, name, shape, res_dtype)
+    def __init__(self, ufunc, name, shape, calc_dtype, res_dtype, values,
+                                                            out_arg=None):
+        VirtualArray.__init__(self, name, shape, res_dtype, out_arg)
         self.values = values
         self.size = values.size
         self.ufunc = ufunc
     def create_sig(self):
         if self.forced_result is not None:
             return self.forced_result.create_sig()
+        if self.shape != self.values.shape:
+            #This happens if out arg is used
+            return signature.BroadcastUfunc(self.ufunc, self.name,
+                                            self.calc_dtype,
+                                            self.values.create_sig(),
+                                            self.res.create_sig())
         return signature.Call1(self.ufunc, self.name, self.calc_dtype,
                                self.values.create_sig())
 
     """
     Intermediate class for performing binary operations.
     """
-    def __init__(self, ufunc, name, shape, calc_dtype, res_dtype, left, right):
-        VirtualArray.__init__(self, name, shape, res_dtype)
+    def __init__(self, ufunc, name, shape, calc_dtype, res_dtype, left, right,
+            out_arg=None):
+        VirtualArray.__init__(self, name, shape, res_dtype, out_arg)
         self.ufunc = ufunc
         self.left = left
         self.right = right
         Call2.__init__(self, None, 'assign', shape, dtype, dtype, res, child)
 
     def create_sig(self):
-        return signature.ResultSignature(self.res_dtype, self.left.create_sig(),
-                                         self.right.create_sig())
+        if self.left.shape != self.right.shape:
+            sig = signature.BroadcastResultSignature(self.res_dtype,
+                        self.left.create_sig(), self.right.create_sig())
+        else:
+            sig = signature.ResultSignature(self.res_dtype, 
+                        self.left.create_sig(), self.right.create_sig())
+        return sig
 
 class ToStringArray(Call1):
     def __init__(self, child):
         self.s = StringBuilder(child.size * self.item_size)
         Call1.__init__(self, None, 'tostring', child.shape, dtype, dtype,
                        child)
-        self.res = W_NDimArray([1], dtype, 'C')
-        self.res_casted = rffi.cast(rffi.CArrayPtr(lltype.Char),
-                                    self.res.storage)
+        self.res_str = W_NDimArray([1], dtype, order='C')
+        self.res_str_casted = rffi.cast(rffi.CArrayPtr(lltype.Char),
+                                    self.res_str.storage)
 
     def create_sig(self):
         return signature.ToStringSignature(self.calc_dtype,
 
     def setitem(self, item, value):
         self.invalidated()
-        self.dtype.setitem(self, item, value)
+        self.dtype.setitem(self, item, value.convert_to(self.dtype))
 
     def calc_strides(self, shape):
         dtype = self.find_dtype()

pypy/module/micronumpy/interp_ufuncs.py

         return self.identity
 
     def descr_call(self, space, __args__):
+	from interp_numarray import BaseArray
         args_w, kwds_w = __args__.unpack()
         # it occurs to me that we don't support any datatypes that
         # require casting, change it later when we do
         kwds_w.pop('casting', None)
         w_subok = kwds_w.pop('subok', None)
         w_out = kwds_w.pop('out', space.w_None)
-        if ((w_subok is not None and space.is_true(w_subok)) or
-            not space.is_w(w_out, space.w_None)):
+        # Setup a default value for out
+        if space.is_w(w_out, space.w_None):
+            out = None
+        else:
+            out = w_out
+        if (w_subok is not None and space.is_true(w_subok)):
             raise OperationError(space.w_NotImplementedError,
                                  space.wrap("parameters unsupported"))
         if kwds_w or len(args_w) < self.argcount:
             raise OperationError(space.w_ValueError,
                 space.wrap("invalid number of arguments")
             )
-        elif len(args_w) > self.argcount:
-            # The extra arguments should actually be the output array, but we
-            # don't support that yet.
+        elif (len(args_w) > self.argcount and out is not None) or \
+             (len(args_w) > self.argcount + 1):
             raise OperationError(space.w_TypeError,
                 space.wrap("invalid number of arguments")
             )
+        # Override the default out value, if it has been provided in w_wargs
+        if len(args_w) > self.argcount:
+            out = args_w[-1]
+        else:
+            args_w = args_w[:] + [out]
+        if out is not None and not isinstance(out, BaseArray):
+            raise OperationError(space.w_TypeError, space.wrap(
+                                            'output must be an array'))
         return self.call(space, args_w)
 
     @unwrap_spec(skipna=bool, keepdims=bool)
         array([[ 1,  5],
                [ 9, 13]])
         """
-        if not space.is_w(w_out, space.w_None):
-            raise OperationError(space.w_NotImplementedError, space.wrap(
-                "out not supported"))
+        from pypy.module.micronumpy.interp_numarray import BaseArray
         if w_axis is None:
             axis = 0
         elif space.is_w(w_axis, space.w_None):
             axis = -1
         else:
             axis = space.int_w(w_axis)
-        return self.reduce(space, w_obj, False, False, axis, keepdims)
+        if space.is_w(w_out, space.w_None):
+            out = None
+        elif not isinstance(w_out, BaseArray):
+            raise OperationError(space.w_TypeError, space.wrap(
+                                                'output must be an array'))
+        else:
+            out = w_out
+        return self.reduce(space, w_obj, False, False, axis, keepdims, out)
 
-    def reduce(self, space, w_obj, multidim, promote_to_largest, dim,
-               keepdims=False):
+    def reduce(self, space, w_obj, multidim, promote_to_largest, axis,
+               keepdims=False, out=None):
         from pypy.module.micronumpy.interp_numarray import convert_to_array, \
-                                                           Scalar, ReduceArray
+                                             Scalar, ReduceArray, W_NDimArray
         if self.argcount != 2:
             raise OperationError(space.w_ValueError, space.wrap("reduce only "
                 "supported for binary functions"))
         assert isinstance(self, W_Ufunc2)
         obj = convert_to_array(space, w_obj)
-        if dim >= len(obj.shape):
-            raise OperationError(space.w_ValueError, space.wrap("axis(=%d) out of bounds" % dim))
+        if axis >= len(obj.shape):
+            raise OperationError(space.w_ValueError, space.wrap("axis(=%d) out of bounds" % axis))
         if isinstance(obj, Scalar):
             raise OperationError(space.w_TypeError, space.wrap("cannot reduce "
                 "on a scalar"))
         if self.identity is None and size == 0:
             raise operationerrfmt(space.w_ValueError, "zero-size array to "
                     "%s.reduce without identity", self.name)
-        if shapelen > 1 and dim >= 0:
-            return self.do_axis_reduce(obj, dtype, dim, keepdims)
-        arr = ReduceArray(self.func, self.name, self.identity, obj, dtype)
-        return loop.compute(arr)
+        if shapelen > 1 and axis >= 0:
+            if keepdims:
+                shape = obj.shape[:axis] + [1] + obj.shape[axis + 1:]
+            else:
+                shape = obj.shape[:axis] + obj.shape[axis + 1:]
+            if out:
+                #Test for shape agreement
+                if len(out.shape) > len(shape):
+                    raise operationerrfmt(space.w_ValueError,
+                        'output parameter for reduction operation %s' +
+                        ' has too many dimensions', self.name)
+                elif len(out.shape) < len(shape):
+                    raise operationerrfmt(space.w_ValueError,
+                        'output parameter for reduction operation %s' +
+                        ' does not have enough dimensions', self.name)
+                elif out.shape != shape:
+                    raise operationerrfmt(space.w_ValueError,
+                        'output parameter shape mismatch, expecting [%s]' +
+                        ' , got [%s]', 
+                        ",".join([str(x) for x in shape]),
+                        ",".join([str(x) for x in out.shape]),
+                        )
+                #Test for dtype agreement, perhaps create an itermediate
+                #if out.dtype != dtype:
+                #    raise OperationError(space.w_TypeError, space.wrap(
+                #        "mismatched  dtypes"))
+                return self.do_axis_reduce(obj, out.find_dtype(), axis, out)
+            else:
+                result = W_NDimArray(shape, dtype)
+                return self.do_axis_reduce(obj, dtype, axis, result)
+        if out:
+            if len(out.shape)>0:
+                raise operationerrfmt(space.w_ValueError, "output parameter "
+                              "for reduction operation %s has too many"
+                              " dimensions",self.name)
+            arr = ReduceArray(self.func, self.name, self.identity, obj,
+                                                            out.find_dtype())
+            val = loop.compute(arr)
+            assert isinstance(out, Scalar)
+            out.value = val
+        else:
+            arr = ReduceArray(self.func, self.name, self.identity, obj, dtype)
+            val = loop.compute(arr)
+        return val 
 
-    def do_axis_reduce(self, obj, dtype, dim, keepdims):
-        from pypy.module.micronumpy.interp_numarray import AxisReduce,\
-             W_NDimArray
-        if keepdims:
-            shape = obj.shape[:dim] + [1] + obj.shape[dim + 1:]
-        else:
-            shape = obj.shape[:dim] + obj.shape[dim + 1:]
-        result = W_NDimArray(shape, dtype)
+    def do_axis_reduce(self, obj, dtype, axis, result):
+        from pypy.module.micronumpy.interp_numarray import AxisReduce
         arr = AxisReduce(self.func, self.name, self.identity, obj.shape, dtype,
-                         result, obj, dim)
+                         result, obj, axis)
         loop.compute(arr)
         return arr.left
 
         self.bool_result = bool_result
 
     def call(self, space, args_w):
-        from pypy.module.micronumpy.interp_numarray import (Call1,
-            convert_to_array, Scalar)
-
-        [w_obj] = args_w
+        from pypy.module.micronumpy.interp_numarray import (Call1, BaseArray,
+            convert_to_array, Scalar, shape_agreement)
+        if len(args_w)<2:
+            [w_obj] = args_w
+            out = None
+        else:
+            [w_obj, out] = args_w
+            if space.is_w(out, space.w_None):
+                out = None
         w_obj = convert_to_array(space, w_obj)
         calc_dtype = find_unaryop_result_dtype(space,
                                   w_obj.find_dtype(),
                                   promote_to_float=self.promote_to_float,
                                   promote_bools=self.promote_bools)
-        if self.bool_result:
+        if out:
+            if not isinstance(out, BaseArray):
+                raise OperationError(space.w_TypeError, space.wrap(
+                                                'output must be an array'))
+            res_dtype = out.find_dtype()
+        elif self.bool_result:
             res_dtype = interp_dtype.get_dtype_cache(space).w_booldtype
         else:
             res_dtype = calc_dtype
         if isinstance(w_obj, Scalar):
-            return space.wrap(self.func(calc_dtype, w_obj.value.convert_to(calc_dtype)))
-
-        w_res = Call1(self.func, self.name, w_obj.shape, calc_dtype, res_dtype,
-                      w_obj)
+            arr = self.func(calc_dtype, w_obj.value.convert_to(calc_dtype))
+            if isinstance(out,Scalar):
+                out.value=arr
+            elif isinstance(out, BaseArray):
+                out.fill(space, arr)
+            else:
+                out = arr
+            return space.wrap(out)
+        if out:
+            assert isinstance(out, BaseArray) # For translation
+            broadcast_shape =  shape_agreement(space, w_obj.shape, out.shape)
+            if not broadcast_shape or broadcast_shape != out.shape:
+                raise operationerrfmt(space.w_ValueError,
+                    'output parameter shape mismatch, could not broadcast [%s]' +
+                    ' to [%s]', 
+                    ",".join([str(x) for x in w_obj.shape]),
+                    ",".join([str(x) for x in out.shape]),
+                    )
+            w_res = Call1(self.func, self.name, out.shape, calc_dtype,
+                                         res_dtype, w_obj, out)
+            #Force it immediately
+            w_res.get_concrete()
+        else:
+            w_res = Call1(self.func, self.name, w_obj.shape, calc_dtype,
+                                         res_dtype, w_obj)
         w_obj.add_invalidates(w_res)
         return w_res
 
 
     def call(self, space, args_w):
         from pypy.module.micronumpy.interp_numarray import (Call2,
-            convert_to_array, Scalar, shape_agreement)
-
-        [w_lhs, w_rhs] = args_w
+            convert_to_array, Scalar, shape_agreement, BaseArray)
+        if len(args_w)>2:
+            [w_lhs, w_rhs, w_out] = args_w
+        else:
+            [w_lhs, w_rhs] = args_w
+            w_out = None
         w_lhs = convert_to_array(space, w_lhs)
         w_rhs = convert_to_array(space, w_rhs)
-        calc_dtype = find_binop_result_dtype(space,
-            w_lhs.find_dtype(), w_rhs.find_dtype(),
-            int_only=self.int_only,
-            promote_to_float=self.promote_to_float,
-            promote_bools=self.promote_bools,
-        )
+        if space.is_w(w_out, space.w_None) or w_out is None:
+            out = None
+            calc_dtype = find_binop_result_dtype(space,
+                w_lhs.find_dtype(), w_rhs.find_dtype(),
+                int_only=self.int_only,
+                promote_to_float=self.promote_to_float,
+                promote_bools=self.promote_bools,
+            )
+        elif not isinstance(w_out, BaseArray):
+            raise OperationError(space.w_TypeError, space.wrap(
+                    'output must be an array'))
+        else:
+            out = w_out
+            calc_dtype = out.find_dtype()
         if self.comparison_func:
             res_dtype = interp_dtype.get_dtype_cache(space).w_booldtype
         else:
             res_dtype = calc_dtype
         if isinstance(w_lhs, Scalar) and isinstance(w_rhs, Scalar):
-            return space.wrap(self.func(calc_dtype,
+            arr = self.func(calc_dtype,
                 w_lhs.value.convert_to(calc_dtype),
                 w_rhs.value.convert_to(calc_dtype)
-            ))
+            )
+            if isinstance(out,Scalar):
+                out.value=arr
+            elif isinstance(out, BaseArray):
+                out.fill(space, arr)
+            else:
+                out = arr
+            return space.wrap(out)
         new_shape = shape_agreement(space, w_lhs.shape, w_rhs.shape)
+        # Test correctness of out.shape
+        if out and out.shape != shape_agreement(space, new_shape, out.shape):
+            raise operationerrfmt(space.w_ValueError,
+                'output parameter shape mismatch, could not broadcast [%s]' +
+                ' to [%s]', 
+                ",".join([str(x) for x in new_shape]),
+                ",".join([str(x) for x in out.shape]),
+                )
         w_res = Call2(self.func, self.name,
                       new_shape, calc_dtype,
-                      res_dtype, w_lhs, w_rhs)
+                      res_dtype, w_lhs, w_rhs, out)
         w_lhs.add_invalidates(w_res)
         w_rhs.add_invalidates(w_res)
+        if out:
+            #out.add_invalidates(w_res) #causes a recursion loop
+            w_res.get_concrete()
         return w_res
 
 

pypy/module/micronumpy/signature.py

         return self.child.eval(frame, arr.child)
 
 class Call1(Signature):
-    _immutable_fields_ = ['unfunc', 'name', 'child', 'dtype']
+    _immutable_fields_ = ['unfunc', 'name', 'child', 'res', 'dtype']
 
-    def __init__(self, func, name, dtype, child):
+    def __init__(self, func, name, dtype, child, res=None):
         self.unfunc = func
         self.child = child
         self.name = name
         self.dtype = dtype
+        self.res  = res
 
     def hash(self):
         return compute_hash(self.name) ^ intmask(self.child.hash() << 1)
         v = self.child.eval(frame, arr.values).convert_to(arr.calc_dtype)
         return self.unfunc(arr.calc_dtype, v)
 
+
+class BroadcastUfunc(Call1):
+    def _invent_numbering(self, cache, allnumbers):
+        self.res._invent_numbering(cache, allnumbers)
+        self.child._invent_numbering(new_cache(), allnumbers)
+
+    def debug_repr(self):
+        return 'BroadcastUfunc(%s, %s)' % (self.name, self.child.debug_repr())
+
+    def _create_iter(self, iterlist, arraylist, arr, transforms):
+        from pypy.module.micronumpy.interp_numarray import Call1
+
+        assert isinstance(arr, Call1)
+        vtransforms = transforms + [BroadcastTransform(arr.values.shape)]
+        self.child._create_iter(iterlist, arraylist, arr.values, vtransforms)
+        self.res._create_iter(iterlist, arraylist, arr.res, transforms)
+
+    def eval(self, frame, arr):
+        from pypy.module.micronumpy.interp_numarray import Call1
+        assert isinstance(arr, Call1)
+        v = self.child.eval(frame, arr.values).convert_to(arr.calc_dtype)
+        return self.unfunc(arr.calc_dtype, v)
+
 class Call2(Signature):
     _immutable_fields_ = ['binfunc', 'name', 'calc_dtype', 'left', 'right']
 
 
         assert isinstance(arr, ResultArray)
         offset = frame.get_final_iter().offset
-        arr.left.setitem(offset, self.right.eval(frame, arr.right))
+        val = self.right.eval(frame, arr.right)
+        arr.left.setitem(offset, val)
+
+class BroadcastResultSignature(ResultSignature):
+    def _create_iter(self, iterlist, arraylist, arr, transforms):
+        from pypy.module.micronumpy.interp_numarray import ResultArray
+
+        assert isinstance(arr, ResultArray)
+        rtransforms = transforms + [BroadcastTransform(arr.left.shape)]
+        self.left._create_iter(iterlist, arraylist, arr.left, transforms)
+        self.right._create_iter(iterlist, arraylist, arr.right, rtransforms)
 
 class ToStringSignature(Call1):
     def __init__(self, dtype, child):
         from pypy.module.micronumpy.interp_numarray import ToStringArray
 
         assert isinstance(arr, ToStringArray)
-        arr.res.setitem(0, self.child.eval(frame, arr.values).convert_to(
+        arr.res_str.setitem(0, self.child.eval(frame, arr.values).convert_to(
             self.dtype))
         for i in range(arr.item_size):
-            arr.s.append(arr.res_casted[i])
+            arr.s.append(arr.res_str_casted[i])
 
 class BroadcastLeft(Call2):
     def _invent_numbering(self, cache, allnumbers):
             cur = arr.left.getitem(iterator.offset)
             value = self.binfunc(self.calc_dtype, cur, v)
         arr.left.setitem(iterator.offset, value)
-    
     def debug_repr(self):
         return 'AxisReduceSig(%s, %s)' % (self.name, self.right.debug_repr())
Add a comment to this file

pypy/module/micronumpy/test/test_dtypes.py

File contents unchanged.

Add a comment to this file

pypy/module/micronumpy/test/test_module.py

File contents unchanged.

pypy/module/micronumpy/test/test_numarray.py

         assert a.sum() == 5
 
         raises(TypeError, 'a.sum(2, 3)')
+        d = array(0.)
+        b = a.sum(out=d)
+        assert b == d
+        assert isinstance(b, float)
 
     def test_reduce_nd(self):
         from numpypy import arange, array, multiply
         a = array([[1, 2], [3, 4], [5, 6], [7, 8],
                    [9, 10], [11, 12], [13, 14]])
         b = a[::2]
-        print a
-        print b
         assert (b == [[1, 2], [5, 6], [9, 10], [13, 14]]).all()
         c = b + b
         assert c[1][1] == 12

pypy/module/micronumpy/test/test_outarg.py

+import py
+from pypy.module.micronumpy.test.test_base import BaseNumpyAppTest
+
+class AppTestOutArg(BaseNumpyAppTest):
+    def test_reduce_out(self):
+        from numpypy import arange, zeros, array
+        a = arange(15).reshape(5, 3)
+        b = arange(12).reshape(4,3)
+        c = a.sum(0, out=b[1])
+        assert (c == [30, 35, 40]).all()
+        assert (c == b[1]).all()
+        raises(ValueError, 'a.prod(0, out=arange(10))')
+        a=arange(12).reshape(3,2,2)
+        raises(ValueError, 'a.sum(0, out=arange(12).reshape(3,2,2))')
+        raises(ValueError, 'a.sum(0, out=arange(3))')
+        c = array([-1, 0, 1]).sum(out=zeros([], dtype=bool))
+        #You could argue that this should product False, but
+        # that would require an itermediate result. Cpython numpy
+        # gives True.
+        assert c == True
+        a = array([[-1, 0, 1], [1, 0, -1]])
+        c = a.sum(0, out=zeros((3,), dtype=bool))
+        assert (c == [True, False, True]).all()
+        c = a.sum(1, out=zeros((2,), dtype=bool))
+        assert (c == [True, True]).all()
+
+    def test_reduce_intermediary(self):
+        from numpypy import arange, array
+        a = arange(15).reshape(5, 3)
+        b = array(range(3), dtype=bool)
+        c = a.prod(0, out=b)
+        assert(b == [False,  True,  True]).all()
+
+    def test_ufunc_out(self):
+        from _numpypy import array, negative, zeros, sin
+        from math import sin as msin
+        a = array([[1, 2], [3, 4]])
+        c = zeros((2,2,2))
+        b = negative(a + a, out=c[1])
+        #test for view, and also test that forcing out also forces b
+        assert (c[:, :, 1] == [[0, 0], [-4, -8]]).all()
+        assert (b == [[-2, -4], [-6, -8]]).all()
+        #Test broadcast, type promotion
+        b = negative(3, out=a)
+        assert (a == -3).all()
+        c = zeros((2, 2), dtype=float)
+        b = negative(3, out=c)
+        assert b.dtype.kind == c.dtype.kind
+        assert b.shape == c.shape
+        a = array([1, 2])
+        b = sin(a, out=c)
+        assert(c == [[msin(1), msin(2)]] * 2).all()
+        b = sin(a, out=c+c)
+        assert (c == b).all()
+
+        #Test shape agreement
+        a = zeros((3,4))
+        b = zeros((3,5))
+        raises(ValueError, 'negative(a, out=b)')
+        b = zeros((1,4))
+        raises(ValueError, 'negative(a, out=b)')
+
+    def test_binfunc_out(self):
+        from _numpypy import array, add
+        a = array([[1, 2], [3, 4]])
+        out = array([[1, 2], [3, 4]])
+        c = add(a, a, out=out)
+        assert (c == out).all()
+        assert c.shape == a.shape
+        assert c.dtype is a.dtype
+        c[0,0] = 100
+        assert out[0, 0] == 100
+        out[:] = 100
+        raises(ValueError, 'c = add(a, a, out=out[1])')
+        c = add(a[0], a[1], out=out[1])
+        assert (c == out[1]).all()
+        assert (c == [4, 6]).all()
+        assert (out[0] == 100).all()
+        c = add(a[0], a[1], out=out)
+        assert (c == out[1]).all()
+        assert (c == out[0]).all()
+        out = array(16, dtype=int)
+        b = add(10, 10, out=out)
+        assert b==out
+        assert b.dtype == out.dtype
+        
+    def test_applevel(self):
+        from _numpypy import array, sum, max, min
+        a = array([[1, 2], [3, 4]])
+        out = array([[0, 0], [0, 0]])
+        c = sum(a, axis=0, out=out[0])
+        assert (c == [4, 6]).all()
+        assert (c == out[0]).all()
+        assert (c != out[1]).all()
+        c = max(a, axis=1, out=out[0])
+        assert (c == [2, 4]).all()
+        assert (c == out[0]).all()
+        assert (c != out[1]).all()
+        
+    def test_ufunc_cast(self):
+        from _numpypy import array, negative, add, sum
+        a = array(16, dtype = int)
+        c = array(0, dtype = float)
+        b = negative(a, out=c)
+        assert b == c
+        b = add(a, a, out=c)
+        assert b == c
+        d = array([16, 16], dtype=int)
+        b = sum(d, out=c)
+        assert b == c
+        try:
+            from _numpypy import version
+            v = version.version.split('.')
+        except:
+            v = ['1', '6', '0'] # numpypy is api compatable to what version?
+        if v[0]<'2':
+            b = negative(c, out=a)
+            assert b == a
+            b = add(c, c, out=a)
+            assert b == a
+            b = sum(array([16, 16], dtype=float), out=a)
+            assert b == a
+        else:
+            cast_error = raises(TypeError, negative, c, a)
+            assert str(cast_error.value) == \
+            "Cannot cast ufunc negative output from dtype('float64') to dtype('int64') with casting rule 'same_kind'"
Add a comment to this file

pypy/module/micronumpy/test/test_ufuncs.py

File contents unchanged.

pypy/module/micronumpy/test/test_zjit.py

         #            bogus. We need to improve the situation somehow.
         self.check_simple_loop({'getinteriorfield_raw': 2,
                                 'setinteriorfield_raw': 1,
-                                'arraylen_gc': 1,
+                                'arraylen_gc': 2,
                                 'guard_true': 1,
                                 'int_lt': 1,
                                 'jump': 1,
Add a comment to this file

pypy/module/mmap/__init__.py

File contents unchanged.

Add a comment to this file

pypy/module/operator/app_operator.py

File contents unchanged.

Add a comment to this file

pypy/module/posix/test/test_posix2.py

File contents unchanged.

Add a comment to this file

pypy/module/pypyjit/interp_jit.py

File contents unchanged.

pypy/module/pypyjit/test_pypy_c/test_containers.py

         loop, = log.loops_by_filename(self.filepath)
         ops = loop.ops_by_id('look')
         assert 'call' not in log.opnames(ops)
+
+    #XXX the following tests only work with strategies enabled
+
+    def test_should_not_create_intobject_with_sets(self):
+        def main(n):
+            i = 0
+            s = set()
+            while i < n:
+                s.add(i)
+                i += 1
+        log = self.run(main, [1000])
+        assert log.result == main(1000)
+        loop, = log.loops_by_filename(self.filepath)
+        opnames = log.opnames(loop.allops())
+        assert opnames.count('new_with_vtable') == 0
+
+    def test_should_not_create_stringobject_with_sets(self):
+        def main(n):
+            i = 0
+            s = set()
+            while i < n:
+                s.add(str(i))
+                i += 1
+        log = self.run(main, [1000])
+        assert log.result == main(1000)
+        loop, = log.loops_by_filename(self.filepath)
+        opnames = log.opnames(loop.allops())
+        assert opnames.count('new_with_vtable') == 0
+
+    def test_should_not_create_intobject_with_lists(self):
+        def main(n):
+            i = 0
+            l = []
+            while i < n:
+                l.append(i)
+                i += 1
+        log = self.run(main, [1000])
+        assert log.result == main(1000)
+        loop, = log.loops_by_filename(self.filepath)
+        opnames = log.opnames(loop.allops())
+        assert opnames.count('new_with_vtable') == 0
+
+    def test_should_not_create_stringobject_with_lists(self):
+        def main(n):
+            i = 0
+            l = []
+            while i < n:
+                l.append(str(i))
+                i += 1
+        log = self.run(main, [1000])
+        assert log.result == main(1000)
+        loop, = log.loops_by_filename(self.filepath)
+        opnames = log.opnames(loop.allops())
+        assert opnames.count('new_with_vtable') == 0
+
+    def test_optimized_create_list_from_string(self):
+        def main(n):
+            i = 0
+            l = []
+            while i < n:
+                l = list("abc" * i)
+                i += 1
+        log = self.run(main, [1000])
+        assert log.result == main(1000)
+        loop, = log.loops_by_filename(self.filepath)
+        opnames = log.opnames(loop.allops())
+        assert opnames.count('new_with_vtable') == 0
+
+    def test_optimized_create_set_from_list(self):
+        def main(n):
+            i = 0
+            while i < n:
+                s = set([1,2,3])
+                i += 1
+        log = self.run(main, [1000])
+        assert log.result == main(1000)
+        loop, = log.loops_by_filename(self.filepath)
+        opnames = log.opnames(loop.allops())
+        assert opnames.count('new_with_vtable') == 0
Add a comment to this file

pypy/module/pypyjit/test_pypy_c/test_string.py

File contents unchanged.

Add a comment to this file

pypy/module/signal/interp_signal.py

File contents unchanged.

Add a comment to this file

pypy/module/sys/test/test_sysmodule.py

File contents unchanged.

Add a comment to this file

pypy/module/test_lib_pypy/ctypes_tests/test_as_parameter.py

File contents unchanged.

Add a comment to this file

pypy/module/test_lib_pypy/ctypes_tests/test_functions.py

File contents unchanged.

Add a comment to this file

pypy/objspace/flow/model.py

File contents unchanged.

Add a comment to this file

pypy/objspace/flow/objspace.py

File contents unchanged.

pypy/objspace/flow/test/test_objspace.py

 from __future__ import with_statement
 import new
-import py
+import py, sys
 from pypy.objspace.flow.model import Constant, Block, Link, Variable
 from pypy.objspace.flow.model import mkentrymap, c_last_exception
 from pypy.interpreter.argument import Arguments
         """ Tests code generated by pypy-c compiled with BUILD_LIST_FROM_ARG
         bytecode
         """
+        if sys.version_info < (2, 7):
+            py.test.skip("2.7 only test")
         self.patch_opcodes('BUILD_LIST_FROM_ARG')
         try:
             def f():

pypy/objspace/std/bytearrayobject.py

     length = len(data)
     start, stop, step, slicelength = w_slice.indices4(space, length)
     assert slicelength >= 0
-    newdata = [data[start + i*step] for i in range(slicelength)]
+    if step == 1 and 0 <= start <= stop:
+        newdata = data[start:stop]
+    else:
+        newdata = _getitem_slice_multistep(data, start, step, slicelength)
     return W_BytearrayObject(newdata)
 
+def _getitem_slice_multistep(data, start, step, slicelength):
+    return [data[start + i*step] for i in range(slicelength)]
+
 def contains__Bytearray_Int(space, w_bytearray, w_char):
     char = space.int_w(w_char)
     if not 0 <= char < 256:

pypy/objspace/std/celldict.py

     def iter(self, w_dict):
         return ModuleDictIteratorImplementation(self.space, self, w_dict)
 
-    def keys(self, w_dict):
+    def w_keys(self, w_dict):
         space = self.space
-        iterator = self.unerase(w_dict.dstorage).iteritems
-        return [space.wrap(key) for key, cell in iterator()]
+        l = self.unerase(w_dict.dstorage).keys()
+        return space.newlist_str(l)
 
     def values(self, w_dict):
         iterator = self.unerase(w_dict.dstorage).itervalues
Add a comment to this file

pypy/objspace/std/complexobject.py

File contents unchanged.

pypy/objspace/std/dictmultiobject.py

 def _add_indirections():
     dict_methods = "setitem setitem_str getitem \
                     getitem_str delitem length \
-                    clear keys values \
+                    clear w_keys values \
                     items iter setdefault \
-                    popitem".split()
+                    popitem listview_str listview_int".split()
 
     def make_method(method):
         def f(self, *args):
     def get_empty_storage(self):
         raise NotImplementedError
 
-    def keys(self, w_dict):
+    def w_keys(self, w_dict):
         iterator = self.iter(w_dict)
         result = []
         while 1:
             if w_key is not None:
                 result.append(w_key)
             else:
-                return result
+                return self.space.newlist(result)
 
     def values(self, w_dict):
         iterator = self.iter(w_dict)
         w_dict.strategy = strategy
         w_dict.dstorage = storage
 
+    def listview_str(self, w_dict):
+        return None
+
+    def listview_int(self, w_dict):
+        return None
 
 class EmptyDictStrategy(DictStrategy):
 
             self.switch_to_object_strategy(w_dict)
             return w_dict.getitem(w_key)
 
-    def keys(self, w_dict):
-        return [self.wrap(key) for key in self.unerase(w_dict.dstorage).iterkeys()]
+    def w_keys(self, w_dict):
+        l = [self.wrap(key) for key in self.unerase(w_dict.dstorage).iterkeys()]
+        return self.space.newlist(l)
 
     def values(self, w_dict):
         return self.unerase(w_dict.dstorage).values()
     def iter(self, w_dict):
         return ObjectIteratorImplementation(self.space, self, w_dict)
 
-    def keys(self, w_dict):
-        return self.unerase(w_dict.dstorage).keys()
+    def w_keys(self, w_dict):
+        return self.space.newlist(self.unerase(w_dict.dstorage).keys())
 
 
 class StringDictStrategy(AbstractTypedStrategy, DictStrategy):
         assert key is not None
         return self.unerase(w_dict.dstorage).get(key, None)
 
+    def listview_str(self, w_dict):
+        return self.unerase(w_dict.dstorage).keys()
+
     def iter(self, w_dict):
         return StrIteratorImplementation(self.space, self, w_dict)
 
+    def w_keys(self, w_dict):
+        return self.space.newlist_str(self.listview_str(w_dict))
+
 
 class _WrappedIteratorMixin(object):
     _mixin_ = True
     def iter(self, w_dict):
         return IntIteratorImplementation(self.space, self, w_dict)
 
+    def listview_int(self, w_dict):
+        return self.unerase(w_dict.dstorage).keys()
+
+    def w_keys(self, w_dict):
+        # XXX there is no space.newlist_int yet
+        space = self.space
+        return space.call_function(space.w_list, w_dict)
+
 class IntIteratorImplementation(_WrappedIteratorMixin, IteratorImplementation):
     pass
 
     return space.newlist(w_self.items())
 
 def dict_keys__DictMulti(space, w_self):
-    return space.newlist(w_self.keys())
+    return w_self.w_keys()
 
 def dict_values__DictMulti(space, w_self):
     return space.newlist(w_self.values())

pypy/objspace/std/dictproxyobject.py

 
     def keys(self, w_dict):
         space = self.space
-        return [space.wrap(key) for key in self.unerase(w_dict.dstorage).dict_w.iterkeys()]
+        return space.newlist_str(self.unerase(w_dict.dstorage).dict_w.keys())
 
     def values(self, w_dict):
         return [unwrap_cell(self.space, w_value) for w_value in self.unerase(w_dict.dstorage).dict_w.itervalues()]

pypy/objspace/std/dicttype.py

         w_fill = space.w_None
     if space.is_w(w_type, space.w_dict):
         w_dict = W_DictMultiObject.allocate_and_init_instance(space, w_type)
-        for w_key in space.listview(w_keys):
-            w_dict.setitem(w_key, w_fill)
+
+        strlist = space.listview_str(w_keys)
+        if strlist is not None:
+            for key in strlist:
+                w_dict.setitem_str(key, w_fill)
+        else:
+            for w_key in space.listview(w_keys):
+                w_dict.setitem(w_key, w_fill)
     else:
         w_dict = space.call_function(w_type)
         for w_key in space.listview(w_keys):
Add a comment to this file

pypy/objspace/std/floatobject.py

File contents unchanged.

pypy/objspace/std/frozensettype.py

 def descr__frozenset__new__(space, w_frozensettype,
                             w_iterable=gateway.NoneNotWrapped):
     from pypy.objspace.std.setobject import W_FrozensetObject
-    from pypy.objspace.std.setobject import make_setdata_from_w_iterable
     if (space.is_w(w_frozensettype, space.w_frozenset) and
         w_iterable is not None and type(w_iterable) is W_FrozensetObject):
         return w_iterable
     w_obj = space.allocate_instance(W_FrozensetObject, w_frozensettype)
-    data = make_setdata_from_w_iterable(space, w_iterable)
-    W_FrozensetObject.__init__(w_obj, space, data)
+    W_FrozensetObject.__init__(w_obj, space, w_iterable)
     return w_obj
 
 frozenset_typedef = StdTypeDef("frozenset",
Add a comment to this file

pypy/objspace/std/intobject.py

File contents unchanged.

pypy/objspace/std/iterobject.py

 class W_SeqIterObject(W_AbstractSeqIterObject):
     """Sequence iterator implementation for general sequences."""
 
-class W_FastListIterObject(W_AbstractSeqIterObject):
-    """Sequence iterator specialized for lists, accessing directly their
-    RPython-level list of wrapped objects.
+class W_FastListIterObject(W_AbstractSeqIterObject): # XXX still needed
+    """Sequence iterator specialized for lists.
     """
 
 class W_FastTupleIterObject(W_AbstractSeqIterObject):

pypy/objspace/std/listobject.py

         new erased object as storage"""
         self.strategy.init_from_list_w(self, list_w)
 
+    def clear(self, space):
+        """Initializes (or overrides) the listobject as empty."""
+        self.space = space
+        if space.config.objspace.std.withliststrategies:
+            strategy = space.fromcache(EmptyListStrategy)
+        else:
+            strategy = space.fromcache(ObjectListStrategy)
+        self.strategy = strategy
+        strategy.clear(self)
+
     def clone(self):
         """Returns a clone by creating a new listobject
         with the same strategy and a copy of the storage"""
         """ Return the items in the list as unwrapped strings. If the list does
         not use the list strategy, return None. """
         return self.strategy.getitems_str(self)
+
+    def getitems_int(self):
+        """ Return the items in the list as unwrapped ints. If the list does
+        not use the list strategy, return None. """
+        return self.strategy.getitems_int(self)
     # ___________________________________________________