np.in1d doesn't work on memmaps

Issue #22 resolved
Al Macmillan created an issue

Hi

I have code that intersects an array b with a memmap a via np.in1d. On cpython numpy this behaves as expected but on pypy numpy it only works if the memmap is converted/is viewed as an ndarray.

In [49]: a[i:j][:,4]
Out[49]: 
memmap([   242,    507,    255,    505,    315,    316,    308,    506,
          309,    255,    211,    505,    315,    316,    308,    506,
          309,    255,    255,    711,    194,    232,    711,    711,
          709,    710,    709,    710,    882,    897,    711,    245,
          711,    711,    168,    245], dtype=uint32)

In [50]: b
Out[50]: array([245, 255, 293], dtype=uint32)

In [51]: in1d(a[i:j][:,4], b)
Out[51]: 
array([False, False, False, False, False, False, False, False, False,
       False, False, False, False, False, False, False, False, False,
       False, False, False, False, False, False, False, False, False,
       False, False, False, False, False, False, False, False, False], dtype=bool)

In [52]: c = a[i:j][:,4].view(type=np.ndarray)

In [53]: in1d(c, b)
Out[53]: 
array([False, False,  True, False, False, False, False, False, False,
        True, False, False, False, False, False, False, False,  True,
        True, False, False, False, False, False, False, False, False,
       False, False, False, False,  True, False, False, False,  True], dtype=bool)

Comments (3)

  1. mattip

    I didn't get too far, so an example script would be the first step. Here is what I tried, following the numpy documentation, which of course failed but differently::

    >>>> import numpy as np
    >>>> data = np.empty((10,10,36,5), dtype='float32')
    >>>> data[5,5][:,4] = [   242,    507,    255,    505,    315,    316,    308,    506,   
                   309,    255,    211,    505,    315,    316,    308,    506,
                   309,    255,    255,    711,    194,    232,    711,    711,
                   709,    710,    709,    710,    882,    897,    711,    245,
                   711,    711,    168,    245]
    >>>> data[5,5][:,4]
    array([ 242.,  507.,  255.,  505.,  315.,  316.,  308.,  506.,  309.,
            255.,  211.,  505.,  315.,  316.,  308.,  506.,  309.,  255.,
            255.,  711.,  194.,  232.,  711.,  711.,  709.,  710.,  709.,
            710.,  882.,  897.,  711.,  245.,  711.,  711.,  168.,  245.], dtype=float32)
    >>>> from tempfile import mkdtemp
    >>>> import os.path as path
    >>>> filename = path.join(mkdtemp(), 'newfile.dat')
    >>>> fp = np.memmap(filename, dtype='float32', mode='w+', shape=data.shape)
    >>>> fp[:] = data[:]
    Traceback (most recent call last):
       File "<stdin>", line 1, in <module>
    ValueError: assignment destination is read-only
    
  2. Log in to comment