Memmap sliced arrays don't evaluate element wide expressions correctly

Create issue
Issue #58 new
Al Macmillan created an issue

Hi I have a large memmap which is a record array and I slice a certain number of rows at a time. But when I come to use element wide expressions or functions on subarrays (columns) the behaviour is not correct. e.g when I then try to filter a column 'f9' (which looks like: [4,4,4,3]) by an expression it evaluates to a single value 'True' not [False, False, False, True]. np.where is the same - which should return (array([3]),) but instead returns (array([0]),) However if I copy the same column to a fresh numpy array it works as expected.

See below:

In [41]: trip = T.get_schedule(schid) #This is 4 rows of a large recarray memmap
In [42]: print trip.shape
Out [42]: (4,)
In [43]: print trip
Out [43]: [ (422582, '1-MET-_-y05-3390311:VJ_1-MET-_-y05-3390311-1-UR:AMD', 5, 'LUL', 147463, 150286, 0, inf, 126.0, 4)
 (422582, '1-MET-_-y05-3390311:VJ_1-MET-_-y05-3390311-1-UR:AMD', 5, 'LUL', 147766, 150246, 0, 274.0, 274.0, 4)
 (422582, '1-MET-_-y05-3390311:VJ_1-MET-_-y05-3390311-1-UR:AMD', 5, 'LUL', 147157, 150363, 0, 422.0, 422.0, 4)
 (422582, '1-MET-_-y05-3390311:VJ_1-MET-_-y05-3390311-1-UR:AMD', 5, 'LUL', 147431, 146844, 146737, 570.0, inf, 2)]
In [44]: print trip.dtype
Out [44]: [('f0', '<i4'), ('f1', '|S54'), ('f2', '|i1'), ('f3', '|S4'), ('f4', '<i4'), ('f5', '<i4'), ('f6', '<i4'), ('f7', '<f2'), ('f8', '<f2'), ('f9', '|i1')]
In [45]: print type(trip)
Out [45]: <class 'numpy.core.memmap.memmap'>
## So trip['f9'] looks like [4,4,4,2]
In [46]: mask = (trip['f9'] != 4); print mask
Out [46]: True
In [47]: print np.where(trip['f9'] ! = 4)
Out [47]: (array([0]),)

# Now copy the sliced memmap to a numpy array
In [48]: copied_column = np.copy(trip['f9'])
In [49]: print type(copied_column)
Out  [49]: <type 'numpy.ndarray'>
In [50]: mask = (copied_column != 4); print mask
Out [50]: [False False False  True]
In [51]: print np.where(copied_column != 4)
Out [51]: (array([3]),)

Comments (0)

  1. Log in to comment