Clash Royale CLAN TAG#URR8PPP

When subclassing a numpy ndarray, how can I modify __getitem__ properly?

I am attempting to subclass numpy's ndarray. In my subclass, called MyClass, I've added a field called time as a parallel array to the main data.

MyClass

time

My goal is the following: suppose I make an instance of MyClass, let's call it mc.
I slice mc, for instance mc[2:6], and I want the resulting object to contain not only the properly sliced np array, but also the correspondingly sliced time array.

mc

mc

mc[2:6]

time

Here is my attempt:

class MyClass(np.ndarray):
def __new__(cls, data, time=None):
obj = np.asarray(data).view(cls)
obj.time = time
return obj
def __array_finalize__(self, obj):
setattr(self, 'time', obj.time)
def __getitem__(self, item):
#print item #for testing
ret = super(MyClass, self).__getitem__(item)
ret.time = self.time.__getitem__(item)
return ret


This does not work. After many hours of messing around, I realized it is because when I call mc[2:6], __getitem__ is actually called multiple times. First when it is called, the item variable, as expected, is slice(2,6,None). But then, the line containing super(MyClass, self)... calls this same function again, presumably to go retrieve the individual elements of the slice.

mc[2:6]

__getitem__

item

slice(2,6,None)

super(MyClass, self)...

The issue is that it supplies __getitem__ with a strange set of parameters, always negative numbers. In the example of mc[2:6], it calls the method 4 more times, with item values of -4, -3, -2, and -1.

__getitem__

mc[2:6]

item

As you can see, this makes it impossible for me to properly adjust the ret.time variable, since it attempts to modify it multiple times, often with nonsensical indices.

ret.time

I have tried working around this in many ways, including copying the object and editing that copy instead, taking various views of the object, and many other hacks, but none seem to overcome this issue that __getitem__ is repeatedly called with negative indices that do not line up with the requested slice.

__getitem__

Any help or explanations as to what is going on would be greatly appreciated.

self

__new__

__getitem___

3 Answers
3

I had a similar problem that I solved using the numpy matrix class as an example. __getitem__ can be called multiple times as you have noticed before the array gets created in __array_finalize__. So the solution is to store the potential new index in __getitem__ but set it in __array_finalize__.

__getitem__

__array_finalize__

__getitem__

__array_finalize__

class MyClass(np.ndarray):
def __new__(cls, data, time=None):
obj = np.asarray(data).view(cls)
obj.time = time
return obj
def __array_finalize__(self, obj):
setattr(self, 'time', obj.time)
try:
self.time = self.time[obj._new_time_index]
except:
pass

def __getitem__(self, item):
try:
if isinstance(item, (slice, int)):
self._new_time_index = item
else:
self._new_time_index = item[0]
except:
pass
return super().__getitem__(item)


As you want to update time on slices, try

time

if isinstance(item, slice):
ret.time = self.time.__getitem__(item)


in your __getitem__ method.

__getitem__

Then your time-adjusting code is only called once per slicing and never executed when getting a single item from your array.

time

The way I solved my issue (which was trying to do something very similar) was the following:

class MyClass(np.ndarray):
...

def __getitem__(self, item):
#print item #for testing
ret = super(MyClass, self).__getitem__(item)
if not isinstance(self, MyClass):
return ret

ret.time = self.time.__getitem__(item)
return ret


This way, if __getitem__ gets called multiple times, you will only adjust the time method on the first call where the calling instance is MyClass.

__getitem__

time

MyClass

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.