The issue is that numpy sets tp_hash after calling PyType_Ready (see numpy issue for details, it's being fixed on their end). Is there anything we should do to better support that kind of broken code?
We store offsets in the type object specifically to enable this use case - the offsets are used at runtime to access a pointer-to-function. Maybe the problem is that we never see a tp_hash at PyType_Ready so the offset is not set up?
In numpy, tp_hash actually got set to PyObject_HashNotimplemented via inheritance in the beginning of PyType_Ready, so the case we don't support is when tp_hash changes from one non-NULL value to another after PyType_Ready. This is rather broken on CPython, so I now think we don't need to support it.