gh-142883: avoid crash when __index__ changes array type during multiplication. #143776
Conversation
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
Co-authored-by: AN Long <[email protected]>
Co-authored-by: AN Long <[email protected]>
|
Thanks, fixed. |
| @@ -9716,13 +9714,19 @@ wrap_indexargfunc(PyObject *self, PyObject *args, void *wrapped) | |||
| ssizeargfunc func = (ssizeargfunc)wrapped; | |||
| PyObject* o; | |||
| Py_ssize_t i; | |||
| PyTypeObject *type_before = Py_TYPE(self); | |||
There was a problem hiding this comment.
I'm not very sure about this, but I'm wondering should we increase the refcount for type_before, because we will call user's code next, thus it may be GCed (like user delete the type in __index__)?
There was a problem hiding this comment.
Yes, that makes sense. Since we call user code next, the type could indeed be GCed if its last reference is dropped. I’ve updated the code to INCREF type_before before calling user code and DECREF it on all exit paths. I also ran the regression test locally and it passes.
picnixz
left a comment
picnixz
left a comment
There was a problem hiding this comment.
Why do we need to make those changes in typeobject? is it only relevant for objects implementing __mul__? or is it relevant only to array.__mul__?
| self.assertRaises(TypeError, op, v, z) | ||
|
|
||
|
|
||
| class IndexMutationDuringNumericOpTest(unittest.TestCase): |
There was a problem hiding this comment.
Why is it in the numeric tower??? it has nothing to do with numeric tower.
There was a problem hiding this comment.
I initially added the test to test_numeric_tower.py since the problem shows up during a numeric operation and involves __index__ being called. That’s why it felt like a reasonable place to put it at first, but I’m happy to move it. Is there a test module you would prefer for this regression test?
|
A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated. Once you have made the requested changes, please leave a comment on this pull request containing the phrase |
|
The change in typeobject.c is intentional. This didn’t seem specific to |
Does it happens e.g. for lists, which also implement sq_repeat slot? |
|
I am not aware of this if it happens with lists. The crash seems to be array-specific, and lists don’t appear to reach this broken state in practice. |
And thus this means that the issue is with |
|
Or more generally, looks like an issue with getting the module's state. However I'm pretty sure it could affect other subclassable heap types. |
Yes, get_array_state() get NULL, from unchecked return value of PyType_GetModuleByDef(). |
|
I'm closing this PR because the approach is incorrect. We should be careful when accessing state of subclassable heap type objects. I think it would be better if we addressed this for all heap types simultaneously. I'm travelling for the next 2 weeks so I won't be able to do it but a pre-emptive investigation on which types suffer from that would be good. |
|
On a simlar note, I've tried to design some (internal) functions for safely getting the module's state from an object or its type, because currently, we need to do multiple calls and it's not always clear whether the caller knows which function to use (with/wo underscores? with/wo checks? etc) |
|
Thanks for the explanation, that makes sense. I understand why a more general solution would be preferable here. Appreciate the feedback. |
4 participants
__index__can change the array’s class during multiplication. After that,the old function is still called, which can cause a crash. This adds a
check to avoid that and raises TypeError instead.
A regression test is also added.
array_repeatvia re-entrant__index__during array multiplication #142883