[FIXED] operator.index with custom class instance

Issue

I have a simple class below:

class MyClass(int):
    def __index__(self):
        return 1

According to operator.index documentation:

operator.index(a)

Return a converted to an integer. Equivalent to a.__index__()

But when I use operator.index with MyClass instance, I got 100 instead of 1 (I am getting 1 if I use a.__index__()). Why is that?

>>> a = MyClass(100)
>>>
>>> import operator
>>> print(operator.index(a))
100
>>> print(a.__index__())
1

Solution

This is because your type is an int subclass. __index__ will not be used because the instance is already an integer. That much is by design, and unlikely to be considered a bug in CPython. PyPy behaves the same.

In _operator.c:

static PyObject *
_operator_index(PyObject *module, PyObject *a)
/*[clinic end generated code: output=d972b0764ac305fc input=6f54d50ea64a579c]*/
{
    return PyNumber_Index(a);
}

Note that operator.py Python code is not used generally, this code is only a fallback in the case that compiled _operator module is not available. That explains why the result a.__index__() differs.

In abstract.c, cropped after the relevant PyLong_Check part:

/* Return an exact Python int from the object item.
   Raise TypeError if the result is not an int
   or if the object cannot be interpreted as an index.
*/
PyObject *
PyNumber_Index(PyObject *item)
{
    PyObject *result = _PyNumber_Index(item);
    if (result != NULL && !PyLong_CheckExact(result)) {
        Py_SETREF(result, _PyLong_Copy((PyLongObject *)result));
    }
    return result;
}

...

/* Return a Python int from the object item.
   Can return an instance of int subclass.
   Raise TypeError if the result is not an int
   or if the object cannot be interpreted as an index.
*/
PyObject *
_PyNumber_Index(PyObject *item)
{
    PyObject *result = NULL;
    if (item == NULL) {
        return null_error();
    }

    if (PyLong_Check(item)) {
        Py_INCREF(item);
        return item;     /* <---- short-circuited here */
    }
    ...
}

The documentation for operator.index is inaccurate, so this may be considered a minor documentation issue:

>>> import operator
>>> operator.index.__doc__
'Same as a.__index__()'

So, why isn't __index__ considered for integers? The probable answer is found in PEP 357, under the discussion section titled Speed:

Implementation should not slow down Python because integers and long integers used as indexes will complete in the same number of instructions. The only change will be that what used to generate an error will now be acceptable.

We do not want to slow down the most common case for slicing with integers, having to check for an nb_index slot every time.

Answered By - wim