[FIXED] Why does `zip` eat one more element from the first iterator?

Issue

I'm studying Item 76 in Effective Python (2nd Ed.), and I run into a case that I don't understand. Specifically, I don't understand why zip consumes one additional element of its first argument. Consider the following code:

l1 = []
l2 = [1]
l1_it = iter(l1)
l2_it = iter(l2)
test_it = zip(l2_it, l1_it)
_ = list(test_it):

try:
    next(l2_it)
except StopIteration:
    print('This should not happen')

This actually prints This should not happen, and I find this very surprising. I would expect zip to leave its first argument in a state where there is still one element to be retrieved. And the fact is that if I use zip(l1_it, l2_it) (that is, the shortest list is first), then indeed I can call next(l2_it) without triggering an exception.

Is this expected?

Solution

zip takes the length of the shortest iterable and limits to that. Since l1 has no items, the item in l2 will not be processed.

However... with an iterable Python doesn't know how many items are available so its only recourse is to try and fetch an item. If there is no item it gets an exception. If there is an item, it has already consumed it from the iterator by now.

Perhaps you were expecting the behaviour of zip_longest? https://docs.python.org/3/library/itertools.html?highlight=itertools#itertools.zip_longest

To illustrate the iter() behaviour:

>>> x = [1]
>>> x_iter = iter(x)

# We cannot get the length and unless we are prepared to fetch it,
# we cannot check if there are items available.
>>> len(x_iter)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: object of type 'list_iterator' has no len()

# First item can be fetched
>>> next(x_iter)
1

# There is no second item so Python raises a StopIteration
>>> next(x_iter)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration

Answered By - Wolph