The Gotcha

If an iterator is looping over the elements of a list and one of those elements is removed, the iterator will "skip" over other elements in the list.

In the example below, the for loop iterates over all elements of my_list and removes any elements that are greater than or equal to 5. However, my_list still contains a 6 and 8 afterwards. The iterator "skipped" over these elements when looping over the list:

my_list = list(range(10))
print(my_list)  # [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

def should_remove(x):
    return x >= 5

for element in my_list:
    if should_remove(element):
        my_list.remove(element)

print(my_list)  # [0, 1, 2, 4, 6, 8]

Why it Happens

The list iterator works by referencing indexes, incrementing an internal i variable and yielding my_list[i] when asked to fetch the next element. This leads to problems whenever an element is removed from the list. All elements to the right of the removed element shift one to the left, and their indexes all decrement by one as well. Thus, when the iterator increments i to access the next element, it "skips" over the element that took over the position of the removed element.

These "skips" are easiest to see by adding print statements to show which elements the iterator loops over:

my_list = list(range(10))
print(my_list)  # [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

def should_remove(x):
    return x >= 5

for element in my_list:
    print(element)
    if should_remove(element):
        my_list.remove(element)

# 0
# 1
# 2
# 3
# 4
# 5
# 7
# 9

From the output, we can see that the iterator never sees the 6 or 8. That's because the 6 shifted into the position of the 5 when the 5 was removed, but the iterator continued on to the next position in the list. Similarly, the 8 shifted into the position of the 7 when the 7 was removed, leading to another skip.

This means that it's technically possible for certain kinds of removals to work properly:

my_list = list(range(10))
print(my_list)  # [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

def is_odd(x):
    return x % 2 == 1

for element in my_list:
    if is_odd(element):
        my_list.remove(element)

print(my_list)  # [0, 2, 4, 6, 8]

However, this is a dangerous practice and should be avoided.

The Fix

Solution 1: Iterate over a copy

Make a copy of the list. Iterate over that list while removing elements from the original. This allows the iterator to work with a list that is never modified, preventing skips.

Lists can be copied with list(my_list) or my_list[:]:

my_list = list(range(10))
my_list)  # [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

def should_remove(x):
    return x >= 5

for element in list(my_list):
    if should_remove(element):
            my_list.remove(element)

print(my_list)  # [0, 1, 2, 3, 4]
my_list = list(range(10))
print(my_list)  # [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

def should_remove(x):
    return x >= 5

for element in my_list[:]:
    if should_remove(element):
        my_list.remove(element)

print(my_list)  # [0, 1, 2, 3, 4]

This solution is ideal when list identity must be preserved. If the code expects the same list object to be used and mutated throughout the program instead of referencing new list objects whenever elements are removed, stick to this solution.

Solution 2: Use a list comprehension

Python's list comprehensions allow an if clause to filter out elements:

my_list = list(range(10))
print(my_list)  # [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

def should_remove(x):
    return x >= 5

my_list = [x for x in my_list if not should_remove(x)]
print(my_list)  # [0, 1, 2, 3, 4]

However, list comprehensions always produce a new list object. It isn't commonly used, but list identity can be preserved if desired by modifying the assignment statement:

my_list[:] = [x for x in my_list if not should_remove(x)]

Solution 3: Use filter

Python has a builtin filter method:

my_list = list(range(10))
my_list  # [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

def should_remove(x):
    return x >= 5

def should_keep(x):
    return not should_remove(x)
 
my_list = list(filter(should_keep, my_list))
print(my_list)  # [0, 1, 2, 3, 4]

Unfortunately, filter is used to determine which elements in a list should be kept. Luckily, filterfalse provides the reverse functionality in Python 3:

from itertools import filterfalse

my_list = list(range(10))
print(my_list)  # [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

def should_remove(x):
    return x >= 5

my_list = list(filterfalse(should_remove, my_list))
print(my_list)  # [0, 1, 2, 3, 4]

Similar to list comprehensions, the filter and filterfalse functions both produce new list objects. To preserve list identity, use the same trick and assign the result to my_list[:].