Understanding Iterable Objects and Iterators in Python

Course: Python Training

Topic: Understanding Iterable Objects and Iterators

Course Link: Python Training Course

Demo Link: Register for a Free Course Demo

Website: JustAcademy

1. Introduction to Iterables and Iterators

Python provides a powerful mechanism for processing a collection of values one at a time. Two important concepts behind this mechanism are iterables and iterators. They are commonly used in loops, data processing, file handling, generators, and custom classes.

An iterable is an object whose elements can be accessed one after another. An iterator is an object that keeps track of its current position and returns the next element when requested.

For example, a list stores multiple values, while an iterator can retrieve those values sequentially without requiring you to manage an index manually.

numbers = [10, 20, 30]

for number in numbers:
    print(number)

Output:

10
20
30

The list numbers is an iterable. The for loop obtains an iterator from it and requests values until there are no more elements.

2. What Is an Iterable?

An iterable is an object that Python can iterate over. In practice, an object is iterable if it can provide an iterator, usually through the __iter__() method.

Python's built-in iterable types include lists, tuples, strings, dictionaries, sets, ranges, and many file objects.

2.1 Examples of Iterable Objects

# List
fruits = ["apple", "banana", "mango"]

# Tuple
coordinates = (10, 20, 30)

# String
message = "Python"

# Set
unique_numbers = {1, 2, 3}

# Range
sequence = range(1, 5)

# Dictionary
student = {"name": "Aman", "age": 21}

for fruit in fruits:
    print(fruit)

for character in message:
    print(character)

for number in sequence:
    print(number)

Each example provides a sequence of values that can be processed one at a time. Dictionaries normally iterate over their keys, while strings iterate over individual characters.

2.2 Common Iterable Types

3. What Is an Iterator?

An iterator is an object that returns values one at a time. It remembers its current position, so each request normally produces the next value rather than restarting from the beginning.

An iterator follows the iterator protocol, which involves two special methods:

• __iter__() returns the iterator itself.
• __next__() returns the next available value. When no values remain, it raises StopIteration.

You usually do not need to call these special methods directly. Python provides the built-in functions iter() and next() as convenient interfaces.

colors = ["red", "green", "blue"]

color_iterator = iter(colors)

print(next(color_iterator))
print(next(color_iterator))
print(next(color_iterator))

Output:

red
green
blue

The iterator advances after each call to next(). When all three values have been returned, another call raises StopIteration.

4. Difference Between Iterable and Iterator

Iterable and iterator are related concepts, but they serve different purposes.

Example: Iterable and Iterator Together

numbers = [10, 20, 30]   # Iterable
iterator = iter(numbers) # Iterator

print(next(iterator))
print(next(iterator))

print(list(numbers))     # A fresh traversal of the list
print(list(iterator))    # Remaining iterator values

Output:

10
20
[10, 20, 30]
[30]

The list remains available for a new traversal. The iterator, however, has already returned the first two values, so only the remaining value is collected.

5. Understanding the iter() Function

The built-in iter() function obtains an iterator from an iterable object. It is commonly used when you need manual control over iteration.

Syntax:

iterator = iter(iterable)

Example:

languages = ["Python", "Java", "JavaScript"]

language_iterator = iter(languages)

print(type(languages))
print(type(language_iterator))
print(next(language_iterator))

The first type represents a list, while the second represents a list iterator. The final statement retrieves the first language.

For a custom object, iter() normally calls its __iter__() method. If the object does not support iteration, Python raises a TypeError, unless it supports the legacy sequence protocol.

6. Understanding the next() Function

The next() function requests the next item from an iterator. Each successful call advances the iterator's state.

Syntax:

next(iterator)
next(iterator, default_value)

6.1 Using next() Normally

numbers = [100, 200, 300]
iterator = iter(numbers)

print(next(iterator))
print(next(iterator))
print(next(iterator))

Output:

100
200
300

6.2 Using next() with a Default Value

If an iterator is exhausted, next(iterator, default_value) returns the specified default instead of raising StopIteration.

numbers = [5, 10]
iterator = iter(numbers)

print(next(iterator, "No value"))
print(next(iterator, "No value"))
print(next(iterator, "No value"))

Output:

5
10
No value

This form is useful when you want to retrieve a value safely without writing an exception handler for exhaustion.

7. Understanding StopIteration

StopIteration is a special exception that signals that an iterator has no more values available. Python's for loop handles this condition automatically.

numbers = [1, 2]
iterator = iter(numbers)

print(next(iterator))
print(next(iterator))

# This call raises StopIteration:
# print(next(iterator))

Uncommenting the last line raises StopIteration because both values have already been consumed.

7.1 Handling StopIteration Explicitly

numbers = [10, 20]
iterator = iter(numbers)

while True:
    try:
        value = next(iterator)
        print(value)
    except StopIteration:
        print("Iteration complete")
        break

Output:

10
20
Iteration complete

This pattern demonstrates manual iteration. In everyday Python code, a for loop is generally simpler and less error-prone.

8. How a for Loop Uses Iterators

A Python for loop works with the iterator protocol behind the scenes. It obtains an iterator from the iterable, repeatedly requests the next value, executes the loop body, and stops when iteration is exhausted.

items = ["pen", "book", "bag"]

for item in items:
    print(item)

The loop behaves conceptually like this:

items = ["pen", "book", "bag"]
iterator = iter(items)

while True:
    try:
        item = next(iterator)
    except StopIteration:
        break
    print(item)

The explicit version illustrates the process. Normally, use the shorter for loop unless manual iterator control is needed.

9. Iterating Over Lists and Tuples

Lists and tuples are common iterables. Lists are mutable, whereas tuples are immutable. Both can be traversed using a for loop or an iterator.

scores = [75, 88, 92]
score_iterator = iter(scores)

print(next(score_iterator))
print(next(score_iterator))

subjects = ("Math", "Science", "English")

for subject in subjects:
    print(subject)

Output:

75
88
Math
Science
English

Use a direct loop for ordinary traversal. Use an explicit iterator when you need to consume values step by step or integrate iteration into another process.

10. Iterating Over Strings

A string is an iterable of Unicode code points. Iterating over a string typically returns one character at a time.

word = "Python"

for character in word:
    print(character)

Output:

P
y
t
h
o
n

String iteration is useful for character counting, text analysis, validation, and simple parsing. Some visible characters, such as emoji formed from multiple code points, may require more advanced Unicode handling if you need to treat them as a single displayed character.

11. Iterating Over Dictionaries

Dictionaries store key-value pairs. By default, iterating over a dictionary returns its keys. You can use keys(), values(), and items() to choose what to process.

student = {
    "name": "Riya",
    "age": 22,
    "course": "Python"
}

# Iterate over keys
for key in student:
    print(key)

# Iterate over values
for value in student.values():
    print(value)

# Iterate over key-value pairs
for key, value in student.items():
    print(key, ":", value)

Key points:

• for key in student iterates over dictionary keys.
• student.values() provides an iterable view of the values.
• student.items() provides an iterable view of key-value pairs.
• Dictionary views reflect changes to the dictionary, so avoid changing its size during active iteration.

12. Iterating Over Sets and Ranges

Sets are iterable collections of unique elements. Their iteration order should not be relied upon for a particular sequence. A range object represents an arithmetic progression and is useful for numeric loops.

unique_values = {30, 10, 20}

for value in unique_values:
    print(value)

for number in range(1, 6):
    print(number)

The range loop produces the integers 1 through 5. The set loop visits each element, but its order is not guaranteed to match the order in which elements were written.

13. Creating a Custom Iterator Class

You can define your own iterator when you need a custom sequence of values. A conventional iterator class implements __iter__() and __next__().

The following example produces numbers from 1 up to a specified limit.

class CountUp:
    def __init__(self, limit):
        self.limit = limit
        self.current = 1

    def __iter__(self):
        return self

    def __next__(self):
        if self.current > self.limit:
            raise StopIteration

        value = self.current
        self.current += 1
        return value


counter = CountUp(5)

for number in counter:
    print(number)

Output:

1
2
3
4
5

How the Custom Iterator Works

1. The constructor stores the upper limit and initializes the current value.
2. The __iter__() method returns the iterator itself.
3. The __next__() method checks whether the limit has been exceeded.
4. If a value is available, it returns that value and advances the counter.
5. When the limit is exceeded, it raises StopIteration.

Because the iterator stores its state in current, it continues from the current position if iteration is interrupted and later resumed.

14. Iterable Classes Versus Iterator Classes

A custom iterable and a custom iterator are not necessarily the same object. Separating them is useful when you want to create a fresh iterator each time a collection is traversed.

class NumberCollection:
    def __init__(self, limit):
        self.limit = limit

    def __iter__(self):
        return NumberIterator(self.limit)


class NumberIterator:
    def __init__(self, limit):
        self.limit = limit
        self.current = 1

    def __iter__(self):
        return self

    def __next__(self):
        if self.current > self.limit:
            raise StopIteration

        value = self.current
        self.current += 1
        return value


numbers = NumberCollection(3)

print(list(numbers))
print(list(numbers))

Output:

[1, 2, 3]
[1, 2, 3]

Each call to iter(numbers) creates a new NumberIterator. Therefore, the collection can be traversed repeatedly from the beginning.

15. Introduction to Generators

A generator is a convenient way to create an iterator. Generator functions use the yield keyword to return values one at a time while preserving their local state between requests.

def generate_numbers():
    yield 10
    yield 20
    yield 30

numbers = generate_numbers()

print(next(numbers))
print(next(numbers))
print(next(numbers))

Output:

10
20
30

Calling generate_numbers() creates a generator object. The function body runs as values are requested, pausing at each yield and resuming when the next value is requested.

15.1 Generator Function with a Loop

def count_up_to(limit):
    number = 1

    while number <= limit:
        yield number
        number += 1

for value in count_up_to(4):
    print(value)

Output:

1
2
3
4

Generators are especially useful for large datasets because values can be produced on demand instead of building a complete list in memory.

16. Generator Expressions

A generator expression resembles a list comprehension but uses parentheses instead of square brackets. It creates a generator that calculates values as they are requested.

# List comprehension: creates a list
squares_list = [number ** 2 for number in range(1, 6)]

# Generator expression: produces values on demand
squares_generator = (number ** 2 for number in range(1, 6))

print(squares_list)
print(next(squares_generator))
print(next(squares_generator))

Output:

[1, 4, 9, 16, 25]
1
4

17. Iterating Through Files

File objects are iterable. Iterating over a text file reads its lines one at a time, which can be more memory-efficient than reading the entire file into a single string or list.

with open("sample.txt", "r", encoding="utf-8") as file:
    for line in file:
        print(line.rstrip())

The with statement ensures the file is closed when the block finishes, including when an error occurs. The rstrip() call removes trailing whitespace, including the newline character.

17.1 Processing a File Line by Line

total_lines = 0

with open("sample.txt", "r", encoding="utf-8") as file:
    for line in file:
        if line.strip():
            total_lines += 1

print("Non-empty lines:", total_lines)

This example counts non-empty lines without loading the whole file into memory. It is a useful pattern for logs, reports, and large text files.

18. Useful Built-in Functions for Iteration

Python includes several functions that work naturally with iterables and iterators.

19. Using enumerate() with Iterables

The enumerate() function produces an iterator of index-value pairs. It is useful when processing items and their positions together.

languages = ["Python", "Java", "SQL"]

for index, language in enumerate(languages, start=1):
    print(index, language)

Output:

1 Python
2 Java
3 SQL

The start=1 argument makes the displayed numbering begin at 1 rather than 0. The original list is not modified.

20. Using zip() with Multiple Iterables

The zip() function combines corresponding values from two or more iterables. It produces tuples and normally stops when the shortest input iterable is exhausted.

names = ["Asha", "Rahul", "Neha"]
scores = [85, 90, 88]

for name, score in zip(names, scores):
    print(name, score)

Output:

Asha 85
Rahul 90
Neha 88

Use zip() when parallel collections contain related data. If inputs have different lengths and every element must be processed, consider itertools.zip_longest() from Python's standard library.

21. Using map() and filter() with Iterators

map() and filter() return iterators in Python 3. This lets you transform or select values without immediately creating a list.

21.1 Using map()

numbers = [1, 2, 3, 4]

squares = map(lambda number: number ** 2, numbers)

print(list(squares))

Output:

[1, 4, 9, 16]

21.2 Using filter()

numbers = [1, 2, 3, 4, 5, 6]

even_numbers = filter(lambda number: number % 2 == 0, numbers)

print(list(even_numbers))

Output:

[2, 4, 6]

Calling list() consumes the iterator and collects its remaining results into a list. If the output is only needed for one pass, it can often be processed directly in a loop.

22. Iterator Exhaustion and Reusability

An important property of many iterators is that they are stateful. After values have been consumed, they do not automatically restart from the beginning.

numbers = [1, 2, 3]
iterator = iter(numbers)

print(list(iterator))
print(list(iterator))

Output:

[1, 2, 3]
[]

The first call consumes all values. The second call returns an empty list because the iterator is exhausted.

To traverse the original list again, create a new iterator:

numbers = [1, 2, 3]

print(list(iter(numbers)))
print(list(iter(numbers)))

Output:

[1, 2, 3]
[1, 2, 3]

This distinction is especially important when passing generators or iterators to functions that consume their input.

23. Common Errors When Working with Iterators

23.1 Example: Avoiding a Consumed Iterator

values = [2, 4, 6]
iterator = iter(values)

first_pass = list(iterator)
second_pass = list(iterator)

print(first_pass)
print(second_pass)

Output:

[2, 4, 6]
[]

If both passes must process the same values, iterate over values directly or create a new iterator for the second pass.

24. Best Practices for Working with Iterables and Iterators

• Use for loops for straightforward iteration because they handle exhaustion automatically.
• Use iter() and next() when you need precise control over when values are consumed.
• Use generators to process large sequences lazily when building a full list is unnecessary.
• Remember that iterators are stateful and are commonly single-pass.
• Do not assume a set has a stable order for application logic.
• Use enumerate() instead of manually maintaining a counter when both an index and an item are needed.
• Use zip() to process corresponding values from multiple iterables.
• Use a context manager, such as with open(...), when working with files.
• Avoid modifying the size of a dictionary or set during direct iteration.
• Keep custom iterator classes simple, and raise StopIteration when the sequence is exhausted.
• Use a separate iterable object and iterator object when repeated traversals should start from the beginning.

25. Mini Project: Student Score Processor

Project objective: Create an iterator-based program that processes student scores one at a time, calculates a total, and counts passing students.

class ScoreIterator:
    def __init__(self, scores):
        self.scores = scores
        self.index = 0

    def __iter__(self):
        return self

    def __next__(self):
        if self.index >= len(self.scores):
            raise StopIteration

        score = self.scores[self.index]
        self.index += 1
        return score


scores = [78, 45, 92, 66, 35]
score_iterator = ScoreIterator(scores)

total = 0
count = 0
passed = 0

for score in score_iterator:
    total += score
    count += 1

    if score >= 40:
        passed += 1

average = total / count if count else 0

print("Total:", total)
print("Average:", round(average, 2))
print("Passing students:", passed)
print("Total students:", count)

Output:

Total: 316
Average: 63.2
Passing students: 4
Total students: 5

Project Explanation

1. The ScoreIterator class stores the scores and the current index.
2. The __iter__() method makes the object usable as an iterator.
3. The __next__() method returns one score at a time.
4. When all scores have been processed, the iterator raises StopIteration.
5. The loop calculates the total, average, number of passing students, and total number of scores.

This project demonstrates how a custom iterator can process values sequentially. In a production application, a generator or a normal loop may be simpler, but implementing an iterator is useful for understanding Python's iteration protocol.

26. Practice Exercises

1. Create a list of five numbers and use iter() and next() to print the first three values.
2. Write a program that iterates through a string and counts its vowels.
3. Create a dictionary containing employee names and salaries. Print each key-value pair using items().
4. Use enumerate() to print a numbered list of your favorite programming languages.
5. Use zip() to combine a list of student names with a list of scores.
6. Create a generator that yields even numbers from 2 through 20.
7. Write a custom iterator that returns numbers from a starting value up to a specified limit.
8. Read a text file line by line and count the non-empty lines.
9. Demonstrate what happens when an iterator is consumed twice.
10. Use filter() to select all numbers greater than 10 from a list.

27. Interview Questions and Answers

Q1. What is an iterable in Python?

An iterable is an object that can provide an iterator, usually through __iter__(). Lists, tuples, strings, dictionaries, and ranges are common examples.

Q2. What is an iterator?

An iterator is an object that returns values one at a time and maintains its traversal state. It implements __iter__() and __next__().

Q3. What is the difference between iter() and next()?

iter() obtains an iterator from an iterable. next() requests the next value from an iterator.

Q4. What is StopIteration?

StopIteration is the exception used to indicate that an iterator has no more values. Python's for loop handles it automatically.

Q5. Can a list be iterated over more than once?

Yes. A list can normally be traversed repeatedly because a fresh iterator can be created each time. A particular list iterator, however, advances as its values are consumed.

Q6. Why does an iterator return an empty list after it has already been consumed?

The iterator has reached its end. Calling list() again cannot recover values that have already been consumed. Create a fresh iterator from a reusable source when needed.

Q7. What is a generator?

A generator is a kind of iterator, commonly created using a function containing yield or a generator expression. It produces values on demand.

Q8. How does a for loop work internally?

A for loop obtains an iterator, requests successive values, executes the loop body, and ends when the iterator is exhausted.

Q9. Why are generators useful for large datasets?

Generators can produce one value at a time, often avoiding the memory cost of creating a complete result list.

Q10. What does a dictionary return during direct iteration?

Direct iteration over a dictionary yields its keys. Use values() to iterate over values or items() to iterate over key-value pairs.

Q11. What is the difference between an iterable and a generator?

An iterable is any object that can provide an iterator. A generator is a particular kind of iterator that can produce values lazily and preserve its execution state.

Q12. What does enumerate() do?

enumerate() produces pairs containing an index and the corresponding item, making it easy to process values with their positions.

28. Quick Revision Checklist

• An iterable provides a way to obtain an iterator.
• An iterator returns values one at a time.
• iter() obtains an iterator.
• next() requests the next value.
• StopIteration signals that no values remain.
• A for loop handles iterator exhaustion automatically.
• Lists, tuples, strings, sets, dictionaries, and ranges are common iterables.
• Dictionaries yield keys by default.
• Generators produce values on demand.
• Iterators are stateful and are generally exhausted after one complete traversal.
• enumerate(), zip(), map(), and filter() support common iteration tasks.
• Custom iterators implement __iter__() and __next__().

29. Conclusion

Understanding iterable objects and iterators is fundamental to writing effective Python programs. Iterables provide the source of values, while iterators manage sequential access to those values. Python uses this protocol throughout loops, generators, file processing, and many built-in functions.

By practicing iter(), next(), custom iterator classes, generator functions, and tools such as enumerate() and zip(), you can write code that is clearer, more flexible, and more memory-efficient.

Continue learning through the Python Training Course, explore the course demo registration page, or visit the JustAcademy website for more learning resources.