Course: Python Training
Topic: Using Context Managers with the with Statement
Course Link: Python Training Course at JustAcademy
Demo Link: Register for a Free Course Demo
JustAcademy Homepage: Visit JustAcademy
Context managers are a Python feature used to manage resources safely and efficiently. They help a program prepare a resource before an operation begins and clean it up when the operation finishes. Resources may include files, database connections, locks, temporary files, and network connections.
The with statement is the most common way to use a context manager. It defines a block of code in which a resource is available. When execution leaves the block, the context manager performs its cleanup operation, including when an exception occurs inside the block.
For example, when working with files, a context manager automatically closes the file after the block finishes. This reduces the risk of leaving files open accidentally.
After studying this topic, learners should be able to:
with statement to manage resources.__enter__() and __exit__() methods.A context manager is an object that defines what should happen when a managed block of code starts and when it ends. It implements the context management protocol, which Python uses when executing a with statement.
A context manager generally performs two responsibilities:
For example, a file context manager opens a file before the block and closes it when the block ends.
with expression as variable:
# Code that uses the managed resource
statement_1
statement_2Explanation:
with starts the context-management operation.expression produces a context manager.as variable stores the value returned by the manager's __enter__() method, if used.Without a context manager, a programmer must remember to clean up resources manually. If an exception interrupts the program before cleanup occurs, the resource may remain open or locked longer than intended.
file = open("sample.txt", "w", encoding="utf-8")
try:
file.write("Learning Python context managers.")
finally:
file.close()This example uses try and finally to ensure the file closes even if writing fails. It is valid, but it requires explicit resource-management code.
with open("sample.txt", "w", encoding="utf-8") as file:
file.write("Learning Python context managers.")The with statement provides the same basic resource-management benefit with less code. The file is automatically closed when execution leaves the block.
When Python encounters a with statement, it uses the context manager protocol to enter a context, execute the block, and exit the context. The exit operation is performed even if an exception occurs inside the block.
__enter__() method.as variable, the returned value is assigned to that variable.with block execute.__exit__() method.If an exception occurs inside the block, Python still invokes __exit__(). The context manager can decide whether the exception should continue propagating or be suppressed.
Custom class-based context managers normally implement two special methods: __enter__() and __exit__().
The __enter__() method runs when the context is entered. It can initialize an object, acquire a resource, or return a value that will be assigned to the variable after as.
The __exit__() method runs when the context is exited. It is commonly used to release resources, restore previous settings, or perform other cleanup operations.
Its method signature is generally:
def __exit__(self, exc_type, exc_value, traceback):
...| Parameter | Description |
|---|---|
exc_type |
The exception type if an exception occurred; otherwise None. |
exc_value |
The exception instance if an exception occurred; otherwise None. |
traceback |
The traceback associated with the exception, if any; otherwise None. |
If __exit__() returns a truthy value, the exception raised inside the block is suppressed. If it returns a false value, including None, the exception continues to propagate normally.
Python allows developers to define their own context managers by creating a class that implements __enter__() and __exit__().
class SimpleContext:
def __enter__(self):
print("Entering the context")
return self
def __exit__(self, exc_type, exc_value, traceback):
print("Exiting the context")
return False
with SimpleContext() as context:
print("Inside the with block")Output:
Entering the context
Inside the with block
Exiting the contextExplanation:
SimpleContext.__enter__() method runs and returns the instance.context.__exit__() method runs when the block finishes.False, so any exception from the block would not be suppressed.File handling is one of the most common examples of context management in Python. The built-in file object supports the context management protocol.
with open("student.txt", "w", encoding="utf-8") as file:
file.write("Name: Asha\n")
file.write("Course: Python Programming\n")
print("File is open:", not file.closed)
print("File is closed automatically:", file.closed)Possible output:
File is open: True
File is closed automatically: TrueThe file remains open inside the block and is closed after the block completes. The variable file may still refer to the file object afterward, but that object is closed and cannot be used for normal reading or writing operations.
A context manager performs its exit logic even when the block raises an exception. This makes it especially useful for resource cleanup.
class ResourceManager:
def __enter__(self):
print("Resource acquired")
return self
def __exit__(self, exc_type, exc_value, traceback):
print("Resource released")
if exc_type is not None:
print("An exception occurred:", exc_type.__name__)
return False
with ResourceManager():
print("Performing an operation")
raise ValueError("Example error")Output before the exception traceback:
Resource acquired
Performing an operation
Resource released
An exception occurred: ValueErrorThe exception is not suppressed because __exit__() returns False. Python therefore continues propagating the ValueError after cleanup.
If __enter__() itself raises an exception, the same manager's __exit__() method is not called because the context was never successfully entered. Therefore, if setup can partially acquire a resource and then fail, the setup method should handle its own partial cleanup.
Python provides the contextlib module to simplify context-manager creation. One useful tool is the @contextmanager decorator, which allows a generator function to act as a context manager.
In this pattern:
yield performs setup.yield is made available to the as variable.yield performs cleanup.from contextlib import contextmanager
@contextmanager
def managed_operation():
print("Setup: preparing the operation")
try:
yield "Resource is ready"
finally:
print("Cleanup: finishing the operation")
with managed_operation() as message:
print(message)
print("Executing the main task")Output:
Setup: preparing the operation
Resource is ready
Executing the main task
Cleanup: finishing the operationThe finally block ensures cleanup happens when the context exits, including when an exception occurs after the generator yields control.
A timer context manager can measure how long a block of code takes to execute. This is useful when testing performance or comparing alternative implementations.
from contextlib import contextmanager
from time import perf_counter
@contextmanager
def timer():
start = perf_counter()
try:
yield
finally:
elapsed = perf_counter() - start
print(f"Elapsed time: {elapsed:.4f} seconds")
with timer():
total = sum(range(1_000_000))
print("Calculation completed:", total)The timer records a start time before the block and calculates elapsed time afterward. The actual duration depends on the computer and current workload.
Python allows multiple context managers to be used in one with statement. This is helpful when an operation requires multiple files or resources.
with open("source.txt", "r", encoding="utf-8") as source, \
open("destination.txt", "w", encoding="utf-8") as destination:
destination.write(source.read())
print("File content copied successfully.")This example reads all text from source.txt and writes it to destination.txt. Both files are closed automatically when the block ends. The source file must exist, and the destination file is created or truncated.
When multiple context managers are used, Python enters them in order and exits them in reverse order.
The ExitStack class from contextlib is useful when the number of context managers is not known until runtime. It lets a program register resources dynamically and ensures their exit operations run when the stack's context ends.
from contextlib import ExitStack
filenames = ["first.txt", "second.txt", "third.txt"]
with ExitStack() as stack:
files = [
stack.enter_context(
open(filename, "r", encoding="utf-8")
)
for filename in filenames
]
for file in files:
print(file.read())All three files are opened inside the stack. If an opening or reading operation fails after earlier files have been opened, the stack still manages the cleanup of resources already registered with it. In this example, the files must exist and be readable.
| Use Case | Example | Benefit |
|---|---|---|
| File handling | with open(...) |
Closes files automatically. |
| Database operations | Database connection or transaction context | Can help manage transaction boundaries and resource cleanup, depending on the library. |
| Thread synchronization | with lock: |
Acquires a lock and releases it when the block exits. |
| Temporary resources | TemporaryDirectory |
Manages temporary directories and their cleanup. |
| Timing operations | A custom timer context manager | Measures elapsed time around a block. |
| Changing settings temporarily | Custom configuration context | Can restore a previous setting after a task. |
In multithreaded programs, locks help protect shared data from simultaneous modification. A lock object can be used as a context manager so it is released when the block exits.
from threading import Lock
lock = Lock()
shared_count = 0
def increment():
global shared_count
with lock:
shared_count += 1
increment()
print(shared_count)Output:
1The with lock: statement acquires the lock before the block and releases it afterward. In real multithreaded applications, all relevant access to shared state must follow an appropriate synchronization strategy.
The tempfile module provides context managers for temporary files and directories. These are useful when a program needs temporary storage without keeping it permanently.
from tempfile import TemporaryDirectory
from pathlib import Path
with TemporaryDirectory() as temp_dir:
file_path = Path(temp_dir) / "example.txt"
file_path.write_text(
"Temporary information",
encoding="utf-8"
)
print(file_path.read_text(encoding="utf-8"))
print("Temporary directory cleanup completed.")The temporary directory is automatically cleaned up when the context exits. Cleanup can depend on operating-system conditions, and files should not be expected to remain available after the context ends.
| Feature | Context Manager | try-finally |
|---|---|---|
| Purpose | Provides a reusable resource-management protocol. | Guarantees that finalization code runs when control leaves the try block. |
| Syntax | Uses with. |
Uses try and finally. |
| Reusability | Can package setup and cleanup into reusable objects or functions. | Cleanup logic is written directly in the statement. |
| Readability | Often concise for resource management. | Useful for general cleanup that does not need a context manager. |
| Exception handling | Calls the context manager's exit logic; the manager may suppress an exception. | The finally block runs, while the original exception normally continues unless separately handled. |
with statement must be indented inside its block.__enter__() becomes the value assigned after as.__exit__() unless suppressing the exception is intentional.yield inside a try/finally structure.with statement whenever a resource supports context management.try and finally inside custom managers when cleanup must always run after setup succeeds.contextlib.contextmanager for straightforward generator-based context managers.ExitStack when managing a dynamic collection of context managers.Objective: Create a custom context manager that logs when an operation starts and finishes. This project demonstrates setup, cleanup, and exception handling.
from contextlib import contextmanager
from datetime import datetime
@contextmanager
def activity_logger(filename):
start_time = datetime.now()
with open(filename, "a", encoding="utf-8") as log:
log.write(f"Started: {start_time}\n")
try:
print("Activity started.")
yield
except Exception as error:
with open(filename, "a", encoding="utf-8") as log:
log.write(f"Error: {error}\n")
raise
finally:
end_time = datetime.now()
with open(filename, "a", encoding="utf-8") as log:
log.write(f"Finished: {end_time}\n")
print("Activity finished.")
with activity_logger("activity.log"):
print("Performing an important task...")
result = sum(range(1, 1001))
print("Calculation result:", result)activity_logger() function creates a context manager using @contextmanager.with statement.finally block records the finish time whether the task succeeds or fails.with open() statements automatically close the log file after each write.Expected behavior: The terminal displays messages indicating that the activity started and finished. The file activity.log stores timestamps and any error details. The exact timestamps depend on when the program runs.
with statement.@contextmanager to create a context manager that temporarily changes a value and restores it afterward.with statement and copy the content of one file into the other.ExitStack to open several files based on a list of filenames.__exit__() suppresses an exception. Explain why this should be used cautiously.Answer: A context manager is an object that manages setup and cleanup around a block of code. It commonly acquires a resource before the block and releases it afterward.
Answer: The with statement uses a context manager to establish and exit a managed context. It helps ensure cleanup occurs when the block ends.
Answer: These are methods in the context management protocol. __enter__() performs setup and returns the value optionally assigned after as. __exit__() performs cleanup and can determine whether an exception is suppressed.
Answer: Yes, if the context was entered successfully, Python invokes __exit__() when the block exits due to an exception. The method receives information about the exception.
Answer: No. If __enter__() raises an exception before the context is successfully entered, that manager's __exit__() is not called. Partial setup cleanup must be handled within the setup logic.
Answer: A truthy return value suppresses an exception raised inside the block. A false value or None allows the exception to propagate normally.
Answer: The contextlib module provides tools for creating and using context managers, including @contextmanager, closing(), and ExitStack.
Answer: It converts a generator function into a context manager. The function yields control to the block, with setup code before yield and cleanup code afterward.
Answer: ExitStack manages multiple context managers dynamically, especially when the number of resources is determined at runtime.
Answer: Yes. Multiple resources can be managed in a single statement, and Python exits them in reverse order from the order in which they were entered.
Answer: A class-based manager implements __enter__() and __exit__() directly. A generator-based manager uses @contextmanager and a generator function containing yield.
Answer: Yes. For example, with lock: acquires a supported lock before the block and releases it when the block exits.
with statement is used to enter and exit a context.__enter__() runs when entering the context.__exit__() runs when leaving a successfully entered context.__exit__() suppresses the exception.@contextmanager simplifies the creation of generator-based context managers.ExitStack is useful for managing resources dynamically.Context managers are an essential Python programming feature for managing resources safely and writing maintainable code. The with statement simplifies setup and cleanup by invoking the context manager protocol automatically. Built-in context managers make common operations, such as working with files and locks, easier to manage.
By understanding __enter__(), __exit__(), exception behavior, and the utilities available in contextlib, developers can create reusable resource-management solutions. Practice with both class-based and generator-based context managers to understand when each approach is most appropriate.
