Python Memory Profiling - tracemalloc Documentation (original) (raw)

What is tracemalloc?

Available since: Python 3.4 (PEP 454)

Tracemalloc is a powerful debugging tool included in Python's standard library that allows developers to trace memory blocks allocated by Python programs. Unlike traditional memory profilers that only show overall memory usage, tracemalloc provides detailed insights into where and how memory is being allocated in your code.

With tracemalloc, you can:

Installation & Setup

Since tracemalloc is part of the Python standard library, no additional installation is required. However, to use it effectively, you need to enable it at the start of your program.

import tracemalloc

# Start tracing Python memory allocations
tracemalloc.start()

# Your code here
# ...

# Get the current snapshot
snapshot = tracemalloc.take_snapshot()
top_stats = snapshot.statistics('lineno')

# Print statistics
for stat in top_stats[:10]:
    print(stat)
# Set environment variable before running your script
# Linux/macOS
export PYTHONTRACEMALLOC=1

# Windows
set PYTHONTRACEMALLOC=1

# Then run your Python program
python your_script.py
import tracemalloc

# Start tracemalloc with custom parameters
# The parameter 25 is the maximum number of frames 
# to store in a traceback
tracemalloc.start(25)

# Get current size and peak
current, peak = tracemalloc.get_traced_memory()
print(f"Current memory usage: {current / 1024}KB")
print(f"Peak memory usage: {peak / 1024}KB")

# Reset peak to current
tracemalloc.reset_peak()

# Stop tracking when done
tracemalloc.stop()

Basic Usage

The workflow for using tracemalloc typically follows these steps:

  1. 1

Start Tracing

Enable the tracemalloc module at the beginning of your program. 2. 2

Run Your Code

Execute the code you want to analyze for memory usage. 3. 3

Take Snapshots

Capture the memory state at specific points in execution. 4. 4

Analyze Results

Review statistics or compare snapshots to identify issues.

Common Use Cases

Key Features

Memory Allocation Tracking

Trace where objects are allocated with detailed tracebacks

Snapshot Capabilities

Capture memory states for analysis and comparison

Memory Usage Statistics

Get comprehensive memory usage data

Debugging Utilities

Tools to help interpret memory data

Working with Snapshots

Snapshots are a core concept in tracemalloc that allow you to capture the memory state at a specific point in time. By comparing snapshots taken at different times, you can identify memory leaks and understand memory usage patterns.

import tracemalloc

# Start tracing memory
tracemalloc.start()

# ... some initial code ...

# Take a first snapshot
snapshot1 = tracemalloc.take_snapshot()

# ... more code that might cause memory changes ...

# Take a second snapshot
snapshot2 = tracemalloc.take_snapshot()

# Compare snapshots
top_stats = snapshot2.compare_to(snapshot1, 'lineno')

print("Memory usage growth:")
for stat in top_stats[:10]:
    print(stat)

Pro Tip

When comparing snapshots, focus on the biggest differences first. These are often the most likely candidates for memory leaks or optimization opportunities.

Memory Statistics

Tracemalloc provides detailed statistics about memory usage in your application, helping you understand where memory is being allocated and how it's being used.

Available Statistics Methods

Method Description
statistics('lineno') Group by source file and line number
statistics('filename') Group by source file
statistics('traceback') Group by traceback
get_traced_memory() Get current and peak memory usage
get_tracemalloc_memory() Memory used by tracemalloc itself

Code Examples

Finding Memory Leaks

import tracemalloc
import gc

def detect_leaks():
    tracemalloc.start()
    
    # Run a baseline collection
    gc.collect()
    
    # Record the starting state
    snapshot1 = tracemalloc.take_snapshot()
    
    # Run the code that might leak memory
    potentially_leaky_function()
    
    # Force garbage collection to remove objects
    # that are no longer referenced
    gc.collect()
    
    # Take a second snapshot
    snapshot2 = tracemalloc.take_snapshot()
    
    # Compare snapshots
    top_stats = snapshot2.compare_to(snapshot1, 'lineno')
    
    print("Potential memory leaks:")
    for stat in top_stats[:5]:
        print(stat)

def potentially_leaky_function():
    # This is just a placeholder for your actual code
    large_list = [0] * 1000000  # Allocate a large list
    
    # In a real leak, this object would be
    # accidentally kept alive somewhere

detect_leaks()

Memory Usage Timeline

import tracemalloc
import time

# Start tracking memory
tracemalloc.start()
snapshots = []

# Function to record a memory snapshot with timestamp
def record_memory():
    snapshot = tracemalloc.take_snapshot()
    current, peak = tracemalloc.get_traced_memory()
    snapshots.append((time.time(), current, snapshot))
    return current

# Record initial state
record_memory()
print("Initial memory:", snapshots[0][1] / 1024, "KB")

# Record several points for analysis
for i in range(3):
    # Simulate work that uses memory
    data = [object() for _ in range(100000)]
    
    # Record the memory at this point
    current = record_memory()
    print(f"Memory after step {i+1}:", current / 1024, "KB")
    
    # For the last iteration, keep the reference
    # to prevent garbage collection
    if i < 2:
        del data  # Remove reference to allow cleanup

# Print memory growth timeline
print("\nMemory Timeline:")
for i, (timestamp, memory, _) in enumerate(snapshots):
    if i > 0:
        prev_time, prev_mem = snapshots[i-1][0], snapshots[i-1][1]
        time_diff = timestamp - prev_time
        mem_diff = memory - prev_mem
        print(f"Step {i}: {memory/1024:.2f} KB " + 
              f"({mem_diff/1024:+.2f} KB in {time_diff:.2f}s)")

Resources

Official

Python Documentation

The comprehensive official documentation for the tracemalloc module in Python 3.13.3, updated April 2025.

Read More

Tutorial

Memory Profiling Guide

A practical guide to memory profiling in Python using the standard library's tracemalloc module.

Read More

Advanced

Fixing Memory Leaks

An in-depth guide on using tracemalloc for memory profiling to diagnose and fix memory leaks in Python applications.

Read More

Video

Memory Profiling in Python

A comprehensive video tutorial on understanding and fixing memory issues in Python using tracemalloc and other tools.

Watch

Frequently Asked Questions

Tracemalloc adds some memory and CPU overhead to your application. The memory overhead depends on the number of frames tracked per allocation and the number of tracked allocations. In general, the overhead ranges from 5% to 20% depending on your application.

Use tracemalloc.get_tracemalloc_memory() to see how much memory tracemalloc itself is using.

No, tracemalloc only tracks memory blocks allocated by Python itself. Memory allocated by C extensions (like NumPy arrays using their own allocators) is not tracked unless the extension explicitly uses Python's memory allocation APIs.

For tracking memory in C extensions, consider external tools like Valgrind or Memory Profiler.

You can control the amount of traceback data collected in two ways:

  1. Limit the number of frames when starting tracemalloc: tracemalloc.start(nframe=1)
  2. Filter traces when taking a snapshot: snapshot.filter_traces(filters)

Both approaches can significantly reduce memory overhead for large applications.