Fix GeneratorExp in LocalPythonExecutor to return Generator instead of list by albertvillanova · Pull Request #1650 · huggingface/smolagents (original) (raw)

This PR fixes a bug in the LocalPythonExecutor where ast.GeneratorExp nodes were incorrectly returning lists instead of proper Python generators.

Previously, generator expressions like (x for x in range(10)) would be evaluated eagerly and return a list, which breaks the expected lazy evaluation semantics of Python generators and could cause memory issues with large sequences.

Changes made:

• Fix evaluate_generatorexp function: Modified the implementation to return an actual Python generator object instead of a list
• Maintaine lazy evaluation: Generator expressions now properly implement lazy evaluation as expected in Python
• Preserve functionality: All existing logic for handling comprehension generators, conditions, and variable scoping remains intact

Tests:

• Add regression tests to verify generator expressions return proper generator objects

This fix ensures that the LocalPythonExecutor correctly handles generator expressions according to Python's standard behavior, maintaining both performance and semantic correctness.