[Python-Dev] Updates to PEP 471, the os.scandir() proposal (original) (raw)

Ethan Furman ethan at stoneleaf.us
Tue Jul 8 22:22:33 CEST 2014

On 07/08/2014 12:34 PM, Ben Hoyt wrote:

Better to just have the attributes be None if they were not fetched. None is better than hasattr anyway, at least in the respect of not having to catch exceptions to function properly. The thing is, isdir() and lstat() are not attributes (for a good reason). Please read the relevant "Rejected ideas" sections and let us know what you think. :-)

I did better than that -- I read the whole thing! ;)

-1 on the PEP's implementation.

Just like an attribute does not imply a system call, having a method named 'is_dir' /does/ imply a system call, and not having one can be just as misleading.

If we have this:

 size = 0
 for entry in scandir('/some/path'):
     size += entry.st_size

and the solution is equally simple:

 for entry in scandir('/some/path', stat=True):

Now, of course, we might get errors. I am not a big fan of wrapping everything in try/except, particularly when we already have a model to follow -- os.walk:

 for entry in scandir('/some/path', stat=True, onerror=record_and_skip):

If we don't care if an error crashes the script, leave off onerror.

If we don't need st_size and friends, leave off stat=True.

If we get better performance on Windows instead of Linux, that's okay.

scandir is going into os because it may not behave the same on every platform. Heck, even some non-os modules (multiprocessing comes to mind) do not behave the same on every platform.

I think caching the attributes for DirEntry is fine, but let's do it as a snapshot of that moment in time, not name now, and attributes in 30 minutes when we finally get to you because we had a lot of processing/files ahead of you (you being a DirEntry ;) .

-- Ethan

More information about the Python-Dev mailing list