[Python-Dev] The future of the wchar_t cache (original) (raw)

Steve Dower steve.dower at python.org
Mon Oct 22 09:24:22 EDT 2018

On 22Oct2018 0913, Victor Stinner wrote:

Le lun. 22 oct. 2018 à 15:08, Steve Dower <steve.dower at python.org> a écrit :

Agreed the cache is useless here, but since the listdir() result came in as wchart we could keep it that way (assuming we'd only be changing it to char), and then there wouldn't have to be a conversion when we immediately pass it back to open(). Serhiy wants to remove the cache which should reduce Python memory footprint on Windows. You are proposing to fill the cache eagierly, that would increase the Python memory footprint :-/ Your proposed change is an optimisation, a benchmark is needed to see the benefit. I expect no significant difference on benchmarks of https://pyperformance.readthedocs.io/ ...

Yes, that's true. But "should reduce ... footprint" is also an optimisation that deserves a benchmark by that standard. Also, I'm proposing keeping the 'kind' as UCS-2 when the string is created from UCS-2 data that is likely to be used as UCS-2. We would not create the UCS-1 version in this case, so it's not the same as prefilling the cache, but it would cost a bit of memory in exchange for CPU. If slicing and concatentation between matching kinds also preserved the kind, a lot of path handling code could avoid back-and-forth conversions.

The import benchmarks ought to be improved on Windows by this new optimisation, as this is a prime case where we regularly convert strings from what the OS gave us into UCS-1 and back into what the OS expects. But if you don't run the benchmarks on all OS's, then sure, you won't see any difference :)

Cheers, Steve

More information about the Python-Dev mailing list