Issue 26042: Consider dropping magic number for more detailed .pyc file name (original) (raw)

This issue has been migrated to GitHub: https://github.com/python/cpython/issues/70230

classification

process

Created on 2016-01-07 17:52 by brett.cannon, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Messages (4)
msg257705 - (view)	Author: Brett Cannon (brett.cannon) *	Date: 2016-01-07 17:52
The reason the magic number exists in .pyc files is because back in Python 2 we only had a single .pyc file per module, which meant if you suddenly switched versions of Python you needed a way to know that the .pyc file was wrong for the running version of Python. This is not the case in Python 3. Thanks to __pycache__ directories we have separate .pyc files per release version of Python (we also have .pyc files for each optimization level of Python). If we changed the sys.implementation.cache_tag to include the bugfix version -- and maybe even release level -- then the magic number wouldn't be necessary for users. It does make developing bytecode a little bit more difficult for core developers since they will have to clear out their .pyc files during development, but users wouldn't be affected. Now I don't know if this is really worth the simplification it provides. I don't think it's worth the compatibility break for any code that may be reading .pyc files and I doubt there is any measurable performance benefit. But the realization struck me and I figured I should at least write it down in case anyone else thinks of it.
msg257716 - (view)	Author: Eric Snow (eric.snow) *	Date: 2016-01-07 21:00
That's an interesting thought. My intuition is that it wouldn't be worth doing. It would certainly trade one development overhead for another, so it's a bit of a wash there, I think. Are there other spots in the stdlib that rely on the magic number? Is it used much outside the stdlib?
msg257722 - (view)	Author: Serhiy Storchaka (serhiy.storchaka) *	Date: 2016-01-07 22:33
What about zipimport? It doesn't use __pycache__ directories, but just try to read *.pyc file, then *.py file. The launcher on Windows also check magic number (it contains a mapping from magic numbers to Python versions).
msg257723 - (view)	Author: Brett Cannon (brett.cannon) *	Date: 2016-01-07 22:40
I figured it was a wash. Closing as "won't fix". As for zipimport, it's busted thanks to the fact that it doesn't work with __pycache__ directories. Yet another reason it needs to be rewritten.