[Python-Dev] Planning on removing cache invalidation for file finders (original) (raw)

Brett Cannon brett at python.org
Sun Mar 3 00:38:15 CET 2013

On Sat, Mar 2, 2013 at 12:24 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

On Sun, Mar 3, 2013 at 2:16 AM, Brett Cannon <brett at python.org> wrote: > On Sat, Mar 2, 2013 at 10:36 AM, Nick Coghlan <ncoghlan at gmail.com> wrote: >> I think you should keep it. A long running service that periodically >> scans the importers for plugins doesn't care if modules take a few >> extra seconds to show up, it just wants to see them eventually. >> Installers (or filesystem copy or move operations!) have no way to >> inform arbitrary processes that new files have been added. > > > But if they are doing the scan they can also easily invalidate the caches > before performing the scan.

"I just upgraded to Python 3.4, and now my server process isn't see new plugins" That's a major backwards compatibility breach, and hence clearly unacceptable in my view. Even the relatively minor compatibility breach of becoming dependent on the filesystem timestamp resolution for picking up added modules, creating a race condition between writing the file and reading it back through the import system, has caused people grief. When you're in a hole, the first thing to do is to stop digging. You can deprecate the heuristic if you want (and can figure out how), but a definite -1 on removing it without at least the usual deprecation period for backwards incompatible changes.

That part is easy: ImportWarning still exists so simply continuing to check the directory and noticing when a difference exists that affects subsequent imports and then raising the warning will handle that.

It may also be worth tweaking the wording of the upgrade note in the What's New to mention the need to always invalidate the cache before scanning for new modules if you want to reliably pick up new modules created since the application started (at the moment the note really only mentions it as something to do after creating a new module). As of right now with the check that's all that is needed, but yes, if the deprecation does occur it would be worth changing it. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20130302/de0a1b0e/attachment.html>

More information about the Python-Dev mailing list