[Python-Dev] Planning on removing cache invalidation for file finders (original) (raw)

Nick Coghlan ncoghlan at gmail.com
Sat Mar 2 18:24:01 CET 2013

On Sun, Mar 3, 2013 at 2:16 AM, Brett Cannon <brett at python.org> wrote:

On Sat, Mar 2, 2013 at 10:36 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:

I think you should keep it. A long running service that periodically scans the importers for plugins doesn't care if modules take a few extra seconds to show up, it just wants to see them eventually. Installers (or filesystem copy or move operations!) have no way to inform arbitrary processes that new files have been added.

But if they are doing the scan they can also easily invalidate the caches before performing the scan.

"I just upgraded to Python 3.4, and now my server process isn't see new plugins"

That's a major backwards compatibility breach, and hence clearly unacceptable in my view. Even the relatively minor compatibility breach of becoming dependent on the filesystem timestamp resolution for picking up added modules, creating a race condition between writing the file and reading it back through the import system, has caused people grief. When you're in a hole, the first thing to do is to stop digging.

You can deprecate the heuristic if you want (and can figure out how), but a definite -1 on removing it without at least the usual deprecation period for backwards incompatible changes.

It may also be worth tweaking the wording of the upgrade note in the What's New to mention the need to always invalidate the cache before scanning for new modules if you want to reliably pick up new modules created since the application started (at the moment the note really only mentions it as something to do after creating a new module).

Cheers, Nick.

-- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia

More information about the Python-Dev mailing list