Issue 7164: pickle test failure after test_imp/test_import (_make_stat_result is not the same object as os._make_stat_result) (original) (raw)

After r75467, which changed test_imp and test_import to use CleanImport, the various pickle tests fail (all in the same common test, I believe). The error looks like this:

Traceback (most recent call last): File "/home/rdmurray/python/trunk/Lib/test/pickletester.py", line 623, in test_structseq s = self.dumps(t, proto) File "/home/rdmurray/python/trunk/Lib/test/test_pickle.py", line 33, in dumps p.dump(arg) File "/home/rdmurray/python/trunk/Lib/pickle.py", line 224, in dump self.save(obj) File "/home/rdmurray/python/trunk/Lib/pickle.py", line 331, in save self.save_reduce(obj=obj, *rv) File "/home/rdmurray/python/trunk/Lib/pickle.py", line 400, in save_reduce save(func) File "/home/rdmurray/python/trunk/Lib/pickle.py", line 286, in save f(self, obj) # Call unbound method with explicit self File "/home/rdmurray/python/trunk/Lib/pickle.py", line 753, in save_global (obj, module, name)) PicklingError: Can't pickle <function _make_stat_result at 0xb7a09494>: it's not the same object as os._make_stat_result

I don't have enough pickle-foo to understand what it is trying to do here or what about CleanImport is messing it up.

Must have missed running the full test suite before checking that one in. Whoops...

Anyway, the culprit is a couple of copy_reg calls that the os module runs on import to register pickle support methods. CleanImport's simplistic approach to reversion leaves the replacement handlers registered even after the original os module has been restored, leading to copy_reg complaining when you attempt to use the registered pickle handlers.

I'll narrow the scope to just reverting os.environ and otherwise leave the new os module in place.

I'm experiencing a variation of this problem on a project where I'm trying to make an application that will work with both 2.7.x and 3.4.x and am mostly successful. The application uses a number of pickle files and I want to make it so that it doesn't matter which version created the pickle files the other can read it. By limiting the 3.4 version to pickle protocol 2 I've succeeded in having 2.7.x read 3.4.x pickle files but encountered a problem going the other way with the following error message:

Traceback (most recent call last): File "/home/peter/SRC/GITHUB/epygibus.git/epygibus3", line 25, in sys.exit(ARGS.run_cmd(ARGS)) File "/home/peter/SRC/GITHUB/epygibus.git/epygibus_pkg/cli/subcmd_restore.py", line 70, in run_cmd snapshot.restore_subdir(args.archive_name, os.sep, seln_fn=lambda l: l[-1-args.back]) File "/home/peter/SRC/GITHUB/epygibus.git/epygibus_pkg/snapshot.py", line 699, in restore_subdir snapshot_fs = get_snapshot_fs(archive_name, seln_fn).get_subdir(abs_subdir_path) File "/home/peter/SRC/GITHUB/epygibus.git/epygibus_pkg/snapshot.py", line 595, in get_snapshot_fs snapshot = read_snapshot(os.path.join(archive.snapshot_dir_path, snapshot_name)) File "/home/peter/SRC/GITHUB/epygibus.git/epygibus_pkg/snapshot.py", line 236, in read_snapshot return pickle.load(fobj) AttributeError: Can't get attribute '_make_stat_result' on <module 'os' from '/usr/lib64/python3.4/os.py'>

I'm pretty sure that this is a result of that pickle file containing instances of the output from os.lstat() generated by 2.7.x which 3.4.x is having trouble instantiating.

The source code for the application with the problem is available on github as repo "epygibus" for user "pwil3058".