[Python-Dev] debugging test_importlib.test_bad_traverse - script status is SUCCESS (original) (raw)

Michael aixtools at felt.demon.nl
Mon Sep 17 03:39:22 EDT 2018

I read the discussion related to issue32374. That seems to be sure that other events that could cause the test to fail (i.e., the program executes successfully) are caught early, and/or ignored so that the program fails - and the test succeeds.

I am having trouble figuring out why the script below does not fail on AIX, and would appreciate your assistance in debugging what is happening, i.e., getting deeper.

Many thanks!

  +270      @unittest.skipIf(not hasattr(sys, 'gettotalrefcount'),   +271              '--with-pydebug has to be enabled for this test')   +272      def test_bad_traverse(self):   +273          ''' Issue #32374: Test that traverse fails when accessing per-module   +274              state before Py_mod_exec was executed.   +275              (Multiphase initialization modules only)   +276          '''   +277          script = """if True:   +278                  try:   +279                      from test import support   +280                      import importlib.util as util   +281                      spec = util.find_spec('_testmultiphase')   +282                      spec.name = '_testmultiphase_with_bad_traverse'   +283   +284                      with support.SuppressCrashReport():   +285                          m = spec.loader.create_module(spec)   +286                  except:   +287                      # Prevent Python-level exceptions from   +288                      # ending the process with non-zero status   +289                      # (We are testing for a crash in C-code)   +290                      pass"""   +291          assert_python_failure("-c", script)

To make sure the full debug info is loaded I added "-X dev", and for your reading added some additional print statements - and for speed run the command directly. Regardless of how I run it (calling as a test, or directly) the end-result is the same.

Note: I was not able to fine the default "loader.create_module() code"

to add debugging statements.

Pointer for that is welcome!

./python -X dev '-X' 'faulthandler' '-I' '-c' "if True:                 try:                     from test import support                     import importlib.util as util                     spec = util.find_spec('_testmultiphase')                     spec.name = '_testmultiphase_with_bad_traverse'

                    m = spec.loader.create_module(spec)                     print(m)                     print(dir(m))                     print(m.doc)                     print(m.loader)                     print(m.name)                     print(m.package)                     print(m.spec)                 except:                     # Prevent Python-level exceptions from                     # ending the process with non-zero status                     # (We are testing for a crash in C-code)                     print('in except')" <module '_testmultiphase_with_bad_traverse'> ['doc', 'loader', 'name', 'package', 'spec'] Test module _testmultiphase_with_bad_traverse None _testmultiphase_with_bad_traverse None None root at x066:[/data/prj/python/git/Python3-3.8.0]echo $? 0

To get some additional idea of what is happening I added some fprintf statements:

The additional debug info is: (see diff below)

  1. bad_traverse:0
  2. bad_traverse:0
  3. bad_traverse:0
  4. bad_traverse:0
  5. bad_traverse:0
  6. bad_traverse:0

*** To my SURPRISE *** only one routine with these print statements is ever called. I was expecting at more. (only bad_traverse(...) gets called, I was expecting both bad_traverse_test (Objects/moduleobject.c) and some kind of initialization of m_state->integer.

Since the macro Py_VISIT includes a return() statement, and my debug statement always print the second line - I assume Py_VISIT(m_state->integer) is not doing anything (i.e., vret == 0)

/* Utility macro to help write tp_traverse functions.  * To use this macro, the tp_traverse function must name its arguments  * "visit" and "arg".  This is intended to keep tp_traverse functions  * looking as much alike as possible.  */ #define Py_VISIT(op)                                                   
    do {                                                               
        if (op) {                                                      
            int vret = visit((PyObject *)(op), arg);                   
            if (vret)                                                  
                return vret;                                           
        }                                                              
    } while (0)

Is this what it should be?

root at x066:[/data/prj/python/git/Python3-3.8.0]git status On branch aix-pr Changes not staged for commit:   (use "git add ..." to update what will be committed)   (use "git checkout -- ..." to discard changes in working directory)

        modified:   Modules/_testmultiphase.c         modified:   Objects/moduleobject.c

no changes added to commit (use "git add" and/or "git commit -a") root at x066:[/data/prj/python/git/Python3-3.8.0]git diff root at x066:[/data/prj/python/git/Python3-3.8.0]git diff | cat diff --git a/Modules/_testmultiphase.c b/Modules/_testmultiphase.c index 5776df7d76..c28aef1455 100644 --- a/Modules/_testmultiphase.c +++ b/Modules/_testmultiphase.c @@ -622,23 +622,34 @@ PyInit__testmultiphase_exec_unreported_exception(PyObject *spec)  static int  bad_traverse(PyObject *self, visitproc visit, void *arg) {      testmultiphase_state *m_state; +FILE *errmsg = fopen("/tmp/err", "a");

     m_state = PyModule_GetState(self); +fprintf(errmsg,"1. bad_traverse:%ld\n", m_state->integer);      Py_VISIT(m_state->integer); +fprintf(errmsg,"2. bad_traverse:%ld\n", m_state->integer); +fclose(errmsg);      return 0;  }

 static int  execfunc_with_bad_traverse(PyObject *mod) {      testmultiphase_state *m_state; +FILE *errmsg; +errmsg = fopen("/tmp/err", "a");

     m_state = PyModule_GetState(mod);      if (m_state == NULL) { +fprintf(errmsg,"0.execfunc:\n"); +fclose(errmsg);          return -1;      }

     m_state->integer = PyLong_FromLong(0x7fffffff); +fprintf(errmsg,"1.execfunc:%ld\n", m_state->integer);      Py_INCREF(m_state->integer); +fprintf(errmsg,"2.execfunc:%ld\n", m_state->integer); +fclose(errmsg);

     return 0;  } diff --git a/Objects/moduleobject.c b/Objects/moduleobject.c index ccf5f8e6d1..603611c686 100644 --- a/Objects/moduleobject.c +++ b/Objects/moduleobject.c @@ -27,6 +27,9 @@ static PyMemberDef module_members[] = {  #ifdef Py_DEBUG  static int  bad_traverse_test(PyObject *self, void *arg) { +FILE *errmsg = fopen("/tmp/err","a"); +fprintf(errmsg,"bad_traverse_test: self!=NULL:%d\n", self != NULL); +fclose(errmsg);      assert(self != NULL);      return 0;  } root at x066:[/data/prj/python/git/Python3-3.8.0]

~

-------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 488 bytes Desc: OpenPGP digital signature URL: <http://mail.python.org/pipermail/python-dev/attachments/20180917/a427d6d1/attachment.sig>

More information about the Python-Dev mailing list