| msg272557 - (view) |
Author: wevsty (wevsty) |
Date: 2016-08-12 18:59 |
| When I use Python 3.5.2 some of the time will be a crash.displayed crash module is python35.dll. I run Python environment is Windows Server 2012R2.I think crash dump maybe will be helpful.Report.wer and carsh dump can be downloaded via the following link. http://o8z0k9748.bkt.clouddn.com/Python_crash_Dump_20160813_AF1A29AF.7z I have another question when i using managers lib multiprocessing Maybe have exception. my code is： Session_lock.G_Process_Lock = multiprocessing.Manager().Lock() def execute_select_command(command): try: with Session_lock.G_Process_Lock: values = [] conn = sqlite3.connect(SQLite_DB_NAME) cursor = conn.cursor() try: cursor.execute(command) values = cursor.fetchall() except Exception as e: pass finally: cursor.close() conn.close() return values except Exception as e: print(e) traceback.print_exc() finally: pass This code is possible to print out the information. Traceback (most recent call last): File "C:/Users/Administrator/Desktop/Monitor_URL\Monitor_SQL.py", line 44, in execute_select_command return values File "C:\Users\Administrator\AppData\Local\Programs\Python\Python35\lib\multiprocessing\managers.py", line 949, in __exit__ return self._callmethod('release') File "C:\Users\Administrator\AppData\Local\Programs\Python\Python35\lib\multiprocessing\managers.py", line 716, in _callmethod conn.send((self._id, methodname, args, kwds)) File "C:\Users\Administrator\AppData\Local\Programs\Python\Python35\lib\multiprocessing\connection.py", line 206, in send self._send_bytes(ForkingPickler.dumps(obj)) File "C:\Users\Administrator\AppData\Local\Programs\Python\Python35\lib\multiprocessing\connection.py", line 280, in _send_bytes ov, err = _winapi.WriteFile(self._handle, buf, overlapped=True) TypeError: WriteFile() argument 1 must be int, not None This maybe seem like a lib problem. |
|
|
| msg272577 - (view) |
Author: Eryk Sun (eryksun) *  |
Date: 2016-08-13 05:34 |
| To diagnose the access violation, it may help if you install the debug binaries and try to reproduce the crash using python_d.exe. Attach the dump file to this issue. Regarding the multiprocessing question, if its a separate issue you need to file it on its own and nosy davin. At first glance, I don't see why the handle is None in the lock's __exit__ method. After you acquire the lock, try logging the connection information. For example: import multiprocessing as mp import multiprocessing.util def test(lock): with lock: mp.util.info('test:lock:token: %r' % (lock._token,)) mp.util.info('test:lock:handle: %r' % (lock._tls.connection._handle,)) if __name__ == '__main__': mp.util.log_to_stderr(mp.util.INFO) lock = mp.Manager().Lock() p = mp.Process(target=test, args=(lock,)) p.start() p.join() |
|
|
| msg272581 - (view) |
Author: wevsty (wevsty) |
Date: 2016-08-13 10:13 |
| About the crash,python_d.exe crash dump download address is http://o8z0k9748.bkt.clouddn.com/python_d_from_vs_20160813_930A2640.7z Crash dump display,at the > python35_d.dll!_Py_ForgetReference(_object * op) Line 1758 C python35_d.dll!_Py_Dealloc(_object * op) Line 1786 C python35_d.dll!tupledealloc(PyTupleObject * op) Line 236 C I hope this can be helpful to you. About the multiprocessing question I modified my code becomes def execute_select_command(command): values = [] try: with Session_lock.G_Process_Lock: mp.util.info('test:lock:token: %r' % (Session_lock.G_Process_Lock._token,)) mp.util.info('test:lock:handle: %r' % (Session_lock.G_Process_Lock._tls.connection._handle,)) conn = sqlite3.connect(SQLite_DB_NAME,check_same_thread = False) cursor = conn.cursor() try: cursor.execute(command) values = cursor.fetchall() except Exception as e: pass finally: cursor.close() conn.close() return values except Exception as e: print(e) traceback.print_exc() mp.util.info('Exception:lock:token: %r' % (Session_lock.G_Process_Lock._token,)) mp.util.info('Exception:lock:handle: %r' % (Session_lock.G_Process_Lock._tls.connection._handle,)) pass return values finally: pass After run，Console output [INFO/SpawnPoolWorker-18] test:lock:token: Token(typeid='Lock', address='\\\\.\\pipe\\pyc-2876-0-n6aj8inv', id='d0b0efa7d0') [INFO/SpawnPoolWorker-18] test:lock:handle: 720 [INFO/SpawnPoolWorker-17] test:lock:token: Token(typeid='Lock', address='\\\\.\\pipe\\pyc-2876-0-n6aj8inv', id='d0b0efa7d0') [INFO/SpawnPoolWorker-17] test:lock:handle: 908 [INFO/SpawnPoolWorker-16] test:lock:token: Token(typeid='Lock', address='\\\\.\\pipe\\pyc-2876-0-n6aj8inv', id='d0b0efa7d0') [INFO/SpawnPoolWorker-16] test:lock:handle: 744 WriteFile() argument 1 must be int, not None [INFO/SpawnPoolWorker-16] Exception:lock:token: Token(typeid='Lock', address='\\\\.\\pipe\\pyc-2876-0-n6aj8inv', id='d0b0efa7d0') [INFO/SpawnPoolWorker-16] Exception:lock:handle: 744 Could not find thread pid_2720_id_389212239784 this problem and crash both the use of multiprocessing lib after appears.Maybe these two issues are related. If you need my full code or need to open a new issue, please let me know.Thinks |
|
|
| msg272584 - (view) |
Author: Decorater (Decorater) * |
Date: 2016-08-13 12:08 |
| Crashes also happen from Recursion if not careful. Idk why but it happens. Although some of it is probably by not using try / catch probably anywhere in the python source code as a safeguard. I always like try / catch even in C++ as you never know when crap might happen to make things crash. A Perfect Example is a Access violation. (Yes I have this is a Standard for me and it gets me to great quality code) (this makes me wonder if Windows does not use try / catch anywhere. It would be nice if Microsoft Required try / catch in every function for drivers as well to hopefully make them more stable.) All I know is I like having a lot of Checks (even if not required) that can actually prevent any and all exceptions to begin with. And yes from what I learned from experiance myself. Always use try and catch anything before it is to late will help you in the long run. (not to mention I blame Microsoft for not replacing their current Windows Update for when Updates Fail with a Detailed list of the names of the update(s) that failed and a description of what the reason is without Error code(s) as the Codes does not tell crap about why they failed. So, yeah I point my fingers at whoever wrote Windows Update, to fix their crap. As this is a issue in windows 7 SP1, 8, 8.1, and 10.) |
|
|
| msg272589 - (view) |
Author: R. David Murray (r.david.murray) *  |
Date: 2016-08-13 14:19 |
| Decorator: In python code you generally want to let the exception bubble up so that you can see what the error was. Sometimes you do want to catch general errors, but mostly you don't. As for C++ try/catch, CPython is written in C, not C++. So, your comment doesn't really have any relevance to this issue, I'm afraid. |
|
|
| msg272615 - (view) |
Author: Decorater (Decorater) * |
Date: 2016-08-13 20:39 |
| I can confirm that I also get the same crash. |
|
|
| msg272616 - (view) |
Author: Decorater (Decorater) * |
Date: 2016-08-13 20:39 |
| I do know for sure the crash happens in python.exe and blames ntdll.dll for it. |
|
|
| msg272647 - (view) |
Author: wevsty (wevsty) |
Date: 2016-08-14 05:41 |
| To Decorater If you are found to have crashes, I'm sorry, you can follow the previous suggestion to use python_d.exe then open a new issue uploading crash dump. It would be better. About the multiprocessing question，I would like to add some information. My code uses a multiprocessing Pool，I used multiprocessing.util.debug opened the debug information.debug information is as follows. [DEBUG/SpawnPoolWorker-7] thread 'MainThread' does not own a connection [DEBUG/SpawnPoolWorker-7] making connection to manager [DEBUG/SyncManager-2] starting server thread to service 'SpawnPoolWorker-7' [INFO/SpawnPoolWorker-4] test:lock:token: Token(typeid='Lock', address='\\\\.\\pipe\\pyc-3816-0-95051ql_', id='e44c9bc720') [INFO/SpawnPoolWorker-4] test:lock:handle: 712 [INFO/SpawnPoolWorker-7] test:lock:token: Token(typeid='Lock', address='\\\\.\\pipe\\pyc-3816-0-95051ql_', id='e44c9bc720') [INFO/SpawnPoolWorker-7] test:lock:handle: 768 [DEBUG/SpawnPoolWorker-7] DECREF 'e44c9bc720' [INFO/SpawnPoolWorker-4] leave:lock:token: Token(typeid='Lock', address='\\\\.\\pipe\\pyc-3816-0-95051ql_', id='e44c9bc720') [DEBUG/SpawnPoolWorker-7] thread 'MainThread' has no more proxies so closing conn [DEBUG/SpawnPoolWorker-7] DECREF 'b0182059f8' [DEBUG/SyncManager-2] got EOF -- exiting thread serving 'SpawnPoolWorker-7' [DEBUG/SpawnPoolWorker-4] INCREF 'e44c9bc720' handle is closed 'ForkAwareLocal' object has no attribute 'connection' [DEBUG/SpawnPoolWorker-7] thread 'MainThread' has no more proxies so closing conn Traceback (most recent call last): File "C:/Users/Administrator/Desktop/Monitor_URL\Monitor_SQL.py", line 51, in execute_select_command conn.close() File "C:\Users\Administrator\AppData\Local\Programs\Python\Python35\lib\multiprocessing\managers.py", line 949, in __exit__ return self._callmethod('release') File "C:\Users\Administrator\AppData\Local\Programs\Python\Python35\lib\multiprocessing\managers.py", line 716, in _callmethod conn.send((self._id, methodname, args, kwds)) File "C:\Users\Administrator\AppData\Local\Programs\Python\Python35\lib\multiprocessing\connection.py", line 204, in send self._check_closed() File "C:\Users\Administrator\AppData\Local\Programs\Python\Python35\lib\multiprocessing\connection.py", line 136, in _check_closed raise OSError("handle is closed") OSError: handle is closed [INFO/SpawnPoolWorker-7] Exception:lock:token: Token(typeid='Lock', address='\\\\.\\pipe\\pyc-3816-0-95051ql_', id='e44c9bc720') [INFO/SpawnPoolWorker-7] Exception process:lock:token: Token(typeid='Lock', address='\\\\.\\pipe\\pyc-3816-0-95051ql_', id='e44c9bc720') [DEBUG/SyncManager-1] got EOF -- exiting thread serving 'SpawnPoolWorker-7' [DEBUG/SpawnPoolWorker-4] INCREF 'b0182059f8' [DEBUG/SpawnPoolWorker-4] DECREF 'b0182059f8' [DEBUG/SpawnPoolWorker-7] INCREF 'e44c9bc720' This time the message and not the same，From the information looks like two processes simultaneously locking the same lock object, and then release happened the problems.I do not know if this is correct， But i will continue to follow the issue. |
|
|
| msg272653 - (view) |
Author: Decorater (Decorater) * |
Date: 2016-08-14 07:29 |
| I use code that does multithreading to from a library that uses ffmpeg for exact it makes daemon threads so it could be from that. |
|
|
| msg272743 - (view) |
Author: wevsty (wevsty) |
Date: 2016-08-15 10:01 |
| I try to change the multiprocessing lib codes，upload the file is debugger display.I try to used while loop make sure connection hanle value is not none,but connection hanle value will still becomes none. I think there may be other threads to modify the handle，But it not my code. |
|
|
| msg272861 - (view) |
Author: wevsty (wevsty) |
Date: 2016-08-16 15:51 |
| today，Find a new crash python_d_crash_dump_20160816_A901A6F3.7z，is full crash dump Download address http://o8z0k9748.bkt.clouddn.com/python_d_crash_dump_20160816_A901A6F3.7z dump display crash in the > python35_d.dll!visit_decref(_object * op, void * data) Line 379 C python35_d.dll!dict_traverse(_object * op, int(*)(_object *, void *) visit, void * arg) Line 2534 C python35_d.dll!subtract_refs(_gc_head * containers) Line 401 C python35_d.dll!collect(int generation, __int64 * n_collected, __int64 * n_uncollectable, int nofail) Line 959 C |
|
|
| msg273007 - (view) |
Author: wevsty (wevsty) |
Date: 2016-08-18 00:54 |
| A similar situation also occurs in the following Linux，A similar situation on stackoverflow http://stackoverflow.com/questions/29277150/python-3-4-multiprocessing-bug-on-lock-acquire-typeerror-integer-required |
|
|
| msg334464 - (view) |
Author: Michael Jacob (Michael Jacob2) |
Date: 2019-01-28 13:14 |
| Does this ticket track the issue in its title or the native crash? If it's the latter, is there a new ticket for the None _handle or shall I create one? |
|
|
| msg334628 - (view) |
Author: Michael Jacob (Michael Jacob2) |
Date: 2019-01-31 13:36 |
| So, I'm experiencing the issue in the title, too. The pipe handle stays valid for between 5 and 60 minutes, then it goes None when written to. I'm far from understanding that code, but this crude re-connect code seems to solve the issue for me: In BaseProxy._callmethod, I changed: conn.send((self._id, methodname, args, kwds)) to: try: conn.send((self._id, methodname, args, kwds)) except TypeError: self._connect() conn = self._tls.connection conn.send((self._id, methodname, args, kwds)) |
|
|
