Issue 21201: Uninformative error message in multiprocessing.Manager() (original) (raw)

While using multiprocessing.Manager() to send data between processes I've noticed that one of the traceback messages is not very informative:

Traceback (most recent call last): File "age_predict.py", line 39, in train_data, train_target, test_data = prepare_data() File "age_predict.py", line 30, in prepare_data train_data = q.get(True, 10000) File "", line 2, in get File "/usr/lib/python2.7/multiprocessing/managers.py", line 777, in _callmethod raise convert_to_error(kind, result) multiprocessing.managers.RemoteError:

Unserializable message: ('#RETURN', [A WHOLE LOT OF DATA GOES HERE]

The real problem here is that my machine is running out of memory, but the traceback message shadows this information and reports "Unserializable message" instead.

After a simple path (see: attached file) the traceback message is as follows:

Traceback (most recent call last): File "age_predict.py", line 39, in train_data, train_target, test_data = prepare_data() File "age_predict.py", line 30, in prepare_data train_data = q.get(True, 10000) File "", line 2, in get File "/usr/lib/python2.7/multiprocessing/managers.py", line 775, in _callmethod raise convert_to_error(kind, result) multiprocessing.managers.RemoteError:

Unserializable message: Traceback (most recent call last): File "/usr/lib/python2.7/multiprocessing/managers.py", line 288, in serve_client send(msg) MemoryError: out of memory

Here's another example (see: http://bugs.python.org/issue6766):

This python3 code:

from multiprocessing import Manager manager = Manager() ns_proxy = manager.Namespace() evt_proxy = manager.Event() ns_proxy.my_event_proxy = evt_proxy print(ns_proxy.my_event_proxy)

Produces following traceback message:

Traceback (most recent call last): File "test.py", line 6, in print(ns_proxy.my_event_proxy) File "/cpython/Lib/multiprocessing/managers.py", line 1032, in getattr return callmethod('getattribute', (key,)) File "/cpython/Lib/multiprocessing/managers.py", line 748, in _callmethod raise convert_to_error(kind, result) multiprocessing.managers.RemoteError:

Unserializable message: ('#RETURN', <threading.Event object at 0x7f9484aff278>)

...and after applying the attached patch it becomes clearer what's going on:

Traceback (most recent call last): File "test.py", line 6, in print(ns_proxy.my_event_proxy) File "/cpython/Lib/multiprocessing/managers.py", line 1032, in getattr return callmethod('getattribute', (key,)) File "/cpython/Lib/multiprocessing/managers.py", line 748, in _callmethod raise convert_to_error(kind, result) multiprocessing.managers.RemoteError:

Unserializable message: Traceback (most recent call last): File "/cpython/Lib/multiprocessing/managers.py", line 276, in serve_client send(msg) File "/cpython/Lib/multiprocessing/connection.py", line 206, in send self._send_bytes(ForkingPickler.dumps(obj)) File "/cpython/Lib/multiprocessing/reduction.py", line 50, in dumps cls(buf, protocol).dump(obj) _pickle.PicklingError: Can't pickle <class '_thread.lock'>: attribute lookup lock on _thread failed

This patch doesn't break any tests.