[Python-Dev] cpython and parallel make (original) (raw)

Jonas Wagner jonas.wagner at epfl.ch
Mon Sep 1 11:51:31 CEST 2014

Hi,

what’s people’s experience with compiling cpython using multiple jobs (e.g., make -j 8)?

In my case, I sometimes experience build errors that happen when using -j, whereas the single-job build always works. I haven’t tracked this down in detail, though… here’s an extract from a sample log:

17:42:56 make[1]: Entering directory /path/to/cpython/build' 17:42:57 clang -c -Wno-unused-result -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -O3 -fsanitize=address -Werror=declaration-after-statement -I. -IInclude -I../../cpython/Include -DPy_BUILD_CORE -o Python/Python-ast.o Python/Python-ast.c 17:42:57 clang -c -Wno-unused-result -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -O3 -fsanitize=address -Werror=declaration-after-statement -I. -IInclude -I../../cpython/Include -DPy_BUILD_CORE -o Python/ast.o ../../cpython/Python/ast.c 17:42:57 clang -c -Wno-unused-result -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -O3 -fsanitize=address -Werror=declaration-after-statement -I. -IInclude -I../../cpython/Include -DPy_BUILD_CORE -o Python/ast.o ../../cpython/Python/ast.c 17:42:57 clang -c -Wno-unused-result -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -O3 -fsanitize=address -Werror=declaration-after-statement -I. -IInclude -I../../cpython/Include -DPy_BUILD_CORE -o Python/ceval.o ../../cpython/Python/ceval.c 17:42:58 clang -c -Wno-unused-result -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -O3 -fsanitize=address -Werror=declaration-after-statement -I. -IInclude -I../../cpython/Include -DPy_BUILD_CORE -o Python/compile.o ../../cpython/Python/compile.c 17:43:07 clang -c -Wno-unused-result -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -O3 -fsanitize=address -Werror=declaration-after-statement -I. -IInclude -I../../cpython/Include -DPy_BUILD_CORE -o Python/graminit.o Python/graminit.c 17:43:07 clang -c -Wno-unused-result -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -O3 -fsanitize=address -Werror=declaration-after-statement -I. -IInclude -I../../cpython/Include -DPy_BUILD_CORE -o Python/ceval.o ../../cpython/Python/ceval.c 17:43:07 clang -c -Wno-unused-result -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -O3 -fsanitize=address -Werror=declaration-after-statement -I. -IInclude -I../../cpython/Include -DPy_BUILD_CORE -o Python/compile.o ../../cpython/Python/compile.c 17:43:08 make Include/graminit.h 17:43:08 make[1]: Entering directory /path/to/cpython/build' 17:43:08 make[1]: Include/graminit.h' is up to date. 17:43:08 make[1]: Leaving directory /path/to/cpython/build'

17:43:24 clang -fsanitize=address -o Modules/_freeze_importlib Modules/_freeze_importlib.o Modules/getbuildinfo.o Parser/acceler.o [...] Objects/abstract.o [...] Objects/weakrefobject.o Python/_warnings.o [...] Python/ceval.o Python/compile.o Python/codecs.o [...] Modules/faulthandler.o Modules/_tracemalloc.o Modules/hashtable.o Modules/symtablemodule.o Modules/xxsubtype.o -ldl -lutil -lm 17:43:24 /usr/bin/ld: error: Python/ceval.o: file is empty 17:43:24 /usr/bin/ld: error: Python/compile.o: file is empty

For some reason, some files like Python/ceval.o seem to be built multiple times. I reckon this causes race conditions that result in some empty object files, which causes the linker to fail.

Any ideas about where this comes from, and how to fix it?

Cheers, Jonas ​ -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20140901/57f13c99/attachment.html>

More information about the Python-Dev mailing list