[llvm-dev] Building LLVM's fuzzers (original) (raw)

Kostya Serebryany via llvm-dev llvm-dev at lists.llvm.org
Thu Aug 24 15:38:29 PDT 2017

On Thu, Aug 24, 2017 at 3:35 PM, Peter Collingbourne <peter at pcc.me.uk> wrote:

On Thu, Aug 24, 2017 at 3:21 PM, Kostya Serebryany via llvm-dev <_ _llvm-dev at lists.llvm.org> wrote:

On Thu, Aug 24, 2017 at 3:20 PM, Justin Bogner <mail at justinbogner.com> wrote: I think the simplest fix is something like this: diff --git a/lib/Transforms/Instrumentation/SanitizerCoverage.cpp b/lib/Transforms/Instrumentation/SanitizerCoverage.cpp index c6f0d17f8fe..e81957ab80a 100644 --- a/lib/Transforms/Instrumentation/SanitizerCoverage.cpp +++ b/lib/Transforms/Instrumentation/SanitizerCoverage.cpp @@ -256,6 +256,7 @@ SanitizerCoverageModule::CreateSecStartEnd(Module &M, const char *Section, new GlobalVariable(M, Ty, false, GlobalVariable::ExternalLinkage, nullptr, getSectionEnd(Section)); SecEnd->setVisibility(GlobalValue::HiddenVisibility); + appendToUsed(M, {SecStart, SecEnd}); return std::makepair(SecStart, SecEnd); } I'm trying it out now. LGTM (if this works), thanks! I wouldn't expect that to work because for ELF targets llvm.used has no effect on the object file (only on the optimizer). Is there a simple way to reproduce the link failure?

ninja compiler-rt echo 'extern "C" int LLVMFuzzerTestOneInput(const unsigned char *a, unsigned long b){return 0; } ' > test.cc clang -O3 test.cc -fsanitize=fuzzer # works clang -O3 test.cc -Wl,-gc-sections -fsanitize=fuzzer # fails

Peter

Kostya Serebryany <kcc at google.com> writes: > With -Wl,-gc-sections I get this: > SimpleTest.cpp:(.text.sancov.modulector[sancov.modulector]+0x1b): _> undefined reference to _start_sancovpcs'_ _> SimpleTest.cpp:(.text.sancov.modulector[sancov.modulector]+0x20):_ _> undefined reference to stop_sancovpcs' > > > > On Thu, Aug 24, 2017 at 3:07 PM, George Karpenkov <_ _ekarpenkov at apple.com> > wrote: > >> >> On Aug 24, 2017, at 2:55 PM, Kostya Serebryany <kcc at google.com> wrote: >> >> Interesting. >> This is a relatively new addition (fsanitize-coverage=pc-tables, which is >> now a part of -fsanitize=fuzzer). >> The tests worked (did they? On Mac?) so I thought everything is ok. >> >> >> For tests we never compile the tested target with -O3 (and that wouldn’t >> be sufficient), >> and for testing fuzzers I was always building them in debug >> >> Yea, we need to make sure the pc-tables are not stripped (this is a >> separate section with globals). >> (I still haven't documented pc-tables, will do soon) >> >> >> Do you know what's the analog of Wl,-deadstrip on Linux? >> >> >> Apparently -Wl,—gc-sections. >> For some reason LLVM does not do it for gold, even though it seems to >> support this flag as well. >> (that could be another reason why you don’t see the failure on Linux) >> >> 1 if(NOT LLVMNODEADSTRIP) >> 2 if(${CMAKESYSTEMNAME} MATCHES "Darwin") >> 3 # ld64's implementation of -deadstrip breaks tools that use >> plugins. >> 4 setproperty(TARGET ${targetname} APPENDSTRING PROPERTY >> 5 LINKFLAGS " -Wl,-deadstrip") >> 6 elseif(${CMAKESYSTEMNAME} MATCHES "SunOS") >> 7 setproperty(TARGET ${targetname} APPENDSTRING PROPERTY >> 8 LINKFLAGS " -Wl,-z -Wl,discard-unused=sections") >> 9 elseif(NOT WIN32 AND NOT LLVMLINKERISGOLD) >> 10 # Object files are compiled with -ffunction-data-sections. >> 11 # Versions of bfd ld < 2.23.1 have a bug in --gc-sections that_ _>> breaks >> 12 # tools that use plugins. Always pass --gc-sections once we require >> 13 # a newer linker. >> 14 setproperty(TARGET ${targetname} APPENDSTRING PROPERTY >> 15 LINKFLAGS " -Wl,--gc-sections") >> 16 endif() >> 17 endif() >> >> >> >> --kcc >> >> >> >> On Thu, Aug 24, 2017 at 2:49 PM, Justin Bogner <mail at justinbogner.com_ _> >> wrote: >> >>> George Karpenkov <ekarpenkov at apple.com> writes: >>> > OK so with Kuba’s help I’ve found the error: with optimization, dead >>> > stripping of produced libraries is enabled, >>> > which removes coverage instrumentation. >>> > >>> > However, this has nothing to do with the move to compiler-rt, so I’m >>> > quite skeptical on whether it has worked >>> > beforehand. >>> > >>> > A trivial fix is to do: >>> > >>> > diff --git a/cmake/modules/HandleLLVMOptions.cmake >>> b/cmake/modules/HandleLLVMOptions.cmake >>> > index 04596a6ff63..5465d8d95ba 100644 >>> > --- a/cmake/modules/HandleLLVMOptions.cmake >>> > +++ b/cmake/modules/HandleLLVMOptions.cmake >>> > @@ -665,6 +665,9 @@ if(LLVMUSESANITIZER) >>> > endif() >>> > if (LLVMUSESANITIZECOVERAGE) >>> > append("-fsanitize=fuzzer-no-link" CMAKECFLAGS CMAKECXXFLAGS) >>> > + >>> > + # Dead stripping messes up coverage instrumentation. >>> > + set(LLVMNODEADSTRIP ON) >>> > endif() >>> > endif() >>> > >>> > Any arguments against that? >>> >>> We shouldn't do this. We really only want to prevent dead stripping of >>> the counters themselves - disabling it completely isn't very nice. >>> >>> > Apparently, a better way is to follow ASAN instrumentation pass, >>> > which uses some magic to protect against dead-stripping. >>> >>> I thought this was already being done - how else did it work before? >>> >>> >> On Aug 24, 2017, at 11:29 AM, Justin Bogner <_ _mail at justinbogner.com> >>> wrote: >>> >> >>> >> (kcc, george: sorry for the re-send, the first was from a non-list >>> email >>> >> address) >>> >> >>> >> My configuration for building the fuzzers in the LLVM tree doesn't >>> seem to >>> >> work any more (possibly as of moving libFuzzer to compiler-rt, but >>> there >>> >> have been a few other changes in the last week or so that may be >>> related). >>> >> >>> >> I'm building with a fresh top-of-tree clang and setting >>> >> -DLLVMUSESANITIZER=Address and -DLLVMUSESANITIZECOVERAGE=On, >>> which >>> >> was working before: >>> >> _>>> >> % cmake -GNinja _ _>>> >> -DCMAKEBUILDTYPE=Release -DLLVMENABLEASSERTIONS=On _ _>>> >> -DLLVMENABLEWERROR=On _ >>> >> -DLLVMUSESANITIZER=Address -DLLVMUSESANITIZECOVERAGE=On _>>> _ _>>> >> -DCMAKECCOMPILER=$HOME/llvm-lkgc/bin/clang _ >>> >> $HOME/code/llvm-src >>> >> >>> >> But when I run any of the fuzzers, it looks like the sanitizer coverage >>> >> hasn't been set up correctly: >>> >> >>> >> % ./bin/llvm-as-fuzzer >>> 2017-08-24 11:14:33 >>> >> INFO: Seed: 4089166883 <(408)%20916-6883> >>> >> INFO: Loaded 1 modules (50607 guards): 50607 [0x10e14ef80, >>> 0x10e18063c), >>> >> INFO: Loaded 1 PC tables (0 PCs): 0 [0x10e2870a8,0x10e2870a8), >>> >> ERROR: The size of coverage PC tables does not match the number of >>> instrumented PCs. This might be a bug in the compiler, please contact the >>> libFuzzer developers. >>> >> >>> >> From the build logs, it looks like we're now building objects with >>> these >>> >> sanitizer flags: >>> >> >>> >> -fsanitize=address >>> >> -fsanitize-address-use-after-scope >>> >> -fsanitize=fuzzer-no-link >>> >> >>> >> We're then linking the fuzzer binaries with these: >>> >> >>> >> -fsanitize=address >>> >> -fsanitize-address-use-after-scope >>> >> -fsanitize=fuzzer-no-link >>> >> -fsanitize=fuzzer >>> >> >>> >> Any idea what's wrong or where to start looking? >>> >> >> >>

LLVM Developers mailing list llvm-dev at lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev -- -- Peter -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170824/92ca74bc/attachment.html>

More information about the llvm-dev mailing list