[llvm-dev] Using source-based code coverage on baremetal (original) (raw)

Peter Smith via llvm-dev llvm-dev at lists.llvm.org
Wed Sep 6 02:51:08 PDT 2017

Hello Eli,

I think that this would be very useful for bare-metal systems, particularly for those without trace capture or the software to analyze it. I seem to remember an internal proof of concept experiment to see if we could get profiling working in an embedded toolchain, which we did after making similar, but toolchain specific, linker/library modifications that you did.

I think the most valuable contribution would be in example documentation such as the steps you outline above. Moreover I think that more documentation and recipes for building and testing compiler-rt for bare-metal would be extremely valuable.

I'm not much of an expert on profiling, or compiler-rt but I'm willing to help out where I can.

My understanding is that COMPILER_RT_BAREMETAL_BUILD is indeed used to exclude objects from the library that would be incompatible with a bare-metal target. It looks like it was introduced in https://reviews.llvm.org/D33018.

I've not looked into how this works in compiler-rt/compiler right now so apologies if this is somewhat speculative. As I understand it, the section __start and __end symbols is a GNU ld convention. I think one way of baking this in would be to separate out functions that access the __start and __end symbols into getSectionStart(), getSectionEnd() functions (probably weak definitions) that are in their own .o file. If a bare-metal toolchain has a different set of linker magic/convention it can implement its own getSectionStart() and getSectionEnd(). On the assumption that most bare-metal toolchains have their own target and driver a flag sounds like a good idea.

Peter

On 6 September 2017 at 02:55, Friedman, Eli via llvm-dev <llvm-dev at lists.llvm.org> wrote:

Hi all,

I think using code coverage on baremetal has come up once or twice on llvmdev, but I don't think anyone has actually written up how the workflow works, or what issues come up. This description is based on work done together with my colleague Weiming Zhao. By "baremetal" here, I mean an embedded environment without an operating system. We specifically used a ARM target with semihosting, but similar steps should work elsewhere. The workflow: 1. First, you need a copy of libclangrtprofile.a, stripped down for the baremetal target. (More on this below.) 2. Then, you need to change the source code to call into it; since a baremetal image doesn't exit like an operating system process, you need to insert code somewhere to write out the profile data yourself. We used __llvmprofilegetsizeforbuffer() and llvmprofilewritebuf() for this (and semihosting APIs to transfer the resulting buffer to the host). 3. Then, you have to edit your linker script to include the necessary _sections. _llvmprfnames, _llvmprfdata, and llvmprfcnts need to be _included in the final image. And llvmcovmap needs to be in a non-allocatable section in the ELF output. And depending on how your linker behaves, you might need to explicitly mark all of these sections KEEP so parts don't get dropped. This is actually the trickiest part, in the sense that messing it up lead to obscure issues which are difficult to debug. At best, you get an error message like "No coverage data found" or "Malformed instrumentation profile data". Or, if you're using a build of LLVM more than a few months old, coverage data can be silently dropped. 4. Then, you add "-fprofile-instr-generate -fcoverage-mapping -mllvm -enable-value-profiling=false" to your CFLAGS. 5. Then, you build and run the image in the semihosted environment. If everything works, you get a file with raw profile data, and use the normal workflow to convert it into a report. Areas that required LLVM changes: 1. The copy of libclangrtprofile.a for the target. Given that we already were using builtins from compiler-rt, the primary changes required are enabling the profile library and excluding a bunch of files from the build (since baremetal doesn't have a filesystem, system calls, etc.). I'll look into posting patches when I have time, but it might take me a little while for me to figure out how to cleanly modify the build, and verify everything actually works on trunk. It looks like there's a CMake variable COMPILERRTBAREMETALBUILD which is supposed to be turned on for this sort of environment? _2. Changing the compiler and compiler-rt to use _start and end symbols to find the sections, rather than .init code. This isn't strictly necessary, _but our linker supports _start and end, and this was easier than changing the baremetal image to handle a .init section. See needsRuntimeRegistrationOfSectionRange in lib/Transforms/Instrumentation/InstrProfiling.cpp; we currently only whitelist a few platforms. Not sure what would be appropriate here; maybe _we could assume any -none- triple supports _start and end symbols? Or maybe control it with a flag somehow? Or something else I'm not thinking of? Other problem areas: 1. We turned value profiling off because we were running into runtime issues; specifically, we had infinite recursion because we instrumented malloc. It isn't really important to have in this context (coverage reports currently don't use the data at all), but is there some way to improve this workflow to involve fewer magic command-line flags? 2. The error messages produced by llvm-profdata and llvm-cov tools could probably be improved, to describe the actual issue in more detail. Next steps: The next steps here depend on community interest, I guess... has anyone else tried something like this? Is anyone interested in my patches? Should we add a section to the coverage documentation? -Eli -- Employee of Qualcomm Innovation Center, Inc. Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project

LLVM Developers mailing list llvm-dev at lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

More information about the llvm-dev mailing list