3. Using and understanding the Valgrind core: Advanced Topics (original) (raw)

3. Using and understanding the Valgrind core: Advanced Topics

This chapter describes advanced aspects of the Valgrind core services, which are mostly of interest to power users who wish to customise and modify Valgrind's default behaviours in certain useful ways. The subjects covered are:

3.1. The Client Request mechanism

Valgrind has a trapdoor mechanism via which the client program can pass all manner of requests and queries to Valgrind and the current tool. Internally, this is used extensively to make various things work, although that's not visible from the outside.

For your convenience, a subset of these so-called client requests is provided to allow you to tell Valgrind facts about the behaviour of your program, and also to make queries. In particular, your program can tell Valgrind about things that it otherwise would not know, leading to better results.

Clients need to include a header file to make this work. Which header file depends on which client requests you use. Some client requests are handled by the core, and are defined in the header file valgrind/valgrind.h. Tool-specific header files are named after the tool, e.g.valgrind/memcheck.h. Each tool-specific header file includes valgrind/valgrind.h so you don't need to include it in your client if you include a tool-specific header. All header files can be found in the include/valgrind directory of wherever Valgrind was installed.

The macros in these header files have the magical property that they generate code in-line which Valgrind can spot. However, the code does nothing when not run on Valgrind, so you are not forced to run your program under Valgrind just because you use the macros in this file. Also, you are not required to link your program with any extra supporting libraries.

The code added to your binary has negligible performance impact: on x86, amd64, ppc32, ppc64 and ARM, the overhead is 6 simple integer instructions and is probably undetectable except in tight loops. However, if you really wish to compile out the client requests, you can compile with -DNVALGRIND (analogous to-DNDEBUG's effect onassert).

You are encouraged to copy the valgrind/*.h headers into your project's include directory, so your program doesn't have a compile-time dependency on Valgrind being installed. The Valgrind headers, unlike most of the rest of the code, are under a BSD-style license so you may include them without worrying about license incompatibility.

Here is a brief description of the macros available invalgrind.h, which work with more than one tool (see the tool-specific documentation for explanations of the tool-specific macros).

RUNNING_ON_VALGRIND:

Returns 1 if running on Valgrind, 0 if running on the real CPU. If you are running Valgrind on itself, returns the number of layers of Valgrind emulation you're running on.

VALGRIND_DISCARD_TRANSLATIONS:

Discards translations of code in the specified address range. Useful if you are debugging a JIT compiler or some other dynamic code generation system. After this call, attempts to execute code in the invalidated address range will cause Valgrind to make new translations of that code, which is probably the semantics you want. Note that code invalidations are expensive because finding all the relevant translations quickly is very difficult, so try not to call it often. Note that you can be clever about this: you only need to call it when an area which previously contained code is overwritten with new code. You can choose to write code into fresh memory, and just call this occasionally to discard large chunks of old code all at once.

Alternatively, for transparent self-modifying-code support, use--smc-check=all, or run on ppc32/Linux, ppc64/Linux or ARM/Linux.

VALGRIND_COUNT_ERRORS:

Returns the number of errors found so far by Valgrind. Can be useful in test harness code when combined with the--log-fd=-1 option; this runs Valgrind silently, but the client program can detect when errors occur. Only useful for tools that report errors, e.g. it's useful for Memcheck, but for Cachegrind it will always return zero because Cachegrind doesn't report errors.

VALGRIND_MALLOCLIKE_BLOCK:

If your program manages its own memory instead of using the standard malloc /new /new[], tools that track information about heap blocks will not do nearly as good a job. For example, Memcheck won't detect nearly as many errors, and the error messages won't be as informative. To improve this situation, use this macro just after your custom allocator allocates some new memory. See the comments invalgrind.h for information on how to use it.

VALGRIND_FREELIKE_BLOCK:

This should be used in conjunction withVALGRIND_MALLOCLIKE_BLOCK. Again, see valgrind.h for information on how to use it.

VALGRIND_RESIZEINPLACE_BLOCK:

Informs a Valgrind tool that the size of an allocated block has been modified but not its address. See valgrind.h for more information on how to use it.

VALGRIND_CREATE_MEMPOOL,VALGRIND_DESTROY_MEMPOOL,VALGRIND_MEMPOOL_ALLOC,VALGRIND_MEMPOOL_FREE,VALGRIND_MOVE_MEMPOOL,VALGRIND_MEMPOOL_CHANGE,VALGRIND_MEMPOOL_EXISTS:

These are similar to VALGRIND_MALLOCLIKE_BLOCK andVALGRIND_FREELIKE_BLOCK but are tailored towards code that uses memory pools. See Memory Pools for a detailed description.

VALGRIND_NON_SIMD_CALL[0123]:

Executes a function in the client program on the_real_ CPU, not the virtual CPU that Valgrind normally runs code on. The function must take an integer (holding a thread ID) as the first argument and then 0, 1, 2 or 3 more arguments (depending on which client request is used). These are used in various ways internally to Valgrind. They might be useful to client programs.

Warning: Only use these if you_really_ know what you are doing. They aren't entirely reliable, and can cause Valgrind to crash. Seevalgrind.h for more details.

VALGRIND_PRINTF(format, ...):

Print a printf-style message to the Valgrind log file. The message is prefixed with the PID between a pair of** markers. (Like all client requests, nothing is output if the client program is not running under Valgrind.) Output is not produced until a newline is encountered, or subsequent Valgrind output is printed; this allows you to build up a single line of output over multiple calls. Returns the number of characters output, excluding the PID prefix.

VALGRIND_PRINTF_BACKTRACE(format, ...):

Like VALGRIND_PRINTF (in particular, the return value is identical), but prints a stack backtrace immediately afterwards.

VALGRIND_MONITOR_COMMAND(command):

Execute the given monitor command (a string). Returns 0 if command is recognised. Returns 1 if command is not recognised. Note that some monitor commands provide access to a functionality also accessible via a specific client request. For example, memcheck leak search can be requested from the client program using VALGRIND_DO_LEAK_CHECK or via the monitor command "leak_search". Note that the syntax of the command string is only verified at run-time. So, if it exists, it is preferable to use a specific client request to have better compile time verifications of the arguments.

VALGRIND_CLO_CHANGE(option):

Changes the value of a dynamically changeable option (a string). See Dynamically Change Options.

VALGRIND_STACK_REGISTER(start, end):

Registers a new stack. Informs Valgrind that the memory range between start and end is a unique stack. Returns a stack identifier that can be used with otherVALGRIND_STACK_* calls.

Valgrind will use this information to determine if a change to the stack pointer is an item pushed onto the stack or a change over to a new stack. Use this if you're using a user-level thread package and are noticing crashes in stack trace recording or spurious errors from Valgrind about uninitialized memory reads.

Warning: Unfortunately, this client request is unreliable and best avoided.

VALGRIND_STACK_DEREGISTER(id):

Deregisters a previously registered stack. Informs Valgrind that previously registered memory range with stack idid is no longer a stack.

Warning: Unfortunately, this client request is unreliable and best avoided.

VALGRIND_STACK_CHANGE(id, start, end):

Changes a previously registered stack. Informs Valgrind that the previously registered stack with stack idid has changed its start and end values. Use this if your user-level thread package implements stack growth.

Warning: Unfortunately, this client request is unreliable and best avoided.

3.2. Debugging your program using Valgrind gdbserver and GDB

A program running under Valgrind is not executed directly by the CPU. Instead it runs on a synthetic CPU provided by Valgrind. This is why a debugger cannot natively debug your program when it runs on Valgrind.

This section describes how GDB can interact with the Valgrind gdbserver to provide a fully debuggable program under Valgrind. Used in this way, GDB also provides an interactive usage of Valgrind core or tool functionalities, including incremental leak search under Memcheck and on-demand Massif snapshot production.

3.2.1. Quick Start: debugging in 3 steps

The simplest way to get started is to run Valgrind with the flag --vgdb-error=0. Then follow the on-screen directions, which give you the precise commands needed to start GDB and connect it to your program.

Otherwise, here's a slightly more verbose overview.

If you want to debug a program with GDB when using the Memcheck tool, start Valgrind like this:

valgrind --vgdb=yes --vgdb-error=0 prog

In another shell, start GDB:

gdb prog

Then give the following command to GDB:

(gdb) target remote | vgdb

You can now debug your program e.g. by inserting a breakpoint and then using the GDB continuecommand.

This quick start information is enough for basic usage of the Valgrind gdbserver. The sections below describe more advanced functionality provided by the combination of Valgrind and GDB. Note that the command line flag --vgdb=yes can be omitted, as this is the default value.

3.2.2. Valgrind gdbserver overall organisation

The GNU GDB debugger is typically used to debug a process running on the same machine. In this mode, GDB uses system calls to control and query the program being debugged. This works well, but only allows GDB to debug a program running on the same computer.

GDB can also debug processes running on a different computer. To achieve this, GDB defines a protocol (that is, a set of query and reply packets) that facilitates fetching the value of memory or registers, setting breakpoints, etc. A gdbserver is an implementation of this "GDB remote debugging" protocol. To debug a process running on a remote computer, a gdbserver (sometimes called a GDB stub) must run at the remote computer side.

The Valgrind core provides a built-in gdbserver implementation, which is activated using --vgdb=yesor --vgdb=full. This gdbserver allows the process running on Valgrind's synthetic CPU to be debugged remotely. GDB sends protocol query packets (such as "get register contents") to the Valgrind embedded gdbserver. The gdbserver executes the queries (for example, it will get the register values of the synthetic CPU) and gives the results back to GDB.

GDB can use various kinds of channels (TCP/IP, serial line, etc) to communicate with the gdbserver. In the case of Valgrind's gdbserver, communication is done via a pipe and a small helper program called vgdb, which acts as an intermediary. If no GDB is in use, vgdb can also be used to send monitor commands to the Valgrind gdbserver from a shell command line.

3.2.3. Connecting GDB to a Valgrind gdbserver

To debug a program "prog" running under Valgrind, you must ensure that the Valgrind gdbserver is activated by specifying either --vgdb=yesor --vgdb=full. A secondary command line option,--vgdb-error=number, can be used to tell the gdbserver only to become active once the specified number of errors have been shown. A value of zero will therefore cause the gdbserver to become active at startup, which allows you to insert breakpoints before starting the run. For example:

valgrind --tool=memcheck --vgdb=yes --vgdb-error=0 ./prog

The Valgrind gdbserver is invoked at startup and indicates it is waiting for a connection from a GDB:

==2418== Memcheck, a memory error detector ==2418== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al. ==2418== Using Valgrind-3.14.0.GIT and LibVEX; rerun with -h for copyright info ==2418== Command: ./prog ==2418== ==2418== (action at startup) vgdb me ...

GDB (in another shell) can then be connected to the Valgrind gdbserver. For this, GDB must be started on the program prog:

gdb ./prog

You then indicate to GDB that you want to debug a remote target:

(gdb) target remote | vgdb

GDB then starts a vgdb relay application to communicate with the Valgrind embedded gdbserver:

(gdb) target remote | vgdb Remote debugging using | vgdb relaying data between gdb and process 2418 Reading symbols from /lib/ld-linux.so.2...done. Reading symbols from /usr/lib/debug/lib/ld-2.11.2.so.debug...done. Loaded symbols for /lib/ld-linux.so.2 [Switching to Thread 2418] 0x001f2850 in _start () from /lib/ld-linux.so.2 (gdb)

Note that vgdb is provided as part of the Valgrind distribution. You do not need to install it separately.

If vgdb detects that there are multiple Valgrind gdbservers that can be connected to, it will list all such servers and their PIDs, and then exit. You can then reissue the GDB "target" command, but specifying the PID of the process you want to debug:

(gdb) target remote | vgdb Remote debugging using | vgdb no --pid= arg given and multiple valgrind pids found: use --pid=2479 for valgrind --tool=memcheck --vgdb=yes --vgdb-error=0 ./prog use --pid=2481 for valgrind --tool=memcheck --vgdb=yes --vgdb-error=0 ./prog use --pid=2483 for valgrind --vgdb=yes --vgdb-error=0 ./another_prog Remote communication error: Resource temporarily unavailable. (gdb) target remote | vgdb --pid=2479 Remote debugging using | vgdb --pid=2479 relaying data between gdb and process 2479 Reading symbols from /lib/ld-linux.so.2...done. Reading symbols from /usr/lib/debug/lib/ld-2.11.2.so.debug...done. Loaded symbols for /lib/ld-linux.so.2 [Switching to Thread 2479] 0x001f2850 in _start () from /lib/ld-linux.so.2 (gdb)

If you want to use the --multi mode which makes vgdb start in extended-remote mode, set the following in GDB:

gdb prog

(gdb) set remote exec-file prog (gdb) set sysroot / (gdb) target extended-remote | vgdb --multi --vargs -q (gdb) start Temporary breakpoint 1 at 0x24e0 Starting program: prog relaying data between gdb and process 2999348

Temporary breakpoint 1, 0x000000000010a4a0 in main () (gdb)

Note that in --multi mode you don't have to start valgrind separately. vgdb will start valgrind for you. vgdb --multi mode is experimental and currently has some limitations like not being able to see program stdin and stdout. Also you have to explicitly set the remote exec-file and sysroot to tell GDB the "remote" and local files are the same.

Once GDB is connected to the Valgrind gdbserver, it can be used in the same way as if you were debugging the program natively:

And so on. Refer to the GDB user manual for a complete description of GDB's functionality.

3.2.4. Connecting to an Android gdbserver

When developping applications for Android, you will typically use a development system (on which the Android NDK is installed) to compile your application. An Android target system or emulator will be used to run the application. In this setup, Valgrind and vgdb will run on the Android system, while GDB will run on the development system. GDB will connect to the vgdb running on the Android system using the Android NDK 'adb forward' application.

Example: on the Android system, execute the following:

valgrind --vgdb-error=0 --vgdb=yes prog

and then in another shell, run:

vgdb --port=1234

On the development system, execute the following commands:

adb forward tcp:1234 tcp:1234 gdb prog (gdb) target remote :1234

GDB will use a local tcp/ip connection to connect to the Android adb forwarder. Adb will establish a relay connection between the host system and the Android target system. Be sure to use the GDB delivered in the Android NDK system (typically, arm-linux-androideabi-gdb), as the host GDB is probably not able to debug Android arm applications. Note that the local port nr (used by GDB) must not necessarily be equal to the port number used by vgdb: adb can forward tcp/ip between different port numbers.

In the current release, the GDB server is not enabled by default for Android, due to problems in establishing a suitable directory in which Valgrind can create the necessary FIFOs (named pipes) for communication purposes. You can stil try to use the GDB server, but you will need to explicitly enable it using the flag --vgdb=yes or--vgdb=full.

Additionally, you will need to select a temporary directory which is (a) writable by Valgrind, and (b) supports FIFOs. This is the main difficult point. Often, /sdcard satisfies requirement (a), but fails for (b) because it is a VFAT file system and VFAT does not support pipes. Possibilities you could try are/data/local,/data/local/Inst (if you installed Valgrind there), or/data/data/name.of.my.app, if you are running a specific application and it has its own directory of that form. This last possibility may have the highest probability of success.

You can specify the temporary directory to use either via the --with-tmpdir= configure time flag, or by setting environment variable TMPDIR when running Valgrind (on the Android device, not on the Android NDK development host). Another alternative is to specify the directory for the FIFOs using the --vgdb-prefix= Valgrind command line option.

We hope to have a better story for temporary directory handling on Android in the future. The difficulty is that, unlike in standard Unixes, there is no single temporary file directory that reliably works across all devices and scenarios.

3.2.5. Monitor command handling by the Valgrind gdbserver

The Valgrind gdbserver provides additional Valgrind-specific functionality via "monitor commands". Such monitor commands can be sent from the GDB command line or from the shell command line or requested by the client program using the VALGRIND_MONITOR_COMMAND client request. SeeValgrind monitor commands for the list of the Valgrind core monitor commands available regardless of the Valgrind tool selected.

The following tools provide tool-specific monitor commands:

An example of a tool specific monitor command is the Memcheck monitor command leak_check full reachable any. This requests a full reporting of the allocated memory blocks. To have this leak check executed, use the GDB command:

(gdb) monitor leak_check full reachable any

GDB sends as a single string all what follows 'monitor' to the Valgrind gdbserver. The Valgrind gdbserver parses the string and will execute the monitor command itself, if it recognises it to be a Valgrind core monitor command. If it is not recognised as such, it is assumed to be tool-specific and is handed to the tool for execution. For example:

(gdb) monitor leak_check full reachable any ==2418== 100 bytes in 1 blocks are still reachable in loss record 1 of 1 ==2418== at 0x4006E9E: malloc (vg_replace_malloc.c:236) ==2418== by 0x804884F: main (prog.c:88) ==2418== ==2418== LEAK SUMMARY: ==2418== definitely lost: 0 bytes in 0 blocks ==2418== indirectly lost: 0 bytes in 0 blocks ==2418== possibly lost: 0 bytes in 0 blocks ==2418== still reachable: 100 bytes in 1 blocks ==2418== suppressed: 0 bytes in 0 blocks ==2418== (gdb)

Similarly to GDB, the Valgrind gdbserver will accept abbreviated monitor command names and arguments, as long as the given abbreviation is unambiguous. For example, the aboveleak_checkcommand can also be typed as:

(gdb) mo l f r a

The letters mo are recognised by GDB as being an abbreviation for monitor. So GDB sends the string l f r a to the Valgrind gdbserver. The letters provided in this string are unambiguous for the Valgrind gdbserver. This therefore gives the same output as the unabbreviated command and arguments. If the provided abbreviation is ambiguous, the Valgrind gdbserver will report the list of commands (or argument values) that can match:

(gdb) mo v. n v. can match v.set v.info v.wait v.kill v.translate v.do (gdb) mo v.i n n_errs_found 0 n_errs_shown 0 (vgdb-error 0) (gdb)

Instead of sending a monitor command from GDB, you can also send these from a shell command line. For example, the following command lines, when given in a shell, will cause the same leak search to be executed by the process 3145:

vgdb --pid=3145 leak_check full reachable any vgdb --pid=3145 l f r a

Note that the Valgrind gdbserver automatically continues the execution of the program after a standalone invocation of vgdb. Monitor commands sent from GDB do not cause the program to continue: the program execution is controlled explicitly using GDB commands such as "continue" or "next".

Many monitor commands (e.g. v.info location, memcheck who_points_at, ...) require an address argument and an optional length: <addr> [<len>]. The arguments can also be provided by using a 'C array like syntax' by providing the address followed by the length between square brackets.

For example, the following two monitor commands provide the same information:

(gdb) mo xb 0x804a2f0 10 ... (gdb) mo xb 0x804a2f0[10] ...

3.2.6. GDB front end commands for Valgrind gdbserver monitor commands

As explained inMonitor command handling by the Valgrind gdbserver, valgrind monitor commands consist in strings that are not interpreted by GDB. GDB has no knowledge of these valgrind monitor commands. The GDB 'command line interface' infrastructure however provides interesting functionalities to help typing commands such as auto-completion, command specific help, searching for a command or command help matching a regexp, ...

To have a better integration of the valgrind monitor commands in the GDB command line interface, Valgrind provides python code defining a GDB front end command for each valgrind monitor command. Similarly, for each tool specific monitor command, the python code provides a matching GDB front end command.

Like other GDB commands, the GDB front end Valgrind monitor commands are hierarchically structured starting from 5 "top" GDB commands. Subcommands are defined below these "top" commands. To ease typing, shorter aliases are also provided.

The usage of a GDB front end command is compatible with a direct usage of the Valgrind monitor command. The below example shows a direct usage of the Memcheck monitor command xb to examine the definedness status of the some_mem array and equivalent usages based on the GDB front end commands.

(gdb) list 1 int main() 2 { 3 char some_mem[5]; 4 return 0; 5 } (gdb) p &some_mem $2 = (char (*)[5]) 0x1ffefffe5b (gdb) p sizeof(some_mem) $3 = 5 (gdb) monitor xb 0x1ffefffe5b 5 ff ff ff ff ff 0x1FFEFFFE5B: 0x00 0x00 0x00 0x00 0x00 (gdb) memcheck xb 0x1ffefffe5b 5 ff ff ff ff ff 0x1FFEFFFE5B: 0x00 0x00 0x00 0x00 0x00 (gdb) mc xb &some_mem sizeof(some_mem) ff ff ff ff ff 0x1FFEFFFE5B: 0x00 0x00 0x00 0x00 0x00 (gdb)

It is worth noting down that the third command uses the alias mc. This command also shows a significant advantage of using the GDB front end commands: as GDB "understands" the structure of these front end commands, where relevant, these front end commands will evaluate their arguments. In the case of the xb command, the GDB xb command evaluates its second argument (which must be an address expression) and its optional second argument (which must be an integer expression).

GDB will auto-load the python code defining the Valgrind front end commands as soon as GDB detects that the executable being debugged is running under valgrind. This detection is based on observing that the Valgrind process has loaded a specific Valgrind shared library. The loading of this library is done by the dynamic loader very early on in the execution of the process. If GDB is used to connect to a Valgrind process that has not yet started its execution (such as when Valgrind was started with the option --vgdb-stop-at=startup or --vgdb-error=0), then the GDB front end commands will not yet be auto-loaded. To have the GDB front end commands auto-loaded, you can put a breakpoint e.g. in main and use the GDB command continue. Alternatively, you can add in your .gdbinit a line that loads the python code at GDB startup such as:

source /path/to/valgrind/python/code/valgrind-monitor.py

The exact path to use in the source command depends on your Valgrind installation. The output of the shell command vgdb --help contains the absolute path name for the python file you can source in your .gdbinit to define the GDB valgrind front end monitor commands.

3.2.7. Valgrind gdbserver thread information

Valgrind's gdbserver enriches the output of the GDB info threads command with Valgrind-specific information. The operating system's thread number is followed by Valgrind's internal index for that thread ("tid") and by the Valgrind scheduler thread state:

(gdb) info threads 4 Thread 6239 (tid 4 VgTs_Yielding) 0x001f2832 in _dl_sysinfo_int80 () from /lib/ld-linux.so.2

3.2.8. Examining and modifying Valgrind shadow registers

When the option --vgdb-shadow-registers=yes is given, the Valgrind gdbserver will let GDB examine and/or modify Valgrind's shadow registers. GDB version 7.1 or later is needed for this to work. For x86 and amd64, GDB version 7.2 or later is needed.

For each CPU register, the Valgrind core maintains two shadow register sets. These shadow registers can be accessed from GDB by giving a postfix s1or s2 for respectively the first and second shadow register. For example, the x86 registereax and its two shadows can be examined using the following commands:

(gdb) p $eax $1 = 0 (gdb) p $eaxs1 $2 = 0 (gdb) p $eaxs2 $3 = 0 (gdb)

Float shadow registers are shown by GDB as unsigned integer values instead of float values, as it is expected that these shadow values are mostly used for memcheck validity bits.

Intel/amd64 AVX registers ymm0to ymm15 have also their shadow registers. However, GDB presents the shadow values using two "half" registers. For example, the half shadow registers for ymm9 arexmm9s1 (lower half for set 1),ymm9hs1 (upper half for set 1),xmm9s2 (lower half for set 2),ymm9hs2 (upper half for set 2). Note the inconsistent notation for the names of the half registers: the lower part starts with an x, the upper part starts with an yand has an h before the shadow postfix.

The special presentation of the AVX shadow registers is due to the fact that GDB independently retrieves the lower and upper half of the ymm registers. GDB does not however know that the shadow half registers have to be shown combined.

3.2.9. Limitations of the Valgrind gdbserver

Debugging with the Valgrind gdbserver is very similar to native debugging. Valgrind's gdbserver implementation is quite complete, and so provides most of the GDB debugging functionality. There are however some limitations and peculiarities:

3.2.10. vgdb command line options

Usage: vgdb [OPTION]... [[-c] COMMAND]...

vgdb ("Valgrind to GDB") is a small program that is used as an intermediary between Valgrind and GDB or a shell. It has three usage modes:

  1. As a standalone utility, it is used from a shell command line to send monitor commands to a process running under Valgrind. For this usage, the vgdb OPTION(s) must be followed by the monitor command to send. To send more than one command, separate them with the -c option.
  2. In combination with GDB "target remote |" command, it is used as the relay application between GDB and the Valgrind gdbserver. For this usage, only OPTION(s) can be given, but no COMMAND can be given.
  3. In the --multi mode, vgdb uses the extended remote protocol to communicate with GDB. This allows you to view output from both valgrind and GDB in the GDB session. This is accomplished via the "target extended-remote | vgdb --multi". In this mode you no longer need to start valgrind yourself. vgdb will start up valgrind when gdb tells it to run a new program. For this usage, the vgdb OPTIONS(s) can also include --valgrind and --vargs to describe how valgrind should be started.

vgdb accepts the following options:

--pid=<number>

Specifies the PID of the process to which vgdb must connect to. This option is useful in case more than one Valgrind gdbserver can be connected to. If the --pid argument is not given and multiple Valgrind gdbserver processes are running, vgdb will report the list of such processes and then exit.

--vgdb-prefix

Must be given to both Valgrind and vgdb if you want to change the default prefix for the FIFOs (named pipes) used for communication between the Valgrind gdbserver and vgdb.

--wait=<number>

Instructs vgdb to search for available Valgrind gdbservers for the specified number of seconds. This makes it possible start a vgdb process before starting the Valgrind gdbserver with which you intend the vgdb to communicate. This option is useful when used in conjunction with a --vgdb-prefix that is unique to the process you want to wait for. Also, if you use the --wait argument in the GDB "target remote" command, you must set the GDB remotetimeout to a value bigger than the --wait argument value. See option--max-invoke-ms (just below) for an example of setting the remotetimeout value.

--max-invoke-ms=<number>

Gives the number of milliseconds after which vgdb will force the invocation of gdbserver embedded in Valgrind. The default value is 100 milliseconds. A value of 0 disables forced invocation. The forced invocation is used when vgdb is connected to a Valgrind gdbserver, and the Valgrind process has all its threads blocked in a system call.

If you specify a large value, you might need to increase the GDB "remotetimeout" value from its default value of 2 seconds. You should ensure that the timeout (in seconds) is bigger than the --max-invoke-ms value. For example, for --max-invoke-ms=5000, the following GDB command is suitable:

(gdb) set remotetimeout 6

--cmd-time-out=<number>

Instructs a standalone vgdb to exit if the Valgrind gdbserver it is connected to does not process a command in the specified number of seconds. The default value is to never time out.

--port=<portnr>

Instructs vgdb to use tcp/ip and listen for GDB on the specified port nr rather than to use a pipe to communicate with GDB. Using tcp/ip allows to have GDB running on one computer and debugging a Valgrind process running on another target computer. Example:

On the target computer, start your program under valgrind using

valgrind --vgdb-error=0 prog

and then in another shell, run:

vgdb --port=1234

On the computer which hosts GDB, execute the command:

gdb prog (gdb) target remote targetip:1234

where targetip is the ip address or hostname of the target computer.

--vgdb-multi

Makes vgdb start in extended-remote mode and to wait for gdb to tell us what to run.

--valgrind

The path to valgrind to use, in extended-remote mode. If not specified, the system valgrind will be launched.

--vargs

Options to run valgrind with, in extended-remote mode. For example -q. Everything following --vargs will be provided as arguments to valgrind as is.

-c

To give more than one command to a standalone vgdb, separate the commands by an option -c. Example:

vgdb v.set log_output -c leak_check any

-l

Instructs a standalone vgdb to report the list of the Valgrind gdbserver processes running and then exit.

-T

Instructs vgdb to add timestamps to vgdb information messages.

-D

Instructs a standalone vgdb to show the state of the shared memory used by the Valgrind gdbserver. vgdb will exit after having shown the Valgrind gdbserver shared memory state.

-d

Instructs vgdb to produce debugging output. Give multiple -d args to increase the verbosity. When giving -d to a relay vgdb, you better redirect the standard error (stderr) of vgdb to a file to avoid interaction between GDB and vgdb debugging output.

3.2.11. Valgrind monitor commands

This section describes the Valgrind monitor commands, available regardless of the Valgrind tool selected. For the tool specific commands, refer to Memcheck Monitor Commands,Helgrind Monitor Commands,Callgrind Monitor Commands andMassif Monitor Commands.

The monitor commands can be sent either from a shell command line, by using a standalone vgdb, or from GDB, by using GDB's "monitor" command (see Monitor command handling by the Valgrind gdbserver) or by GDB's "valgrind" front end commands (seeGDB front end commands for Valgrind gdbserver monitor commands). They can also be launched by the client program, using the VALGRIND_MONITOR_COMMAND client request.

Whatever the way the monitor command is launched, it will behave the same way. However, using the GDB's valgrind front end commands allows to benefit from the GDB infrastructure, such as expression evaluation. When relevant, the description of a monitor command below describes the additional flexibility provided by the GDB valgrind front end command. To launch a valgrind monitor command via its GDB front end command, instead of prefixing the command with "monitor", you must use the GDB valgrind command (or the shorter aliases vg or v). In GDB, you can use help valgrind to get help about the valgrind front end monitor commands and you can use apropos valgrind to get all the commands mentionning the word "valgrind" in their name or on-line help.

The following Valgrind monitor commands are useful for investigating the behaviour of Valgrind or its gdbserver in case of problems or bugs.

3.3. Function wrapping

Valgrind allows calls to some specified functions to be intercepted and rerouted to a different, user-supplied function. This can do whatever it likes, typically examining the arguments, calling onwards to the original, and possibly examining the result. Any number of functions may be wrapped.

Function wrapping is useful for instrumenting an API in some way. For example, Helgrind wraps functions in the POSIX pthreads API so it can know about thread status changes, and the core is able to wrap functions in the MPI (message-passing) API so it can know of memory status changes associated with message arrival/departure. Such information is usually passed to Valgrind by using client requests in the wrapper functions, although the exact mechanism may vary.

3.3.1. A Simple Example

Supposing we want to wrap some function

int foo ( int x, int y ) { return x + y; }

A wrapper is a function of identical type, but with a special name which identifies it as the wrapper for foo. Wrappers need to include supporting macros from valgrind.h. Here is a simple wrapper which prints the arguments and return value:

#include <stdio.h> #include "valgrind.h" int I_WRAP_SONAME_FNNAME_ZU(NONE,foo)( int x, int y ) { int result; OrigFn fn; VALGRIND_GET_ORIG_FN(fn); printf("foo's wrapper: args %d %d\n", x, y); CALL_FN_W_WW(result, fn, x,y); printf("foo's wrapper: result %d\n", result); return result; }

To become active, the wrapper merely needs to be present in a text section somewhere in the same process' address space as the function it wraps, and for its ELF symbol name to be visible to Valgrind. In practice, this means either compiling to a .o and linking it in, or compiling to a .so and LD_PRELOADing it in. The latter is more convenient in that it doesn't require relinking.

All wrappers have approximately the above form. There are three crucial macros:

I_WRAP_SONAME_FNNAME_ZU: this generates the real name of the wrapper. This is an encoded name which Valgrind notices when reading symbol table information. What it says is: I am the wrapper for any function named foo which is found in an ELF shared object with an empty ("NONE") soname field. The specification mechanism is powerful in that wildcards are allowed for both sonames and function names. The details are discussed below.

VALGRIND_GET_ORIG_FN: once in the wrapper, the first priority is to get hold of the address of the original (and any other supporting information needed). This is stored in a value of opaque type OrigFn. The information is acquired using VALGRIND_GET_ORIG_FN. It is crucial to make this macro call before calling any other wrapped function in the same thread.

CALL_FN_W_WW: eventually we will want to call the function being wrapped. Calling it directly does not work, since that just gets us back to the wrapper and leads to an infinite loop. Instead, the result lvalue, OrigFn and arguments are handed to one of a family of macros of the form CALL_FN_*. These cause Valgrind to call the original and avoid recursion back to the wrapper.

3.3.2. Wrapping Specifications

This scheme has the advantage of being self-contained. A library of wrappers can be compiled to object code in the normal way, and does not rely on an external script telling Valgrind which wrappers pertain to which originals.

Each wrapper has a name which, in the most general case says: I am the wrapper for any function whose name matches FNPATT and whose ELF "soname" matches SOPATT. Both FNPATT and SOPATT may contain wildcards (asterisks) and other characters (spaces, dots, @, etc) which are not generally regarded as valid C identifier names.

This flexibility is needed to write robust wrappers for POSIX pthread functions, where typically we are not completely sure of either the function name or the soname, or alternatively we want to wrap a whole set of functions at once.

For example, pthread_createin GNU libpthread is usually a versioned symbol - one whose name ends in, eg, @GLIBC_2.3. Hence we are not sure what its real name is. We also want to cover any soname of the form libpthread.so*. So the header of the wrapper will be

int I_WRAP_SONAME_FNNAME_ZZ(libpthreadZdsoZd0,pthreadZucreateZAZa) ( ... formals ... ) { ... body ... }

In order to write unusual characters as valid C function names, a Z-encoding scheme is used. Names are written literally, except that a capital Z acts as an escape character, with the following encoding:

 Za   encodes    *
 Zp              +
 Zc              :
 Zd              .
 Zu              _
 Zh              -
 Zs              (space)
 ZA              @
 ZZ              Z
 ZL              (       # only in valgrind 3.3.0 and later
 ZR              )       # only in valgrind 3.3.0 and later

Hence libpthreadZdsoZd0 is an encoding of the soname libpthread.so.0and pthreadZucreateZAZa is an encoding of the function name pthread_create@*.

The macro I_WRAP_SONAME_FNNAME_ZZconstructs a wrapper name in which both the soname (first component) and function name (second component) are Z-encoded. Encoding the function name can be tiresome and is often unnecessary, so a second macro,I_WRAP_SONAME_FNNAME_ZU, can be used instead. The _ZU variant is also useful for writing wrappers for C++ functions, in which the function name is usually already mangled using some other convention in which Z plays an important role. Having to encode a second time quickly becomes confusing.

Since the function name field may contain wildcards, it can be anything, including just *. The same is true for the soname. However, some ELF objects - specifically, main executables - do not have sonames. Any object lacking a soname is treated as if its soname was NONE, which is why the original example above had a nameI_WRAP_SONAME_FNNAME_ZU(NONE,foo).

Note that the soname of an ELF object is not the same as its file name, although it is often similar. You can find the soname of an object libfoo.so using the commandreadelf -a libfoo.so | grep soname.

3.3.3. Wrapping Semantics

The ability for a wrapper to replace an infinite family of functions is powerful but brings complications in situations where ELF objects appear and disappear (are dlopen'd and dlclose'd) on the fly. Valgrind tries to maintain sensible behaviour in such situations.

For example, suppose a process has dlopened (an ELF object with soname) object1.so, which contains function1. It starts to usefunction1 immediately.

After a while it dlopens wrappers.so, which contains a wrapper for function1 in (soname) object1.so. All subsequent calls to function1 are rerouted to the wrapper.

If wrappers.so is later dlclose'd, calls to function1 are naturally routed back to the original.

Alternatively, if object1.sois dlclose'd but wrappers.so remains, then the wrapper exported by wrappers.sobecomes inactive, since there is no way to get to it - there is no original to call any more. However, Valgrind remembers that the wrapper is still present. If object1.so is eventually dlopen'd again, the wrapper will become active again.

In short, valgrind inspects all code loading/unloading events to ensure that the set of currently active wrappers remains consistent.

A second possible problem is that of conflicting wrappers. It is easily possible to load two or more wrappers, both of which claim to be wrappers for some third function. In such cases Valgrind will complain about conflicting wrappers when the second one appears, and will honour only the first one.

3.3.4. Debugging

Figuring out what's going on given the dynamic nature of wrapping can be difficult. The --trace-redir=yes option makes this possible by showing the complete state of the redirection subsystem after everymmap/munmapevent affecting code (text).

There are two central concepts:

The state of the wrapping-and-redirection subsystem comprises a set of specifications and a set of active bindings. The specifications are acquired/discarded by watching all mmap/munmapevents on code (text) sections. The active binding set is (conceptually) recomputed from the specifications, and all known symbol names, following any change to the specification set.

--trace-redir=yes shows the contents of both sets following any such event.

-v prints a line of text each time an active specification is used for the first time.

Hence for maximum debugging effectiveness you will need to use both options.

One final comment. The function-wrapping facility is closely tied to Valgrind's ability to replace (redirect) specified functions, for example to redirect calls to malloc to its own implementation. Indeed, a replacement function can be regarded as a wrapper function which does not call the original. However, to make the implementation more robust, the two kinds of interception (wrapping vs replacement) are treated differently.

--trace-redir=yes shows specifications and bindings for both replacement and wrapper functions. To differentiate the two, replacement bindings are printed using R-> whereas wraps are printed using W->.

3.3.5. Limitations - control flow

For the most part, the function wrapping implementation is robust. The only important caveat is: in a wrapper, get hold of the OrigFn information using VALGRIND_GET_ORIG_FN before calling any other wrapped function. Once you have the OrigFn, arbitrary calls between, recursion between, and longjumps out of wrappers should work correctly. There is never any interaction between wrapped functions and merely replaced functions (eg malloc), so you can callmalloc etc safely from within wrappers.

The above comments are true for {x86,amd64,ppc32,arm,mips32,s390}-linux. On ppc64-linux function wrapping is more fragile due to the (arguably poorly designed) ppc64-linux ABI. This mandates the use of a shadow stack which tracks entries/exits of both wrapper and replacement functions. This gives two limitations: firstly, longjumping out of wrappers will rapidly lead to disaster, since the shadow stack will not get correctly cleared. Secondly, since the shadow stack has finite size, recursion between wrapper/replacement functions is only possible to a limited depth, beyond which Valgrind has to abort the run. This depth is currently 16 calls.

For all platforms ({x86,amd64,ppc32,ppc64,arm,mips32,s390}-linux) all the above comments apply on a per-thread basis. In other words, wrapping is thread-safe: each thread must individually observe the above restrictions, but there is no need for any kind of inter-thread cooperation.

3.3.6. Limitations - original function signatures

As shown in the above example, to call the original you must use a macro of the form CALL_FN_*. For technical reasons it is impossible to create a single macro to deal with all argument types and numbers, so a family of macros covering the most common cases is supplied. In what follows, 'W' denotes a machine-word-typed value (a pointer or a C long), and 'v' denotes C's void type. The currently available macros are:

CALL_FN_v_v -- call an original of type void fn ( void ) CALL_FN_W_v -- call an original of type long fn ( void )

CALL_FN_v_W -- call an original of type void fn ( long ) CALL_FN_W_W -- call an original of type long fn ( long )

CALL_FN_v_WW -- call an original of type void fn ( long, long ) CALL_FN_W_WW -- call an original of type long fn ( long, long )

CALL_FN_v_WWW -- call an original of type void fn ( long, long, long ) CALL_FN_W_WWW -- call an original of type long fn ( long, long, long )

CALL_FN_W_WWWW -- call an original of type long fn ( long, long, long, long ) CALL_FN_W_5W -- call an original of type long fn ( long, long, long, long, long ) CALL_FN_W_6W -- call an original of type long fn ( long, long, long, long, long, long ) and so on, up to CALL_FN_W_12W

The set of supported types can be expanded as needed. It is regrettable that this limitation exists. Function wrapping has proven difficult to implement, with a certain apparently unavoidable level of ickiness. After several implementation attempts, the present arrangement appears to be the least-worst tradeoff. At least it works reliably in the presence of dynamic linking and dynamic code loading/unloading.

You should not attempt to wrap a function of one type signature with a wrapper of a different type signature. Such trickery will surely lead to crashes or strange behaviour. This is not a limitation of the function wrapping implementation, merely a reflection of the fact that it gives you sweeping powers to shoot yourself in the foot if you are not careful. Imagine the instant havoc you could wreak by writing a wrapper which matched any function name in any soname - in effect, one which claimed to be a wrapper for all functions in the process.

3.3.7. Examples

In the source tree, memcheck/tests/wrap[1-8].c provide a series of examples, ranging from very simple to quite advanced.

mpi/libmpiwrap.c is an example of wrapping a big, complex API (the MPI-2 interface). This file defines almost 300 different wrappers.