Allocation Tokens — Clang 22.0.0git documentation (original) (raw)

• Introduction
• Token Assignment Mode
• Querying Token IDs with __builtin_infer_alloc_token
• Allocation Token Instrumentation
  • Runtime Interface
  • Fast ABI
  • Instrumenting Non-Standard Allocation Functions
  • Disabling Instrumentation
  • Suppressions File (Ignorelist)
  • Conditional Compilation with __SANITIZE_ALLOC_TOKEN__

Introduction¶

Clang provides support for allocation tokens to enable allocator-level heap organization strategies. Clang assigns mode-dependent token IDs to allocation calls; the runtime behavior depends entirely on the implementation of a compatible memory allocator.

Possible allocator strategies include:

• Security Hardening: Placing allocations into separate, isolated heap partitions. For example, separating pointer-containing types from raw data can mitigate exploits that rely on overflowing a primitive buffer to corrupt object metadata.
• Memory Layout Optimization: Grouping related allocations to improve data locality and cache utilization.
• Custom Allocation Policies: Applying different management strategies to different partitions.

Token Assignment Mode¶

The default mode to calculate tokens is:

• typehashpointersplit: This mode assigns a token ID based on the hash of the allocated type’s name, where the top half ID-space is reserved for types that contain pointers and the bottom half for types that do not contain pointers.

Other token ID assignment modes are supported, but they may be subject to change or removal. These may (experimentally) be selected with -Xclang -falloc-token-mode=<mode>:

• typehash: This mode assigns a token ID based on the hash of the allocated type’s name.
• random: This mode assigns a statically-determined random token ID to each allocation site.
• increment: This mode assigns a simple, incrementally increasing token ID to each allocation site.

The following command-line options affect generated token IDs:

• -falloc-token-max=<N>
  Configures the maximum number of token IDs. By default the number of tokens is bounded by SIZE_MAX.

Querying Token IDs with __builtin_infer_alloc_token¶

For use cases where the token ID must be known at compile time, Clang provides a builtin function:

size_t __builtin_infer_alloc_token(, ...);

This builtin returns the token ID inferred from its argument expressions, which mirror arguments normally passed to any allocation function. The argument expressions are unevaluated, so it can be used with expressions that would have side effects without any runtime impact.

For example, it can be used as follows:

struct MyType { ... }; void *__partition_alloc(size_t size, size_t partition); #define partition_alloc(...) __partition_alloc(VA_ARGS, __builtin_infer_alloc_token(VA_ARGS))

void foo(void) { MyType *x = partition_alloc(sizeof(*x)); }

Allocation Token Instrumentation¶

To enable instrumentation of allocation functions, code can be compiled with the -fsanitize=alloc-token flag:

% clang++ -fsanitize=alloc-token example.cc

The instrumentation transforms allocation calls to include a token ID. For example:

// Original: ptr = malloc(size);

// Instrumented: ptr = __alloc_token_malloc(size, );

Runtime Interface¶

A compatible runtime must be provided that implements the token-enabled allocation functions. The instrumentation generates calls to functions that take a final size_t token_id argument.

// C standard library functions void *__alloc_token_malloc(size_t size, size_t token_id); void *__alloc_token_calloc(size_t count, size_t size, size_t token_id); void *__alloc_token_realloc(void *ptr, size_t size, size_t token_id); // ...

// C++ operators (mangled names) // operator new(size_t, size_t) void *__alloc_token__Znwm(size_t size, size_t token_id); // operator new[](size_t, size_t) void *__alloc_token__Znam(size_t size, size_t token_id); // ... other variants like nothrow, etc., are also instrumented.

Fast ABI¶

An alternative ABI can be enabled with -fsanitize-alloc-token-fast-abi, which encodes the token ID in the allocation function name.

void *__alloc_token_0_malloc(size_t size); void *__alloc_token_1_malloc(size_t size); void *__alloc_token_2_malloc(size_t size); ... void *__alloc_token_0_Znwm(size_t size); void *__alloc_token_1_Znwm(size_t size); void *__alloc_token_2_Znwm(size_t size); ...

This ABI provides a more efficient alternative where-falloc-token-max is small.

Instrumenting Non-Standard Allocation Functions¶

By default, AllocToken only instruments standard library allocation functions. This simplifies adoption, as a compatible allocator only needs to provide token-enabled variants for a well-defined set of standard functions.

To extend instrumentation to custom allocation functions, enable broader coverage with -fsanitize-alloc-token-extended. Such functions require being marked with the malloc or alloc_size attributes (or a combination).

For example:

void *custom_malloc(size_t size) attribute((malloc)); void *my_malloc(size_t size) attribute((alloc_size(1)));

// Original: ptr1 = custom_malloc(size); ptr2 = my_malloc(size);

// Instrumented: ptr1 = __alloc_token_custom_malloc(size, token_id); ptr2 = __alloc_token_my_malloc(size, token_id);

Disabling Instrumentation¶

To exclude specific functions from instrumentation, you can use theno_sanitize("alloc-token") attribute:

attribute((no_sanitize("alloc-token"))) void* custom_allocator(size_t size) { return malloc(size); // Uses original malloc }

Note: Independent of any given allocator support, the instrumentation aims to remain performance neutral. As such, no_sanitize("alloc-token")functions may be inlined into instrumented functions and vice-versa. If correctness is affected, such functions should explicitly be markednoinline.

The __attribute__((disable_sanitizer_instrumentation)) is also supported to disable this and other sanitizer instrumentations.

Suppressions File (Ignorelist)¶

AllocToken respects the src and fun entity types in theSanitizer special case list, which can be used to omit specified source files or functions from instrumentation.

[alloc-token]

Exclude specific source files

src:third_party/allocator.c

Exclude function name patterns

fun:custom_malloc fun:LowLevel::*

% clang++ -fsanitize=alloc-token -fsanitize-ignorelist=my_ignorelist.txt example.cc

Conditional Compilation with __SANITIZE_ALLOC_TOKEN__¶

In some cases, one may need to execute different code depending on whether AllocToken instrumentation is enabled. The __SANITIZE_ALLOC_TOKEN__ macro can be used for this purpose.

#ifdef SANITIZE_ALLOC_TOKEN // Code specific to -fsanitize=alloc-token builds #endif