[RFC] Function type attribute to prevent CFI instrumentation (original) (raw)

Background

We are experimenting with CFI in Fuchsia’s kernel using the normal suite of CFI schemes. We’ve run into a couple of issues around function which can be grouped into one of these categories:

  1. We jump from one kernel module to another but the destination address isn’t a normal function pointer known in LTO’d code. CFI will trap during these calls because we’re effectively jumping to an arbitrary address outside the current module.
  2. We indirectly call functions from a previous kernel module in the current kernel module. CFI will also trap when calling these functions because they are defined outside the current module.
  3. We compare addresses of functions defined outside the LTO-unit. These comparisons fail because CFI unconditionally replaces direct references to each function with its corresponding entry in the CFI jump table, but there’s no guaranteed order between different jump table entries.
extern "C" void region_start();	// Defined outside the LTO unit.
extern "C" void region_end();	// Defined outside the LTO unit.
void check(uintptr_t addr) {
  assert((uintptr_t)region_start <= addr && addr < (uintptr_t)region_end);
}

None of these are indicative of actual bugs since these are instances where CFI can’t possibly know about these functions, but current workarounds for each of these aren’t very desirable.

Existing Solutions

[[no_sanitize(“cfi”)]] on functions that would invoke indirect calls

That is, we have a large function body that does a handful of indirect calls and we suppress them by disabling CFI for the whole function. This is not very desirable because it omits checks for places where we would want CFI checks. If we ever move around the code that would do indirect calls, we’d need to also double check that this attribute would be attached to the new functions.

-fno-sanitize-cfi-canonical-jump-tables
This flag ensures that references to functions defined inside the LTO-unit refer to the real function definition rather than the CFI jump table. However, this doesn’t do the reciprocal that we want for issue (3). That is, references in the LTO-unit to functions outside the LTO-unit still refer to the jump table.

function_nocfi
This is a macro used by Linux that expands to asm for directly getting the address of a function. It works and addresses issue (3) but isn’t very clean and requires an implementation for each unique arch. Linux maintainers weren’t very interested in this solution either.

__builtin_function_start
This was originally added for the ClangBuiltLinux project as an alternative to function_nocfi. Wrapping a direct reference to a function with this ensures we can get the address of the function definition rather than the jump table entry. This is a cleaner approach that helps address issue (3) but doesn’t address issues (1) or (2).

Wrapper Class
This would involve having something akin to std::function that would just have [[no_sanitize(“cfi”)]] on its Call method. This addresses issues (1) and (2) but it would be nicer if there was a compiler-based approach that would do this rather than having a downstream solution used by one project.

Proposed Attribute + Semantics

We would like to propose a new attribute called no_cfi that is only applied to function types. The new attribute would have the following semantics:

  1. Indirect calls to a function type with this attribute will not be instrumented with CFI. That is, the indirect call will not be checked. Note that this only changes the behavior for indirect calls on pointers to function types having this attribute. It does not prevent all indirect function calls for a given type from being checked.
  2. All direct references to a function whose type has this attribute will always reference the true function definition rather than an entry in the CFI jump table.
  3. When a pointer to a function with this attribute is implicitly cast to a pointer to a function without this attribute, the compiler will give a warning saying this attribute is discarded. This warning can be silenced with an explicit C-style cast or C++ static_cast.

Having an attribute should allow us to only make changes on appropriate function signatures and function pointer types.

Example Usage

#define NO_CFI __attribute__((no_cfi))

struct Args {
  void (NO_CFI *no_cfi_func)(); 
};

// Module 1
void local_func() {}

// CFI is still enabled for the whole function but the `handoff` invocation is not checked.
void do_handoff(void (NO_CFI *unchecked_handoff)(struct Args *)) {
  Args args = {&local_func};
  ...
  unchecked_handoff(&args);
}

// Module 2 - We came here from the call to `unchecked_handoff`.
void handoff_entry(struct Args *args) {
  args->no_cfi_func();  // No CFI check here. We can safely call `local_func` from Module 1.
  ...
  void (*func)() = args->no_cfi_func;  // warning: Cast discards `no_cfi` attribute.
}

extern "C" void NO_CFI region_start();	// Defined outside the LTO unit.
extern "C" void NO_CFI region_end();	// Defined outside the LTO unit.
void check(uintptr_t addr) {
  // References to these functions are not instrumented and refer to the real function definition.
  assert((uintptr_t)region_start <= addr && addr < (uintptr_t)region_end);
}

Implementation

I think implementation should be straightforward. On the AST-level, we track any indirect calls to a function whose type has this attribute and simply ignore CFI instrumentation at that callsite. Inside CodeGenFunction::EmitCall, we do a check against the CallExpr function type. There’s a block that emits the type test on an indirect call. We essentially just don’t enter this block if the CallExpr type has this attribute. No unique IR emission should be needed here and all future passes which would depend on this intrinsic will ignore this callsite. If we ever get the address of a symbol (via a DeclRefExpr) and that symbol is a function type with this attribute, we can wrap it with a no_cfi constant, then LowerTypeTestsModule::replaceCfiUses will ignore this reference.

This is a great idea! I even had a thought of doing something similar myself.

For my use-case, this would solve the problem of having no way to get the address of the real implementation of a function.

Have you tried -fsanitize-cfi-cross-dso?

We’ve considered it but haven’t experimented with it since it would need a custom runtime for the kernel which I believe would be a bit more complex with respect to tracking shadow and there’s a couple other relative-vtables changes we’d need to land first to start experimenting. For our case I believe we know the small subset of functions that should be allowed to cross module boundaries and ideally we’d just allow those to pass rather than add an extra cross-module check for every callsite. I think the cross DSO implementation also doesn’t prevent function references from referring to the jump table rather than the actual definition.

In general this seems reasonable even as type attribute (which I often try to convince folks away from: though this would ALSO work as a declaration attribute, which has easier implement-ability, at the expense of defined type conversion rules), though I think we should be pretty strict about the conversion rules rather than letting a warning be load-bearing.

Typically, when modifying a type via attribute, it makes sense for ONE direction to be implicit (that is, widening conversion), and one to be explicit-only (that is, narrowing conversion). At this point, I’m not sure WHICH direction is widening/which is narrowing (though frankly, I’ve convinced myself over the last 10 minutes of BOTH directions being narrowing…), and thus requiring explicit conversions.

Another thing to be aware of, if we permit this in C++ mode, there are a lot of other things that end up being concerning that we should pay attention to. For example, lambda-call-operators (and for captureless lambdas, the implicit fptr conversion operator, AND if captureless/implicit conversion operator, the ones that mix with calling conventions, etc). There are some other challenges around some of the ‘special’ member functions that I haven’t though through, but exposing this in C++ has particular challenges (many of which are alleviated by prohibiting this on member functions/conversions from member functions, but special operator function types are also interesting).

ALSO ALSO (and hopefully finally) this has some interactions with the ‘struct redefinition’ feature(N3037, PR #132939) that we’re currently implementing for C23 that we should make sure we are aware of, and handle properly.

rnk April 30, 2025, 10:48pm 7

I was reviewing RFCs ahead of the clang area team call, and I don’t see any blocking design concerns, so I would say you can carry on with the code review.