Multiple incompatible function declarations · Issue #198 · rust-lang/unsafe-code-guidelines (original) (raw)

Does the following example Rust code has undefined behavior ?

#[no_mangle] extern "C" fn foo(x: *mut i32) { ... } mod maybe_bad0 { extern "C" { fn foo(x: Option<&mut i32>); } // OK or UB ? } mod maybe_bad1 { extern "C" { fn foo(x: NonNull); } // OK or UB ? } mod maybe_bad2 { extern "C" { fn foo(x: *mut i32); } // OK or UB ? }

Note: the #[no_mangle] definition is used for exposition, and we can replace it
with the following C function definition in a TU that's linked with that Rust program:

void foo(int32_t* x) { ... }

There is a lot of Rust code using Rust types to "enrich" C APIs, the unsafe keyword does not appear anywhere in the example, and the equivalent C code would have undefined behavior: it is instant undefined behavior in C to provide a function declaration whose types do not "properly" match the function definition - LLVM uses this information to optimize the LLVM-IR that we currently produces for this example, and ends up adding noalias, nonnull, dereferenceable, etc. to the maybe_bad2::foodeclaration whose type actually matches the definition. This is an instance of rust-lang/rust#46188 . It is unclear whether it would be a legal optimization on LLVM-IR to propagate such attributes to the function definition, which would result in severe mis-compilations. This interacts with LTO and therefore probably with xLTO as well (e.g. a Rust declaration can probably propagate attributes to a C declaration when xLTO is involved).

In C, declarations are not only unsafe to call, but also unsafe to declare. I don't think we can do that in Rust, since that would break pretty much all existing FFI code, and it would not allow users to use Rust types to better express the API of C function declarations.

So AFAICT, we have to rule the examples above as correct, and implement them in such a way that does not cause miscompilations - that is, we could close this and just handle this by fixing: rust-lang/rust#46188 , and maybe adding a PR to the reference explaining that this is explicitly ok.