RFC: Existential types with external definition by Ericson2314 · Pull Request #2492 · rust-lang/rfcs (original) (raw)

Hi! As a user of Rust, trying to implement cryptographic protocols for embedded devices, I wanted to weigh in on what my use-case is, the approaches I have considered, and ask if there's anything I can do to help this along. If there's a more appropriate place to post this, please let me know.

Use Case

The specific use case here is that the cryptographic protocols in question make use of a lot of common cryptographic primitives, e.g. the BLAKE2b hash function. The embedded devices likely have an existing implementation (which we don't want to duplicate because there are strict firmware size constraints), or an optimized implementations (e.g. if the device includes a cryptographic accelerator, we want to take advantage of it rather than using a slow software implementation).

Therefore, we desire the following:

• Allow the downstream user to provide their own implementation of each cryptographic primitive (i.e. provide a concrete implementation of a trait).
• Allow the downstream user to run the test suite with their own implementations because this is high-assurance code and we need to test it with the cryptographic implementations being used in production.

Approaches

Extern existential types

The real world example is a main firmware crate (which is completely different for each downstream user and contains their application-specific code), which depends on firmware_sys since the cryptographic primitives are currently implemented in C 🙈, and also depends on library (which is the library where we want to pass these primitives to). This is represented by the simple dependency graph below:

This RFC solves the problem since we can have our library crate declare an extern existential type, e.g.

pub trait Blake512 { fn new() -> Self; // ... } pub extern existential type Blake2b512: Blake512;

The library crate can use this type as usual (e.g. Blake2b512::new() Just Works:tm:) and the firmware crate can implement this type:

impl library::Blake512 for Blake2b512 { // ... } extern existential type library::Blake2b512 = Blake2b512;

I presume it will also be possible to run tests on the library crate with the extern existential implementations from another crate (i.e. firmware) in the dependency graph. ✨

Cargo dependency patching

Another solution is to swap out a crate using the [patch] section in Cargo.toml:

[patch.crates-io] library_crypto = { package = "firmware_library_crypto" }

Since we've broken out the default cryptographic implementations into a new crate (library_crypto), we need to also break out the relevant traits (so firmware_library_crypto can depend on them and library can depend on them without a circular dependency). And we need to break out the cryptographic implementations from firmware into firmware_crypto, otherwise we'd have another circular dependency. And we end up with this absolute mess of a dependency graph (the library_crypto -> firmware_library_crypto dependency isn't actually a dependency relation, rather the latter replaces the former in the dependency graph).

Apart from the horrific dependency tree (and the foot-guns around trying to emulate the library_crypto crate's public interface when implementing firmware_library_crypto), this should work just as well as the existential types RFC, e.g. cargo test -p library should work but I haven't tested it.

I'm fine with the friction introduced by the various library_ crates. However, splitting firmware into firmware_crypto is a pain for the downstream user and shouldn't be necessary 😕

no_mangle hacks

For a ZST or a singleton, you can use a hack like the following in the firmware crate (obviously the library crate can provide a macro for this):

#[no_mangle] pub static LIBRARY_EXTERN_SINGLETON: &(dyn Trait + 'static) = &SINGLETON;

However, this doesn't work if you need a non-ZST type that you can instantiate and work with (e.g. as is the case for cryptographic hash function contexts) since there's no way to use the linker to get the size of the firmware concrete type to the library crate. You could use a #[no_mangle] function which returns Box<dyn Trait>, but this doesn't work for embedded devices without allocators (or in other zero-allocation contexts):

#[no_mangle] pub fn library_extern_blake2b512_new() -> Box { // ... }

Moreover, this probably doesn't work with cargo test -p library, since cargo will probably ignore the firmware crate and fail with a linker error. (I haven't tested this.)

Add trait bounds to every single impl/struct/fn/enum

This "solves" the problem in that it does not require changes to the dependency graph and it works on embedded devices in a zero-allocation context.

But this is a huge pain for both the library author - who is forced to add the trait bounds to every single type (for ergonomics and because adding the trait bound in the future would be a breaking change) - and consumer - who winds up adding a lot of turbofishes in. In the case of cryptographic primitives, they don't provide types which are used in structure fields.. so now every struct needs a PhantomData too 😭.

The ergonomics issue can perhaps be solved by #424 (but, as far as I can tell, there are no plans for this in the near future).

Moreover, this doesn't solve the tests problem at all. It's possible I could use #![feature(custom_test_frameworks)] (rust-lang/rust#50297) to provide a way for the library user to import the library test cases into their own crate.

Conclusion

I'm currently exploring the Cargo dependency patching approach and the trait bounds approach (since these are the only two approaches that "work" - for varying definitions of "work" 🤣 - and exist currently). However, the extern existential types approach is perfect and it would hugely improve the ergonomics for the library author and consumer.