Allow "artifact dependencies" on bin, cdylib, and staticlib crates by jyn514 · Pull Request #3028 · rust-lang/rfcs (original) (raw)

@alexcrichton wrote:

One question I would have though is how is it expected to hook up a foreign binary into Cargo? For example it looks like the env vars specified to downstream crates are based on Cargo-based-[[bin]] targets, so how would binaries produced by the build script (e.g. cmake) get specified?

With the RFC as specified, if you want to produce and depend on a bin artifact, it needs to be a Cargo [[bin]] target. I can think of many use cases for other possibilities, such as passing through a binary produced by one of your dependencies, or building a binary in your build script. If you're open to the possibility, I'd love to have a way to declare a [[bin]] target that's produced by build.rs.

Perhaps something like this?

[[bin]] name = "foo" manual = true

A [[bin]] target with manual=true will not have any logic emitted by Cargo; Cargo will just give build.rs an environment variable specifying a directory to put it in, and then error if it doesn't get built by build.rs.

Thoughts?

I do feel like this is a bit pie-in-the-sky for tools like cmake/clang/gcc though given that their build times is so long and ubiquity is so high. For smaller binaries and/or less-widely-known ones, however, I could imagine this being quite useful.

It's a tradeoff, just like any vendoring is a tradeoff. Many -sys crates offer the option of building a vendored copy, with the same tradeoff in build time.

Also, if we have the above mechanism for supplying binaries via build.rs, we could consider supplying a sysroot crate (installable via rustup) containing the LLVM we built rustc with.

One concern about this RFC with targets I'm realizing now is that "it's just cargo build" isn't quite right. If a crate has a variety of targets specified it's actually "run this set of rustup target add commands" (which is project-specific and perhaps not easily discoverable) and then "run cargo build". In terms of ease-of-use that may be a relatively high-priority item to tackle around this RFC as well?

As @bjorn3 observed, that's supported in rustup today, by specifying a rust-toolchain toml file that requires the targets you want. I don't think we need to do anything more than that in this RFC. Any further support for that in Cargo would be a different RFC.

@demurgos that's actually an example I'm worried about with this RFC, actually. It seems like it should work for wasm-bindgen but this would not be usable for wasm-bindgen. This RFC isn't intended for postprocessing artifacts, but only processing preexisting artifacts with external tools into the rest of the build. Nothing here would help wasm-bindgen's experience, I believe.

One day I hope we have a way to run Rust code post-build to generate or post-process artifacts. I want to avoid rolling too much into this RFC, though, and I'd expect post-build code to invite more controversy.

In the meantime, though, if we add a mechanism for supplying binaries, people could technically handle this by building a workspace with two crates, one building the un-preprocessed binary, and the other depending on that and wasm-bindgen and emitting a processed binary. That's not ideal, but it works.

@jyn514 wrote:

FWIW I would be ok with removing the target-specific parts of the RFC, or limiting it only to the host or --target. @joshtriplett what do you think?

Given the ability to enable those targets via a rust-toolchain file, and the value of this for cases like wasm, I'd like to keep this in the RFC.

@XAMPPRocky wrote:

Hey, just wanted to add a use case for artifact dependencies. In rust-gpu we want to be able to build and include SPIR-V shader crates in the main binary which is actually responsible for running them. Currently this implemented in a very hacky way with having a spirv-builder crate which runs cargo inside a build script to build the SPIR-V crate and expose the path to the binary in an environment variable.

This isn't really ideal since there have been a number of issues with calling cargo inside of cargo (from it easily creating directories that are beyond MAX_PATH in Windows and failing to build, to environment variables from the top level cargo build affecting the shader build). It would be so much nicer and simpler if you could say shader = { path = "...", target = "spirv-unknown-unknown" } in your Cargo.toml.

That's an awesome example! I've added it to the examples in the RFC.