Introduce run-make V2 infrastructure, a run_make_support library and port over 2 tests as example by jieyouxu · Pull Request #113026 · rust-lang/rust (original) (raw)

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Conversation199 Commits1 Checks0 Files changed

Conversation

This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.Learn more about bidirectional Unicode characters

[ Show hidden characters]({{ revealButtonHref }})

jieyouxu

Preface

See issue #40713: Switch run-make tests from Makefiles to rust for more context.

Basic Description of run-make V2

run-make V2 aims to eliminate the dependency on make and Makefiles for building run-make-style tests. Makefiles are replaced by recipes (rmake.rs). The current implementation runs run-make V2 tests in 3 steps:

  1. We build the support library run_make_support which the rmake.rs recipes depend on as a tool lib.
  2. We build the recipe rmake.rs and link in the support library.
  3. We run the recipe to build and run the tests.

rmake.rs is basically a replacement for Makefile, and allows running arbitrary Rust code. The support library is built using cargo, and so can depend on external crates if desired.

The infrastructure implemented by this PR is very barebones, and is the minimally required infrastructure needed to build, run and pass the two example run-make tests ported over to the new infrastructure.

Example run-make V2 test

// ignore-tidy-linelength

extern crate run_make_support;

use std::path::PathBuf;

use run_make_support::{aux_build, rustc};

fn main() { aux_build() .arg("--emit=metadata") .arg("stable.rs") .run(); let mut stable_path = PathBuf::from(env!("TMPDIR")); stable_path.push("libstable.rmeta"); let output = rustc() .arg("--emit=metadata") .arg("--extern") .arg(&format!("stable={}", &stable_path.to_string_lossy())) .arg("main.rs") .run();

let stderr = String::from_utf8_lossy(&output.stderr);
let version = include_str!(concat!(env!("S"), "/src/version"));
let expected_string = format!("stable since {}", version.trim());
assert!(stderr.contains(&expected_string));

}

Follow Up Work

@rustbot

@rustbot rustbot added A-testsuite

Area: The testsuite used to check the correctness of rustc

S-waiting-on-review

Status: Awaiting review from the assignee but also interested parties.

T-bootstrap

Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)

labels

Jun 25, 2023

@rust-log-analyzer

This comment has been minimized.

@rust-log-analyzer

This comment has been minimized.

bjorn3

Comment on lines 78 to 85

let mut cmd = $crate::xshell::cmd!(sh, "{rustc}")
.arg("--out-dir")
.arg($scx.tmpdir())
.arg("-L")
.arg($scx.tmpdir());
for arg in args.split_whitespace() {
cmd = cmd.arg(arg);
}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe do:

let mut cmd = $crate::xshell::cmd!(sh, "{rustc}")
.arg("--out-dir")
.arg($scx.tmpdir())
.arg("-L")
.arg($scx.tmpdir());
for arg in args.split_whitespace() {
cmd = cmd.arg(arg);
}
let mut cmd = crate::xshell::cmd!(sh,concatstr!("rustc",crate::xshell::cmd!(sh, concat_str!("{rustc} ", crate::xshell::cmd!(sh,concatstr!("rustc",args))
.arg("--out-dir")
.arg($scx.tmpdir())
.arg("-L")
.arg($scx.tmpdir());

This should allow interpolations the same way as xshell itself and should correctly handle quoting arguments. I haven't tested it though.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A bit unfortunate but cmd!() only accepts a literal, can concat_str! output a literal? I don't think so right? And yeah quoting arguments is a problem with the current incorrect behavior of split_whitespace.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could force the input to be a &[], since

let args = ["hello", "world"]; let c = cmd!(sh, "echo {args...}");

works I think. This however forces the use site to become something like

let _output = rustc!(scx, ["--emit=metadata", "--crate-type=lib", "stable.rs"]); let output = rustc!( scx, [ "--emit=metadata", "--extern", &format!("stable={}/libstable.rmeta", scx.tmpdir()), "main.rs" ] );

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Of course. cmd!() would see the concat_str!() before expanding. @matklad would you accept adding something like a method on Shell that defines a wrapper program for all commands spawned using this Shell instance? Such that this could be cmd!(sh, $args) where sh is Shell::new().wrapper_command(rustc_path).

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While a proper fix to this requires a change to xshell, I will temporarily use the iterable form for the recipes since they're at least more correct than split_whitespace.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let _output = rustc!(scx, ["--emit=metadata", "--crate-type=lib", "stable.rs"]);

We are passing argument as an array here, so I don't think we even need xshell? The whole purpose of xshell is to avoid needing an array literal, but, if you are okay with that, I don't think xshell brings much to the table here?

would you accept adding something like a method on Shell that defines a wrapper program for all commands spawned using this Shell instance?

This would technically work, but I don't think that's an obvious API with a clear use-case. My answer would be, if you want to control the first arg, to use something like

cmd!(sh, "{rustc} --emit=metadata --crate-type=lib")

Like shells, xshell supports interpolation of the first argumetn (or at least it should support it).

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are passing argument as an array here, so I don't think we even need xshell? The whole purpose of xshell is to avoid needing an array literal, but, if you are okay with that, I don't think xshell brings much to the table here?

(Note that passing-as-an-array here is a temporary band-aid fix, the intention is to avoid needing an array literal.) That is, ideally, the API would look something more like

let _output = rustc!("--emit=metadata --crate-type=lib stable.rs");

so the test writer doesn't have to bother with an array literal.

bjorn3

bjorn3

bjorn3

bjorn3

bjorn3

bjorn3

@jieyouxu jieyouxu changed the title[PROTOTYPE] Introduce run-make-v2 infrastructure, a rmake_support library and port over 2 tests as example [PROTOTYPE] Introduce run-make V2 infrastructure, a run_make_support library and port over 2 tests as example

Jun 26, 2023

bjorn3

matklad

Cargo.lock Outdated Show resolved Hide resolved

Comment on lines 78 to 85

let mut cmd = $crate::xshell::cmd!(sh, "{rustc}")
.arg("--out-dir")
.arg($scx.tmpdir())
.arg("-L")
.arg($scx.tmpdir());
for arg in args.split_whitespace() {
cmd = cmd.arg(arg);
}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let _output = rustc!(scx, ["--emit=metadata", "--crate-type=lib", "stable.rs"]);

We are passing argument as an array here, so I don't think we even need xshell? The whole purpose of xshell is to avoid needing an array literal, but, if you are okay with that, I don't think xshell brings much to the table here?

would you accept adding something like a method on Shell that defines a wrapper program for all commands spawned using this Shell instance?

This would technically work, but I don't think that's an obvious API with a clear use-case. My answer would be, if you want to control the first arg, to use something like

cmd!(sh, "{rustc} --emit=metadata --crate-type=lib")

Like shells, xshell supports interpolation of the first argumetn (or at least it should support it).

@jieyouxu

I changed the implementation to rely on shell-words instead to split the input string into words that can be passed to Command::arg. The API now looks something like

extern crate run_make_support;

use run_make_support::{rustc, run, run_fail};

fn main() { let _output = rustc("a.rs --cfg x -C prefer-dynamic -Z unstable-options -C symbol-mangling-version=legacy"); let _output = rustc("b.rs -C prefer-dynamic -Z unstable-options -C symbol-mangling-version=legacy"); let _output = run("b"); let _output = rustc("a.rs --cfg y -C prefer-dynamic -Z unstable-options -C symbol-mangling-version=legacy"); let _output = run_fail("b"); }

This unfortunately isn't as nice as having straight up string interpolation like xshell does, e.g. in CURRENT_RUSTC_VERSION:

extern crate run_make_support;

use run_make_support::{rustc, aux_build};

fn main() { let _output = aux_build("stable.rs"); let output = rustc(format!("--emit=metadata --extern stable={}/libstable.rmeta main.rs", env!("TMPDIR")));

let stderr = String::from_utf8_lossy(&output.stderr);
let version = include_str!(concat!(env!("S"), "/src/version"));
let expected_string = format!("stable since {}", version.trim());
assert!(stderr.contains(&expected_string));

}

@jieyouxu jieyouxu marked this pull request as ready for review

June 27, 2023 05:40

@rustbot

These commits modify the Cargo.lock file. Unintentional changes to Cargo.lock can be introduced when switching branches and rebasing PRs.

If this was unintentional then you should revert the changes before this PR is merged.
Otherwise, you can ignore this comment.

@matklad

It might be interesting to combine "split-on-whitespace" idea with builder-style interface. This is the style of testing employed by Cargo:

https://github.com/rust-lang/cargo/blob/5febbe5587b74108165f748e79a4f8badbdf5e0e/tests/testsuite/build.rs#L4168-L4169

So,

let tmpdir = env!("TMPDIR"); let output = rustc("main.rs --emit=metadata") .arg(format!("--extern=stable={tempdir}/libstable.rmeta")) .stderr_utf8_lossy()? ;

Thinking more about this, I think "specific builders" are probably a better fit here than "generic shell".

I expect these tests to be relatively few, so we don't have to optimize suuuper hard for readability.

At the same time, I expect most of these tests to be similar, and to benefit from abstracting common functionality. Abstracting through a builder is easier than abstracting through macro-based DSL.

@bors

Mark-Simulacrum

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

r? @bjorn3 - this looks good to me in broad strokes, but looks like you've done a bunch of review here already.

@jieyouxu

tools.mk special cases Windows and uses PATH=PATH:$TARGET_RPATH_DIR for executables instead of just using $TARGET_RPATH_ENV for non-Windows platforms...

@rust-log-analyzer

This comment has been minimized.

@rust-log-analyzer

This comment has been minimized.

@jieyouxu

DLL_NOT_FOUND but am not sure which DLL is not found... One definite bug is PATH=$1;$2 seperator is ; on Windows and not : like *nixes.

bjorn3

target_rpath_env_path.push_str(&std::env::var("TARGET_RPATH_ENV").unwrap());
target_rpath_env_path.push(':');
target_rpath_env_path.push(path_sep(&target));

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is the host that matters here, right? You can probably use std::env::join_paths here.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, that did it!

@rust-log-analyzer

This comment has been minimized.

@jieyouxu

@jieyouxu

Seems to finally work for msvc!
@rustbot ready

@rustbot rustbot added S-waiting-on-review

Status: Awaiting review from the assignee but also interested parties.

and removed S-waiting-on-author

Status: This is awaiting some action (such as code changes or more information) from the author.

labels

Feb 29, 2024

@bjorn3

@bors

📌 Commit 48e9f92 has been approved by bjorn3

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors

Status: Waiting on bors to run and complete tests. Bors will change the label on completion.

and removed S-waiting-on-review

Status: Awaiting review from the assignee but also interested parties.

labels

Mar 1, 2024

@bors

@bors

This was referenced

Mar 1, 2024

@rust-timer

Finished benchmarking commit (17edace): comparison URL.

Overall result: no relevant changes - no action needed

@rustbot label: -perf-regression

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

This benchmark run did not return any relevant results for this metric.

Cycles

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌ (primary) - - 0
Regressions ❌ (secondary) 2.7% [1.7%, 3.5%] 9
Improvements ✅ (primary) - - 0
Improvements ✅ (secondary) - - 0
All ❌✅ (primary) - - 0

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 651.67s -> 651.533s (-0.02%)
Artifact size: 311.10 MiB -> 311.14 MiB (0.01%)

wip-sync pushed a commit to NetBSD/pkgsrc-wip that referenced this pull request

May 4, 2024

@he32

Pkgsrc changes:

Upstream chnages:

Version 1.78.0 (2024-05-02)

Language

Compiler

Target changes:

Refer to Rust's [platform support page][platform-support-doc] for more information on Rust's tiered platform support.

Libraries

Stabilized APIs

These APIs are now stable in const contexts:

Cargo

Misc

Compatibility Notes

Internal Changes

These changes do not affect any public interfaces of Rust, but they represent significant improvements to the performance or internals of rustc and related tools.

netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this pull request

Oct 13, 2024

@he32

This is based on the pkgsrc-wip rust180 package, retaining the main pkgsrc changes as best as I could.

Pkgsrc changes:

Upstream chnages:

Version 1.80.1 (2024-08-08)

Version 1.80.0 (2024-07-25)

Language

Compiler

Libraries

Stabilized APIs

These APIs are now stable in const contexts:

Cargo

Rustdoc

Compatibility Notes

Internal Changes

These changes do not affect any public interfaces of Rust, but they represent significant improvements to the performance or internals of rustc and related tools.

Version 1.79.0 (2024-06-13)

Language

Compiler

Refer to Rust's [platform support page][platform-support-doc] for more information on Rust's tiered platform support.

Libraries

Stabilized APIs

These APIs are now stable in const contexts:

Cargo

Rustdoc

Misc

Compatibility Notes

Version 1.78.0 (2024-05-02)

Language

Compiler

Target changes:

Refer to Rust's [platform support page][platform-support-doc] for more information on Rust's tiered platform support.

Libraries

Stabilized APIs

These APIs are now stable in const contexts:

Cargo

Misc

Compatibility Notes

Internal Changes

These changes do not affect any public interfaces of Rust, but they represent significant improvements to the performance or internals of rustc and related tools.

Version 1.77.0 (2024-03-21)

Compiler

Refer to Rust's [platform support page][platform-support-doc] for more information on Rust's tiered platform support.

Libraries

Stabilized APIs

Cargo

Rustdoc

Misc

Internal Changes

These changes do not affect any public interfaces of Rust, but they represent significant improvements to the performance or internals of rustc and related tools.

Labels

A-testsuite

Area: The testsuite used to check the correctness of rustc

merged-by-bors

This PR was explicitly merged by bors.

S-waiting-on-bors

Status: Waiting on bors to run and complete tests. Bors will change the label on completion.

T-bootstrap

Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)

T-infra

Relevant to the infrastructure team, which will review and decide on the PR/issue.