Bindgen as a workspace by pvdrz · Pull Request #2284 · rust-lang/rust-bindgen (original) (raw)

This reorganizes the repo into a virtual workspace. The main change is that bindgen as a build-script library and bindgen as a command line utility have been split into the bindgen and bindgen-cli crates respectively.

This means that bindgen no longer depends on clap. Transforming it's cargo tree from this:

bindgen v0.60.1 (/home/christian/Workspace/bindgen/rust-bindgen)
├── bitflags v1.3.2
├── cexpr v0.6.0
│   └── nom v7.0.0
│       ├── memchr v2.5.0
│       └── minimal-lexical v0.1.4
│       [build-dependencies]
│       └── version_check v0.9.3
├── clang-sys v1.3.3
│   ├── glob v0.3.0
│   ├── libc v0.2.126
│   └── libloading v0.7.0
│       └── cfg-if v1.0.0
│   [build-dependencies]
│   └── glob v0.3.0
├── clap v3.2.12
│   ├── atty v0.2.14
│   │   └── libc v0.2.126
│   ├── bitflags v1.3.2
│   ├── clap_lex v0.2.4
│   │   └── os_str_bytes v6.2.0
│   ├── indexmap v1.9.1
│   │   └── hashbrown v0.12.2
│   │   [build-dependencies]
│   │   └── autocfg v1.1.0
│   ├── strsim v0.10.0
│   ├── termcolor v1.1.3
│   └── textwrap v0.15.0
├── env_logger v0.9.0
│   ├── atty v0.2.14 (*)
│   ├── humantime v2.1.0
│   ├── log v0.4.14
│   │   └── cfg-if v1.0.0
│   ├── regex v1.5.5
│   │   ├── aho-corasick v0.7.18
│   │   │   └── memchr v2.5.0
│   │   ├── memchr v2.5.0
│   │   └── regex-syntax v0.6.25
│   └── termcolor v1.1.3
├── lazy_static v1.4.0
├── lazycell v1.3.0
├── log v0.4.14 (*)
├── peeking_take_while v0.1.2
├── proc-macro2 v1.0.43
│   └── unicode-ident v1.0.3
├── quote v1.0.9
│   └── proc-macro2 v1.0.43 (*)
├── regex v1.5.5 (*)
├── rustc-hash v1.1.0
├── shlex v1.0.0
├── syn v1.0.99
│   ├── proc-macro2 v1.0.43 (*)
│   ├── quote v1.0.9 (*)
│   └── unicode-ident v1.0.3
└── which v4.2.2
    ├── either v1.6.1
    └── libc v0.2.126
[dev-dependencies]
├── clap v3.2.12 (*)
├── diff v0.1.12
├── shlex v1.0.0
└── tempfile v3.2.0
    ├── cfg-if v1.0.0
    ├── libc v0.2.126
    ├── rand v0.8.4
    │   ├── libc v0.2.126
    │   ├── rand_chacha v0.3.1
    │   │   ├── ppv-lite86 v0.2.10
    │   │   └── rand_core v0.6.3
    │   │       └── getrandom v0.2.3
    │   │           ├── cfg-if v1.0.0
    │   │           └── libc v0.2.126
    │   └── rand_core v0.6.3 (*)
    └── remove_dir_all v0.5.3

to this:

bindgen v0.60.1 (/home/christian/Workspace/bindgen/rust-bindgen/bindgen)
├── bitflags v1.3.2
├── cexpr v0.6.0
│   └── nom v7.0.0
│       ├── memchr v2.5.0
│       └── minimal-lexical v0.1.4
│       [build-dependencies]
│       └── version_check v0.9.3
├── clang-sys v1.3.3
│   ├── glob v0.3.0
│   ├── libc v0.2.126
│   └── libloading v0.7.0
│       └── cfg-if v1.0.0
│   [build-dependencies]
│   └── glob v0.3.0
├── lazy_static v1.4.0
├── lazycell v1.3.0
├── log v0.4.14
│   └── cfg-if v1.0.0
├── peeking_take_while v0.1.2
├── proc-macro2 v1.0.43
│   └── unicode-ident v1.0.3
├── quote v1.0.9
│   └── proc-macro2 v1.0.43 (*)
├── regex v1.5.5
│   └── regex-syntax v0.6.25
├── rustc-hash v1.1.0
├── shlex v1.0.0
├── syn v1.0.99
│   ├── proc-macro2 v1.0.43 (*)
│   ├── quote v1.0.9 (*)
│   └── unicode-ident v1.0.3
└── which v4.2.2
    ├── either v1.6.1
    └── libc v0.2.126

There are some questions that I'd like to resolve before moving forward with this:

Is CI working correctly? There are many test crates like quickcheck and expect but I'm not sure if my changes to the ci scripts are actually running everything they should.
How do we distribute bindgen-cli? RIght now you can test cargo install running the command
cargo install --git=https://github.com/ferrous-systems/rust-bindgen --branch=bindgen-as-workspace bindgen-cli
Which installs a bindgen-cli binary instead of just bindgen. I haven't found out how to change the name of the binary back to just bindgen.

pvdrz changed the title~~[WIP] Bindgen as a workspace~~ Bindgen as a workspace

Sep 27, 2022

☔ The latest upstream changes (presumably #2282) made this pull request unmergeable. Please resolve the merge conflicts.

☔ The latest upstream changes (presumably #2287) made this pull request unmergeable. Please resolve the merge conflicts.

☔ The latest upstream changes (presumably 8b69ba0) made this pull request unmergeable. Please resolve the merge conflicts.

☔ The latest upstream changes (presumably #2285) made this pull request unmergeable. Please resolve the merge conflicts.

This looks great, though it'd be great to not change the name of the CLI tool. It seems something like this would do?

diff --git a/bindgen-cli/Cargo.toml b/bindgen-cli/Cargo.toml index 2aef9b5b..ebeceacc 100644 --- a/bindgen-cli/Cargo.toml +++ b/bindgen-cli/Cargo.toml @@ -21,6 +21,10 @@ include = [ "build.rs", ]

+[[bin]] +path = "src/main.rs" +name = "bindgen" + [badges] travis-ci = { repository = "rust-lang/rust-bindgen" }

Also, we need to update the docs that mention cargo install and so on.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other thoughts:

Maybe we can flatten the directory structure a bit by using Cargo.toml magic (e.g. don't require bindgen/src, bindgen-cli/src and so on).
It seems some of the test scripts need a bit of work.
The naming of the cli crate I believe should probably be kept, I think what I posted earlier should work but lmk if it doesn't for any reason.

Other than that, this looks pretty good!

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Huh?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this seems to be a merge conflict casualty

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apparently the file was created here and it was empty from the start. I'll remove it in an upcoming PR: fitzgen@b4895d9

ci/test.sh Outdated Show resolved Hide resolved

@emilio the [[bin]] section on the manifest worked. You still have to install bindgen-cli but get a bindgen executable.

pvdrz mentioned this pull request

Oct 4, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's get this merged sooner rather than later to avoid all the conflicts.

Can you squash your commits before landing please? Thank you for doing this!

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple last minute comments.

remove clap dependency 🎉

update the book installation instructions

pvdrz deleted the bindgen-as-workspace branch

October 5, 2022 01:56

This was referenced

Oct 5, 2022

Hi everyone! If you are reading this message because this PR broke yours, I'm sorry 😬. For most of the cases it is enough to rebase against master as there won't be any conflicts, just files being moved here and there

This was referenced

Oct 17, 2022

lopopolo added a commit to artichoke/docker-artichoke-nightly that referenced this pull request

Oct 22, 2022

lopopolo added a commit to artichoke/artichoke that referenced this pull request

Oct 22, 2022

ojeda pushed a commit to Rust-for-Linux/linux that referenced this pull request

Aug 14, 2023

In LLVM 16, anonymous items may return names like (unnamed union at ..) rather than empty names [1], which breaks Rust-enabled builds because bindgen assumed an empty name instead of detecting them via clang_Cursor_isAnonymous [2]:

$ make rustdoc LLVM=1 CLIPPY=1 -j$(nproc)
  RUSTC L rust/core.o
  BINDGEN rust/bindings/bindings_generated.rs
  BINDGEN rust/bindings/bindings_helpers_generated.rs
  BINDGEN rust/uapi/uapi_generated.rs
thread 'main' panicked at '"ftrace_branch_data_union_(anonymous_at__/_/include/linux/compiler_types_h_146_2)" is not a valid Ident', .../proc-macro2-1.0.24/src/fallback.rs:693:9
...
thread 'main' panicked at '"ftrace_branch_data_union_(anonymous_at__/_/include/linux/compiler_types_h_146_2)" is not a valid Ident', .../proc-macro2-1.0.24/src/fallback.rs:693:9
...

This was fixed in bindgen 0.62.0. Therefore, upgrade bindgen to a more recent version, 0.65.1, to support LLVM 16.

Since bindgen 0.58.0 changed the --{white,black}list-* flags to --{allow,block}list-* [3], update them on our side too.

In addition, bindgen 0.61.0 moved its CLI utility into a binary crate called bindgen-cli [4]. Thus update the installation command in the Quick Start guide.

Moreover, bindgen 0.61.0 changed the default functionality to bind size_t to usize [5] and added the --no-size_t-is-usize flag to not bind size_t as usize. Then bindgen 0.65.0 removed the --size_t-is-usize flag [6]. Thus stop passing the flag to bindgen.

Finally, bindgen 0.61.0 added support for the noreturn attribute (in its different forms) [7]. Thus remove the infinite loop in our Rust panic handler after calling BUG(), since bindgen now correctly generates a BUG() binding that returns ! instead of ().

Link: llvm/llvm-project@19e984e [1] Link: rust-lang/rust-bindgen#2319 [2] Link: rust-lang/rust-bindgen#1990 [3] Link: rust-lang/rust-bindgen#2284 [4] Link: rust-lang/rust-bindgen@cc78b6f [5] Link: rust-lang/rust-bindgen#2408 [6] Link: rust-lang/rust-bindgen#2094 [7] Signed-off-by: Aakash Sen Sharma aakashsensharma@gmail.com Closes: #1013 Tested-by: Ariel Miculas amiculas@cisco.com Reviewed-by: Gary Guo gary@garyguo.net Link: https://lore.kernel.org/r/20230612194311.24826-1-aakashsensharma@gmail.com [ Reworded commit message. Mentioned the bindgen-cli binary crate change, linked to it and updated the Quick Start guide. Re-added a deleted "as" word in a code comment and reflowed comment to respect the maximum length. ] Signed-off-by: Miguel Ojeda ojeda@kernel.org

ojeda added a commit to ojeda/kernelci-core that referenced this pull request

Aug 22, 2023

bindgen 0.65.1 will be the next version supported by the kernel [1].

Therefore, add it to the latest build environment.

This also requires updating the name of the package from bindgen to bindgen-cli, since bindgen 0.61.0 moved its CLI utility into a binary crate called bindgen-cli [2].

Link: Rust-for-Linux/linux@08ab786 [1] Link: rust-lang/rust-bindgen#2284 [2] Signed-off-by: Miguel Ojeda ojeda@kernel.org

cthbleachbit pushed a commit to AOSC-Tracking/linux that referenced this pull request

Aug 23, 2023

$ make rustdoc LLVM=1 CLIPPY=1 -j$(nproc)
  RUSTC L rust/core.o
  BINDGEN rust/bindings/bindings_generated.rs
  BINDGEN rust/bindings/bindings_helpers_generated.rs
  BINDGEN rust/uapi/uapi_generated.rs
thread 'main' panicked at '"ftrace_branch_data_union_(anonymous_at__/_/include/linux/compiler_types_h_146_2)" is not a valid Ident', .../proc-macro2-1.0.24/src/fallback.rs:693:9
...
thread 'main' panicked at '"ftrace_branch_data_union_(anonymous_at__/_/include/linux/compiler_types_h_146_2)" is not a valid Ident', .../proc-macro2-1.0.24/src/fallback.rs:693:9
...

This was fixed in bindgen 0.62.0. Therefore, upgrade bindgen to a more recent version, 0.65.1, to support LLVM 16.

Since bindgen 0.58.0 changed the --{white,black}list-* flags to --{allow,block}list-* [3], update them on our side too.

In addition, bindgen 0.61.0 moved its CLI utility into a binary crate called bindgen-cli [4]. Thus update the installation command in the Quick Start guide.

Link: llvm/llvm-project@19e984e [1] Link: rust-lang/rust-bindgen#2319 [2] Link: rust-lang/rust-bindgen#1990 [3] Link: rust-lang/rust-bindgen#2284 [4] Link: rust-lang/rust-bindgen@cc78b6f [5] Link: rust-lang/rust-bindgen#2408 [6] Link: rust-lang/rust-bindgen#2094 [7] Signed-off-by: Aakash Sen Sharma aakashsensharma@gmail.com Closes: Rust-for-Linux#1013 Tested-by: Ariel Miculas amiculas@cisco.com Reviewed-by: Gary Guo gary@garyguo.net Link: https://lore.kernel.org/r/20230612194311.24826-1-aakashsensharma@gmail.com [ Reworded commit message. Mentioned the bindgen-cli binary crate change, linked to it and updated the Quick Start guide. Re-added a deleted "as" word in a code comment and reflowed comment to respect the maximum length. ] Signed-off-by: Miguel Ojeda ojeda@kernel.org

github-merge-queue bot pushed a commit to kernelci/kernelci-core that referenced this pull request

Aug 25, 2023

bindgen 0.65.1 will be the next version supported by the kernel [1].

Therefore, add it to the latest build environment.

This also requires updating the name of the package from bindgen to bindgen-cli, since bindgen 0.61.0 moved its CLI utility into a binary crate called bindgen-cli [2].

Link: Rust-for-Linux/linux@08ab786 [1] Link: rust-lang/rust-bindgen#2284 [2] Signed-off-by: Miguel Ojeda ojeda@kernel.org

cthbleachbit pushed a commit to AOSC-Tracking/linux that referenced this pull request

Sep 13, 2023

$ make rustdoc LLVM=1 CLIPPY=1 -j$(nproc)
  RUSTC L rust/core.o
  BINDGEN rust/bindings/bindings_generated.rs
  BINDGEN rust/bindings/bindings_helpers_generated.rs
  BINDGEN rust/uapi/uapi_generated.rs
thread 'main' panicked at '"ftrace_branch_data_union_(anonymous_at__/_/include/linux/compiler_types_h_146_2)" is not a valid Ident', .../proc-macro2-1.0.24/src/fallback.rs:693:9
...
thread 'main' panicked at '"ftrace_branch_data_union_(anonymous_at__/_/include/linux/compiler_types_h_146_2)" is not a valid Ident', .../proc-macro2-1.0.24/src/fallback.rs:693:9
...

This was fixed in bindgen 0.62.0. Therefore, upgrade bindgen to a more recent version, 0.65.1, to support LLVM 16.

Since bindgen 0.58.0 changed the --{white,black}list-* flags to --{allow,block}list-* [3], update them on our side too.

In addition, bindgen 0.61.0 moved its CLI utility into a binary crate called bindgen-cli [4]. Thus update the installation command in the Quick Start guide.

Link: llvm/llvm-project@19e984e [1] Link: rust-lang/rust-bindgen#2319 [2] Link: rust-lang/rust-bindgen#1990 [3] Link: rust-lang/rust-bindgen#2284 [4] Link: rust-lang/rust-bindgen@cc78b6f [5] Link: rust-lang/rust-bindgen#2408 [6] Link: rust-lang/rust-bindgen#2094 [7] Signed-off-by: Aakash Sen Sharma aakashsensharma@gmail.com Closes: Rust-for-Linux#1013 Tested-by: Ariel Miculas amiculas@cisco.com Reviewed-by: Gary Guo gary@garyguo.net Link: https://lore.kernel.org/r/20230612194311.24826-1-aakashsensharma@gmail.com [ Reworded commit message. Mentioned the bindgen-cli binary crate change, linked to it and updated the Quick Start guide. Re-added a deleted "as" word in a code comment and reflowed comment to respect the maximum length. ] Signed-off-by: Miguel Ojeda ojeda@kernel.org

vtta pushed a commit to vtta/linux-archive that referenced this pull request

Sep 29, 2023

$ make rustdoc LLVM=1 CLIPPY=1 -j$(nproc)
  RUSTC L rust/core.o
  BINDGEN rust/bindings/bindings_generated.rs
  BINDGEN rust/bindings/bindings_helpers_generated.rs
  BINDGEN rust/uapi/uapi_generated.rs
thread 'main' panicked at '"ftrace_branch_data_union_(anonymous_at__/_/include/linux/compiler_types_h_146_2)" is not a valid Ident', .../proc-macro2-1.0.24/src/fallback.rs:693:9
...
thread 'main' panicked at '"ftrace_branch_data_union_(anonymous_at__/_/include/linux/compiler_types_h_146_2)" is not a valid Ident', .../proc-macro2-1.0.24/src/fallback.rs:693:9
...

This was fixed in bindgen 0.62.0. Therefore, upgrade bindgen to a more recent version, 0.65.1, to support LLVM 16.

Since bindgen 0.58.0 changed the --{white,black}list-* flags to --{allow,block}list-* [3], update them on our side too.

In addition, bindgen 0.61.0 moved its CLI utility into a binary crate called bindgen-cli [4]. Thus update the installation command in the Quick Start guide.

Link: llvm/llvm-project@19e984e [1] Link: rust-lang/rust-bindgen#2319 [2] Link: rust-lang/rust-bindgen#1990 [3] Link: rust-lang/rust-bindgen#2284 [4] Link: rust-lang/rust-bindgen@cc78b6f [5] Link: rust-lang/rust-bindgen#2408 [6] Link: rust-lang/rust-bindgen#2094 [7] Signed-off-by: Aakash Sen Sharma aakashsensharma@gmail.com Closes: Rust-for-Linux#1013 Tested-by: Ariel Miculas amiculas@cisco.com Reviewed-by: Gary Guo gary@garyguo.net Link: https://lore.kernel.org/r/20230612194311.24826-1-aakashsensharma@gmail.com [ Reworded commit message. Mentioned the bindgen-cli binary crate change, linked to it and updated the Quick Start guide. Re-added a deleted "as" word in a code comment and reflowed comment to respect the maximum length. ] Signed-off-by: Miguel Ojeda ojeda@kernel.org

ishitatsuyuki pushed a commit to ishitatsuyuki/linux that referenced this pull request

Oct 6, 2023

$ make rustdoc LLVM=1 CLIPPY=1 -j$(nproc)
  RUSTC L rust/core.o
  BINDGEN rust/bindings/bindings_generated.rs
  BINDGEN rust/bindings/bindings_helpers_generated.rs
  BINDGEN rust/uapi/uapi_generated.rs
thread 'main' panicked at '"ftrace_branch_data_union_(anonymous_at__/_/include/linux/compiler_types_h_146_2)" is not a valid Ident', .../proc-macro2-1.0.24/src/fallback.rs:693:9
...
thread 'main' panicked at '"ftrace_branch_data_union_(anonymous_at__/_/include/linux/compiler_types_h_146_2)" is not a valid Ident', .../proc-macro2-1.0.24/src/fallback.rs:693:9
...

This was fixed in bindgen 0.62.0. Therefore, upgrade bindgen to a more recent version, 0.65.1, to support LLVM 16.

Since bindgen 0.58.0 changed the --{white,black}list-* flags to --{allow,block}list-* [3], update them on our side too.

In addition, bindgen 0.61.0 moved its CLI utility into a binary crate called bindgen-cli [4]. Thus update the installation command in the Quick Start guide.

Link: llvm/llvm-project@19e984e [1] Link: rust-lang/rust-bindgen#2319 [2] Link: rust-lang/rust-bindgen#1990 [3] Link: rust-lang/rust-bindgen#2284 [4] Link: rust-lang/rust-bindgen@cc78b6f [5] Link: rust-lang/rust-bindgen#2408 [6] Link: rust-lang/rust-bindgen#2094 [7] Signed-off-by: Aakash Sen Sharma aakashsensharma@gmail.com Closes: Rust-for-Linux#1013 Tested-by: Ariel Miculas amiculas@cisco.com Reviewed-by: Gary Guo gary@garyguo.net Link: https://lore.kernel.org/r/20230612194311.24826-1-aakashsensharma@gmail.com [ Reworded commit message. Mentioned the bindgen-cli binary crate change, linked to it and updated the Quick Start guide. Re-added a deleted "as" word in a code comment and reflowed comment to respect the maximum length. ] Signed-off-by: Miguel Ojeda ojeda@kernel.org

kulp mentioned this pull request

Nov 20, 2024