Add future-incompat lint for doc(primitive) by jyn514 · Pull Request #87050 · rust-lang/rust (original) (raw)

jyn514 added T-rustdoc

Relevant to the rustdoc team, which will review and decide on the PR/issue.

S-experimental

Status: Ongoing experiment that does not require reviewing and won't be merged in its current state.

labels

Jul 11, 2021

bors added the S-waiting-on-author

Status: This is awaiting some action (such as code changes or more information) from the author.

label

Jul 11, 2021

craterbot added S-waiting-on-crater

Status: Waiting on a crater run to be completed.

and removed S-experimental

Status: Ongoing experiment that does not require reviewing and won't be merged in its current state.

S-waiting-on-author

Status: This is awaiting some action (such as code changes or more information) from the author.

labels

Jul 11, 2021

jyn514 marked this pull request as draft

July 11, 2021 06:19

bors mentioned this pull request

Aug 16, 2021

GuillaumeGomez added a commit to GuillaumeGomez/rust that referenced this pull request

Aug 26, 2021

…omez

Fix rustdoc handling of primitive items

This is a complicated PR and does a lot of things. I'm willing to split it up a little more if it would help reviewing, but it would be tricky and I'd rather not unless it's necessary.

What does this do?

• Fixes rust-lang#73423.
• Fixes rust-lang#79630. I'm not sure how to test this for the standard library explicitly, but you can see from some of the diffs from the no_std tests. I also tested it locally and it works correctly:
• Fixes rust-lang#83083.

Why are these changes interconnected?

• Allowing anchors (rust-lang#83083) without fixing the online/offline problem (rust-lang#79630) will actually just silently discard the anchors, that's not a fix. The online/offline problem is directly related to the fragment hack; links need to go through fn href() to be fixed.
• Technically I could fix the online/offline problem without removing the error on anchors; I am willing to separate that out if it would be helpful for reviewing. However I can't fix the anchor problem without adding docs to core, since rustdoc needs all those primitives to have docs to avoid a fallback, and currently #![no_std] crates don't have docs for primitives. I also can't fix the online/offline problem without removing the fragment hack, since otherwise diffs like this will be wrong for some primitives but not others:

`@@` -385,7 +381,7 `@@` fn resolve_primitive_associated_item(
                         ty::AssocKind::Const => "associatedconstant",
                         ty::AssocKind::Type => "associatedtype",
                     };
-                    let fragment = format!("{}#{}.{}", prim_ty.as_sym(), out, item_name);
+                    let fragment = format!("{}.{}", out, item_name);
                     (Res::Primitive(prim_ty), fragment, Some((kind.as_def_kind(), item.def_id)))
                 })
         })

• Adding primitive docs to core without making any other change will cause links to go to core instead of std, even for crates with extern crate std. See "Breaking changes to doc(primitive)" below for why this is the case. That said, I could add some special casing to rustdoc at the same time that would let me separate this change from the others (it would fix rust-lang#73423 but still special-case intra-doc links). I'm willing to separate that out if helpful for reviewing.

Add primitive documentation to libcore

This works by reusing the same include!("primitive_docs.rs") file in both core and std, and then special-casing links in core to use relative links instead of intra-doc links. This doesn't use purely intra-doc links because some of the primitive docs links to items only in std; this doesn't use purely relative links because that introduces new broken links when the docs are re-exported (e.g. String's &str deref impl, or Vec's slice deref impl).

Fix inconsistent online/offline primitive docs

This does four things:

• Records modules with doc(primitive) in cache.external_paths. This is necessary for href() to find them later.
• Makes cache.primitive_locations available to the intra-doc link pass, by refactoring out a PrimitiveType::primitive_locations function that only uses TyCtxt.
• Special cases modules with doc(primitive) to be treated as always public for the purpose of links.
• Removes the fragment hack. cc @notriddle, I know you added some comments about this in the code (thank you for that!)

Breaking changes to doc(primitive)

"Breaking" is a little misleading here - these are changes in behavior, none of them will cause code to fail to compile.

Let me preface this by saying I think stabilizing doc(primitive) was a uniquely terrible idea. As far as I can tell, it was stabilized by oversight; it's been stable since 1.0. No one should have need to use it except the standard library, and a crater run shows that in fact no one is using it: rust-lang#87050 (comment). I hope to actually make doc(primitive) a no-op unless you opt-in with a nightly feature, which will keep crates compiling without forcing rustdoc into trying to keep somewhat arbitrary behavior guarantees; but for now, this just subtly changes some of the behavior if you use doc(primitive) in a dependency.

That said, here are the changes:

• Refactoring out primitive_locations() is technically a change in behavior, since it no longer looks for primitives in crates that were passed through --extern, but not used by the crate; however, that seems like such an unlikely edge case it's not worth dealing with.
• The precedence given to primitive locations is no longer just arbitrary, it can also be inconsistent from run to run. Let me explain that more: previously, primitive locations were sorted by the CrateNum; the comment on that sort said "Favor linking to as local extern as possible, so iterate all crates in reverse topological order." Unfortunately, that's not actually what CrateNum tracks: it measures the order crates are loaded, not the number of intermediate crates between that dependency and the root crate. It happened to work as intended before because the compiler injects extern crate std; at the top of every crate, which ensured it would have the first CrateNum other than the current, but every other CrateNum was completely arbitrary (for example, core often had a later CrateNum than std). This now removes the sort on CrateNum completely and special-cases core instead. In particular, if you depend on both std and a crate which defines a doc(primitive) module, it's arbitrary whether rustdoc will use the docs from std or the ones from the other crate. cc @alexcrichton, you wrote this originally.

cc @rust-lang/rustdoc cc @rust-lang/libs for the addition to core (the commit you're interested in is rust-lang@36729b0)

jyn514 deleted the no-doc-primitive branch

September 9, 2021 02:05

bors added a commit to rust-lang-ci/rust that referenced this pull request

Sep 12, 2021

…ez,jyn514

Fix rustdoc handling of primitive items

This is a complicated PR and does a lot of things. I'm willing to split it up a little more if it would help reviewing, but it would be tricky and I'd rather not unless it's necessary.

What does this do?

• Fixes rust-lang#73423.
• Fixes rust-lang#79630. I'm not sure how to test this for the standard library explicitly, but you can see from some of the diffs from the no_std tests. I also tested it locally and it works correctly:
• Fixes rust-lang#83083.

Why are these changes interconnected?

• Allowing anchors (rust-lang#83083) without fixing the online/offline problem (rust-lang#79630) will actually just silently discard the anchors, that's not a fix. The online/offline problem is directly related to the fragment hack; links need to go through fn href() to be fixed.
• Technically I could fix the online/offline problem without removing the error on anchors; I am willing to separate that out if it would be helpful for reviewing. However I can't fix the anchor problem without adding docs to core, since rustdoc needs all those primitives to have docs to avoid a fallback, and currently #![no_std] crates don't have docs for primitives. I also can't fix the online/offline problem without removing the fragment hack, since otherwise diffs like this will be wrong for some primitives but not others:

`@@` -385,7 +381,7 `@@` fn resolve_primitive_associated_item(
                         ty::AssocKind::Const => "associatedconstant",
                         ty::AssocKind::Type => "associatedtype",
                     };
-                    let fragment = format!("{}#{}.{}", prim_ty.as_sym(), out, item_name);
+                    let fragment = format!("{}.{}", out, item_name);
                     (Res::Primitive(prim_ty), fragment, Some((kind.as_def_kind(), item.def_id)))
                 })
         })

• Adding primitive docs to core without making any other change will cause links to go to core instead of std, even for crates with extern crate std. See "Breaking changes to doc(primitive)" below for why this is the case. That said, I could add some special casing to rustdoc at the same time that would let me separate this change from the others (it would fix rust-lang#73423 but still special-case intra-doc links). I'm willing to separate that out if helpful for reviewing.

Add primitive documentation to libcore

This works by reusing the same include!("primitive_docs.rs") file in both core and std, and then special-casing links in core to use relative links instead of intra-doc links. This doesn't use purely intra-doc links because some of the primitive docs links to items only in std; this doesn't use purely relative links because that introduces new broken links when the docs are re-exported (e.g. String's &str deref impl, or Vec's slice deref impl).

Note that this copies the whole file to core, to avoid anyone compiling core to have to set CARGO_PKG_NAME. See https://rust-lang.zulipchat.com/#narrow/stream/122651-general/topic/Who.20should.20review.20changes.20to.20linkchecker.3F/near/249939598 for more context. It also adds a tidy check to make sure the two files are kept in sync.

Fix inconsistent online/offline primitive docs

This does four things:

• Records modules with doc(primitive) in cache.external_paths. This is necessary for href() to find them later.
• Makes cache.primitive_locations available to the intra-doc link pass, by refactoring out a PrimitiveType::primitive_locations function that only uses TyCtxt.
• Special cases modules with doc(primitive) to be treated as always public for the purpose of links.
• Removes the fragment hack. cc @notriddle, I know you added some comments about this in the code (thank you for that!)

Breaking changes to doc(primitive)

"Breaking" is a little misleading here - these are changes in behavior, none of them will cause code to fail to compile.

Let me preface this by saying I think stabilizing doc(primitive) was a uniquely terrible idea. As far as I can tell, it was stabilized by oversight; it's been stable since 1.0. No one should have need to use it except the standard library, and a crater run shows that in fact no one is using it: rust-lang#87050 (comment). I hope to actually make doc(primitive) a no-op unless you opt-in with a nightly feature, which will keep crates compiling without forcing rustdoc into trying to keep somewhat arbitrary behavior guarantees; but for now, this just subtly changes some of the behavior if you use doc(primitive) in a dependency.

That said, here are the changes:

• Refactoring out primitive_locations() is technically a change in behavior, since it no longer looks for primitives in crates that were passed through --extern, but not used by the crate; however, that seems like such an unlikely edge case it's not worth dealing with.
• The precedence given to primitive locations is no longer just arbitrary, it can also be inconsistent from run to run. Let me explain that more: previously, primitive locations were sorted by the CrateNum; the comment on that sort said "Favor linking to as local extern as possible, so iterate all crates in reverse topological order." Unfortunately, that's not actually what CrateNum tracks: it measures the order crates are loaded, not the number of intermediate crates between that dependency and the root crate. It happened to work as intended before because the compiler injects extern crate std; at the top of every crate, which ensured it would have the first CrateNum other than the current, but every other CrateNum was completely arbitrary (for example, core often had a later CrateNum than std). This now removes the sort on CrateNum completely and special-cases core instead. In particular, if you depend on both std and a crate which defines a doc(primitive) module, it's arbitrary whether rustdoc will use the docs from std or the ones from the other crate. cc @alexcrichton, you wrote this originally.

cc @rust-lang/rustdoc cc @rust-lang/libs for the addition to core (the commit you're interested in is rust-lang@91346c8)

wip-sync pushed a commit to NetBSD/pkgsrc-wip that referenced this pull request

Oct 22, 2021

Pkgsrc changes:

• Remove one now-longer-applicable patch, adjust a few others
• Bump bootstrap requirements to 1.55.0.

Upstream changes:

Version 1.56.0 (2021-10-21)

Language

• The 2021 Edition is now stable. See the edition guide for more details.
• [The pattern in binding @ pattern can now also introduce new bindings.] rust#85305
• [Union field access is permitted in const fn.][rust#85769]

Compiler

• Upgrade to LLVM 13.
• Support memory, address, and thread sanitizers on aarch64-unknown-freebsd.
• Allow specifying a deployment target version for all iOS targets
• Warnings can be forced on with --force-warn. This feature is primarily intended for usage by cargo fix, rather than end users.
• Promote aarch64-apple-ios-sim to Tier 2*.
• Add powerpc-unknown-freebsd at Tier 3*.
• [Add riscv32imc-esp-espidf at Tier 3*.][rust#87666]

* Refer to Rust's [platform support page][platform-support-doc] for more information on Rust's tiered platform support.

Libraries

• [Allow writing of incomplete UTF-8 sequences via stdout/stderr on Windows.] rust#83342 The Windows console still requires valid Unicode, but this change allows splitting a UTF-8 character across multiple write calls. This allows, for instance, programs that just read and write data buffers (e.g. copying a file to stdout) without regard for Unicode or character boundaries.
• [Prefer AtomicU{64,128} over Mutex for Instant backsliding protection.] rust#83093 For this use case, atomics scale much better under contention.
• Implement Extend<(A, B)> for (Extend<A>, Extend<B>)
• impl Default, Copy, Clone for std::io::Sink and std::io::Empty
• impl From<[(K, V); N]> for all collections.
• Remove P: Unpin bound on impl Future for Pin.
• Treat invalid environment variable names as non-existent. Previously, the environment functions would panic if given a variable name with an internal null character or equal sign (=). Now, these functions will just treat such names as non-existent variables, since the OS cannot represent the existence of a variable with such a name.

Stabilised APIs

• std::os::unix::fs::chroot
• UnsafeCell::raw_get
• BufWriter::into_parts
• core::panic::{UnwindSafe, RefUnwindSafe, AssertUnwindSafe} These APIs were previously stable in std, but are now also available in core.
• Vec::shrink_to
• String::shrink_to
• OsString::shrink_to
• PathBuf::shrink_to
• BinaryHeap::shrink_to
• VecDeque::shrink_to
• HashMap::shrink_to
• HashSet::shrink_to

These APIs are now usable in const contexts:

• std::mem::transmute
• [T]::first
• [T]::split_first
• [T]::last
• [T]::split_last

Cargo

• [Cargo supports specifying a minimum supported Rust version in Cargo.toml.] rust-version This has no effect at present on dependency version selection. We encourage crates to specify their minimum supported Rust version, and we encourage CI systems that support Rust code to include a crate's specified minimum version in the text matrix for that crate by default.

Compatibility notes

• Update to new argument parsing rules on Windows. This adjusts Rust's standard library to match the behavior of the standard libraries for C/C++. The rules have changed slightly over time, and this PR brings us to the latest set of rules (changed in 2008).
• [Disallow the aapcs calling convention on aarch64][rust#88399] This was already not supported by LLVM; this change surfaces this lack of support with a better error message.
• Make SEMICOLON_IN_EXPRESSIONS_FROM_MACROS warn by default
• Warn when an escaped newline skips multiple lines.
• [Calls to libc::getpid / std::process::id from Command::pre_exec may return different values on glibc <= 2.24.][rust#81825] Rust now invokes the clone3 system call directly, when available, to use new functionality available via that system call. Older versions of glibc cache the result of getpid, and only update that cache when calling glibc's clone/fork functions, so a direct system call bypasses that cache update. glibc 2.25 and newer no longer cache getpid for exactly this reason.

Internal changes

These changes provide no direct user facing benefits, but represent significant improvements to the internals and overall performance of rustc and related tools.

• LLVM is compiled with PGO in published x86_64-unknown-linux-gnu artifacts. This improves the performance of most Rust builds.

• [Unify representation of macros in internal data structures.][rust#88019] This change fixes a host of bugs with the handling of macros by the compiler, as well as rustdoc.

[rust#85769]: rust-lang/rust#85769 (comment) [rust#88490]: rust-lang/rust#88490 [rust#88269]: rust-lang/rust#88269 [rust#84176]: rust-lang/rust#84176 [rust#88399]: rust-lang/rust#88399 [rust#88227]: rust-lang/rust#88227 [rust#88200]: rust-lang/rust#88200 [rust#82776]: rust-lang/rust#82776 [rust#88077]: rust-lang/rust#88077 [rust#87728]: rust-lang/rust#87728 [rust#87050]: rust-lang/rust#87050 [rust#87619]: rust-lang/rust#87619 [rust#81825]: rust-lang/rust#81825 (comment) [rust#88019]: rust-lang/rust#88019 [rust#87666]: rust-lang/rust#87666

netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this pull request

Nov 20, 2021

Pkgsrc changes:

• Bump bootstrap kit version to 1.55.0.
• Adjust patches as needed, some no longer apply (so removed)
• Update checksum adjustments.
• Avoid rust-llvm on SunOS
• Optionally build docs
• Remove reference to closed/old PR#54621

Upstream changes:

Version 1.56.1 (2021-11-01)

• New lints to detect the presence of bidirectional-override Unicode codepoints in the compiled source code (CVE-2021-42574)

Version 1.56.0 (2021-10-21)

Language

• The 2021 Edition is now stable. See the edition guide for more details.
• [The pattern in binding @ pattern can now also introduce new bindings.] rust#85305
• [Union field access is permitted in const fn.][rust#85769]

Compiler

• Upgrade to LLVM 13.
• [Support memory, address, and thread sanitizers on aarch64-unknown-freebsd.] rust#88023
• Allow specifying a deployment target version for all iOS targets
• Warnings can be forced on with --force-warn. This feature is primarily intended for usage by cargo fix, rather than end users.
• Promote aarch64-apple-ios-sim to Tier 2*.
• Add powerpc-unknown-freebsd at Tier 3*.
• [Add riscv32imc-esp-espidf at Tier 3*.][rust#87666]

* Refer to Rust's [platform support page][platform-support-doc] for more information on Rust's tiered platform support.

Libraries

• [Allow writing of incomplete UTF-8 sequences via stdout/stderr on Windows.] rust#83342 The Windows console still requires valid Unicode, but this change allows splitting a UTF-8 character across multiple write calls. This allows, for instance, programs that just read and write data buffers (e.g. copying a file to stdout) without regard for Unicode or character boundaries.
• [Prefer AtomicU{64,128} over Mutex for Instant backsliding protection.] rust#83093 For this use case, atomics scale much better under contention.
• Implement Extend<(A, B)> for (Extend<A>, Extend<B>)
• impl Default, Copy, Clone for std::io::Sink and std::io::Empty
• impl From<[(K, V); N]> for all collections.
• Remove P: Unpin bound on impl Future for Pin.
• Treat invalid environment variable names as non-existent. Previously, the environment functions would panic if given a variable name with an internal null character or equal sign (=). Now, these functions will just treat such names as non-existent variables, since the OS cannot represent the existence of a variable with such a name.

Stabilised APIs

• std::os::unix::fs::chroot
• UnsafeCell::raw_get
• BufWriter::into_parts
• core::panic::{UnwindSafe, RefUnwindSafe, AssertUnwindSafe} These APIs were previously stable in std, but are now also available in core.
• Vec::shrink_to
• String::shrink_to
• OsString::shrink_to
• PathBuf::shrink_to
• BinaryHeap::shrink_to
• VecDeque::shrink_to
• HashMap::shrink_to
• HashSet::shrink_to

These APIs are now usable in const contexts:

• std::mem::transmute
• [T]::first
• [T]::split_first
• [T]::last
• [T]::split_last

Cargo

• [Cargo supports specifying a minimum supported Rust version in Cargo.toml.] rust-version This has no effect at present on dependency version selection. We encourage crates to specify their minimum supported Rust version, and we encourage CI systems that support Rust code to include a crate's specified minimum version in the text matrix for that crate by default.

Compatibility notes

• Update to new argument parsing rules on Windows. This adjusts Rust's standard library to match the behavior of the standard libraries for C/C++. The rules have changed slightly over time, and this PR brings us to the latest set of rules (changed in 2008).
• [Disallow the aapcs calling convention on aarch64][rust#88399] This was already not supported by LLVM; this change surfaces this lack of support with a better error message.
• Make SEMICOLON_IN_EXPRESSIONS_FROM_MACROS warn by default
• Warn when an escaped newline skips multiple lines.
• [Calls to libc::getpid / std::process::id from Command::pre_exec may return different values on glibc <= 2.24.][rust#81825] Rust now invokes the clone3 system call directly, when available, to use new functionality available via that system call. Older versions of glibc cache the result of getpid, and only update that cache when calling glibc's clone/fork functions, so a direct system call bypasses that cache update. glibc 2.25 and newer no longer cache getpid for exactly this reason.

Internal changes

These changes provide no direct user facing benefits, but represent significant improvements to the internals and overall performance of rustc and related tools.

• [LLVM is compiled with PGO in published x86_64-unknown-linux-gnu artifacts.] rust#88069 This improves the performance of most Rust builds.
• [Unify representation of macros in internal data structures.][rust#88019] This change fixes a host of bugs with the handling of macros by the compiler, as well as rustdoc.

[rust#85769]: rust-lang/rust#85769 (comment) [rust#88490]: rust-lang/rust#88490 [rust#88269]: rust-lang/rust#88269 [rust#84176]: rust-lang/rust#84176 [rust#88399]: rust-lang/rust#88399 [rust#88227]: rust-lang/rust#88227 [rust#88200]: rust-lang/rust#88200 [rust#82776]: rust-lang/rust#82776 [rust#88077]: rust-lang/rust#88077 [rust#87728]: rust-lang/rust#87728 [rust#87050]: rust-lang/rust#87050 [rust#87619]: rust-lang/rust#87619 [rust#81825]: rust-lang/rust#81825 (comment) [rust#88019]: rust-lang/rust#88019 [rust#87666]: rust-lang/rust#87666

Version 1.55.0 (2021-09-09)

Language

• You can now write open "from" range patterns (X..), which will start at X and will end at the maximum value of the integer.
• You can now explicitly import the prelude of different editions through std::prelude (e.g. use std::prelude::rust_2021::*;).

Compiler

• Added tier 3* support for powerpc64le-unknown-freebsd.

* Refer to Rust's [platform support page][platform-support-doc] for more information on Rust's tiered platform support.

Libraries

• Updated std's float parsing to use the Eisel-Lemire algorithm. These improvements should in general provide faster string parsing of floats, no longer reject certain valid floating point values, and reduce the produced code size for non-stripped artifacts.
• string::Drain now implements AsRef<str> and AsRef<[u8]>.

Stabilised APIs

• Bound::cloned
• Drain::as_str
• IntoInnerError::into_error
• IntoInnerError::into_parts
• MaybeUninit::assume_init_mut
• MaybeUninit::assume_init_ref
• MaybeUninit::write
• array::map
• ops::ControlFlow
• x86::_bittest
• x86::_bittestandcomplement
• x86::_bittestandreset
• x86::_bittestandset
• x86_64::_bittest64
• x86_64::_bittestandcomplement64
• x86_64::_bittestandreset64
• x86_64::_bittestandset64

The following previously stable functions are now const.

• str::from_utf8_unchecked

Cargo

• Cargo will now deduplicate compiler diagnostics to the terminal when invoking rustc in parallel such as when using cargo test.
• The package definition in cargo metadata now includes the "default_run" field from the manifest.
• Added cargo d as an alias for cargo doc.
• Added {lib} as formatting option for cargo tree to print the "lib_name" of packages.

Rustdoc

• Added "Go to item on exact match" search option.
• The "Implementors" section on traits no longer shows redundant method definitions.
• Trait implementations are toggled open by default. This should make the implementations more searchable by tools like CTRL+F in your browser.
• Intra-doc links should now correctly resolve associated items (e.g. methods) through type aliases.
• Traits which are marked with #[doc(hidden)] will no longer appear in the "Trait Implementations" section.

Compatibility Notes

• std functions that return an io::Error will no longer use the ErrorKind::Other variant. This is to better reflect that these kinds of errors could be categorised into newer more specific ErrorKind variants, and that they do not represent a user error.
• Using environment variable names with process::Command on Windows now behaves as expected. Previously using envionment variables with Command would cause them to be ASCII-uppercased.
• Rustdoc will now warn on using rustdoc lints that aren't prefixed with rustdoc::