rustdoc-json: add rlib path to ExternalCrate to enable robust crate resolution #149043

aDotInTheVoid · 2025-11-17T23:37:42Z

Historically, it's not been possible to robustly resolve a cross-crate item in rustdoc-json. If you had a Id that wasn't in Crate::index (because it was defined in a different crate), you could only look it up it Crate::paths. But there, you don't get the full information, only an ItemSummary. This tells you the path and the crate_id.

But knowing the crate_id isn't enough to be able to build/find the rustdoc-json output with this item. It's only use is to get a ExternalCrate (via Crate::external_crates). But that only tells you the name (as a string). This isn't enough to uniquely identify a crate, as there could be multiple versions/features ¹ ².

This was originally proposed to be solved via @LukeMathWalker's --orchestrator-id proposal (rust-lang/compiler-team#635). But that requires invasive changes to cargo/rustc. This PR instead implements @Urgau's proposal to re-use the path to a crate's rmeta/rlib as a unique identifer. Callers can use that to determine which package it corresponds to in the language of the build-system above rustc. E.g. for cargo, cargo rustdoc --message-format=json --output-format=json -Zunstable-options).

(Once you've found the right external crate's rustdoc-json output, you still need to resolve the path->id in that crate. But that's """just""" a matter of walking the module tree. We should probably still make that nicer (by, for example, allowing sharing Ids between rustdoc-json document), but that's a future concern)

For some notes from RustWeek 2025, where this was designed, see https://hackmd.io/0jkdguobTnW7nXoGKAxfEQ

CC @obi1kenobi (who wants this for cargo-semver-checks ³), @epage (who's conversations on what and wasn't possible with cargo informed taking this approach to solve this problem)

r? @GuillaumeGomez

TODO:

Docs: Done
Tests: Done

rustbot · 2025-11-17T23:37:46Z

rustdoc-json-types is a public (although nightly-only) API. If possible, consider changing src/librustdoc/json/conversions.rs; otherwise, make sure you bump the FORMAT_VERSION constant.

cc @CraftSpider, @aDotInTheVoid, @Enselic, @obi1kenobi

GuillaumeGomez · 2025-11-18T10:58:41Z

Makes sense to me! Ping me once PR is ready for final review. :)

This is useful for allowing writing run-make tests that test rustdoc-json.

rustbot · 2025-11-19T20:54:14Z

The run-make-support library was changed

cc @jieyouxu

These commits modify the Cargo.lock file. Unintentional changes to Cargo.lock can be introduced when switching branches and rebasing PRs.

If this was unintentional then you should revert the changes before this PR is merged.
Otherwise, you can ignore this comment.

aDotInTheVoid · 2025-11-19T20:55:14Z

@GuillaumeGomez ready for review.

GuillaumeGomez · 2025-11-19T21:18:59Z

Looks all good to me, thanks! r=me once CI pass

aDotInTheVoid · 2025-11-19T22:29:34Z

@bors r=GuillaumeGomez rollup

bors · 2025-11-19T22:29:37Z

📌 Commit 457ed4e has been approved by GuillaumeGomez

It is now in the queue for this repository.

…-out-of-spite-does-it-matter, r=GuillaumeGomez rustdoc-json: add rlib path to ExternalCrate to enable robust crate resolution Historically, it's not been possible to robustly resolve a cross-crate item in rustdoc-json. If you had a `Id` that wasn't in `Crate::index` (because it was defined in a different crate), you could only look it up it `Crate::paths`. But there, you don't get the full information, only an `ItemSummary`. This tells you the `path` and the `crate_id`. But knowing the `crate_id` isn't enough to be able to build/find the rustdoc-json output with this item. It's only use is to get a `ExternalCrate` (via `Crate::external_crates`). But that only tells you the `name` (as a string). This isn't enough to uniquely identify a crate, as there could be multiple versions/features [^1] [^2]. This was originally proposed to be solved via `@LukeMathWalker's` `--orchestrator-id` proposal (rust-lang/compiler-team#635). But that requires invasive changes to cargo/rustc. This PR instead implements `@Urgau's` proposal to re-use the path to a crate's rmeta/rlib as a unique identifer. Callers can use that to determine which package it corresponds to in the language of the build-system above rustc. E.g. for cargo, `cargo rustdoc --message-format=json --output-format=json -Zunstable-options`). (Once you've found the right external crate's rustdoc-json output, you still need to resolve the path->id in that crate. But that's """just""" a matter of walking the module tree. We should probably still make that nicer (by, for example, allowing sharing `Id`s between rustdoc-json document), but that's a future concern) For some notes from RustWeek 2025, where this was designed, see https://hackmd.io/0jkdguobTnW7nXoGKAxfEQ CC `@obi1kenobi` (who wants this for cargo-semver-checks [^3]), `@epage` (who's conversations on what and wasn't possible with cargo informed taking this approach to solve this problem) r? `@GuillaumeGomez` ## TODO: - [x] Docs: [Done](https://github.com/rust-lang/rust/compare/e4cdd0c24a994fed354081b5f907680a11f2ddc5..457ed4edb184997d5d6f879c6a220bc4d69ff6fd) - [x] Tests: [Done](https://github.com/rust-lang/rust/compare/2e1b954dc52bf7e5a6e9311394df760db37d383f..4d00c1a7ee5e03d1e78801cc01a85dac08ab603b) [^1]: rust-lang/compiler-team#635 (comment) § Problem [^2]: https://rust-lang.zulipchat.com/#narrow/channel/266220-t-rustdoc/topic/Identifying.20external.20crates.20in.20Rustdoc.20JSON/with/352701211 [^3]: obi1kenobi/cargo-semver-checks#638

rust-bors · 2025-11-20T23:11:15Z

☀️ Try build successful (CI)
Build commit: be66802 (be668027128df82bcb3a17cf267a40ec00593413, parent: 7281a3bc4b89a010aed05528d648450feb8b69d3)

aDotInTheVoid · 2025-11-21T00:00:50Z

@bors r=GuillaumeGomez rollup

bors · 2025-11-21T00:00:53Z

📌 Commit a47dff8 has been approved by GuillaumeGomez

It is now in the queue for this repository.

…-out-of-spite-does-it-matter, r=GuillaumeGomez rustdoc-json: add rlib path to ExternalCrate to enable robust crate resolution Historically, it's not been possible to robustly resolve a cross-crate item in rustdoc-json. If you had a `Id` that wasn't in `Crate::index` (because it was defined in a different crate), you could only look it up it `Crate::paths`. But there, you don't get the full information, only an `ItemSummary`. This tells you the `path` and the `crate_id`. But knowing the `crate_id` isn't enough to be able to build/find the rustdoc-json output with this item. It's only use is to get a `ExternalCrate` (via `Crate::external_crates`). But that only tells you the `name` (as a string). This isn't enough to uniquely identify a crate, as there could be multiple versions/features [^1] [^2]. This was originally proposed to be solved via `@LukeMathWalker's` `--orchestrator-id` proposal (rust-lang/compiler-team#635). But that requires invasive changes to cargo/rustc. This PR instead implements `@Urgau's` proposal to re-use the path to a crate's rmeta/rlib as a unique identifer. Callers can use that to determine which package it corresponds to in the language of the build-system above rustc. E.g. for cargo, `cargo rustdoc --message-format=json --output-format=json -Zunstable-options`). (Once you've found the right external crate's rustdoc-json output, you still need to resolve the path->id in that crate. But that's """just""" a matter of walking the module tree. We should probably still make that nicer (by, for example, allowing sharing `Id`s between rustdoc-json document), but that's a future concern) For some notes from RustWeek 2025, where this was designed, see https://hackmd.io/0jkdguobTnW7nXoGKAxfEQ CC `@obi1kenobi` (who wants this for cargo-semver-checks [^3]), `@epage` (who's conversations on what and wasn't possible with cargo informed taking this approach to solve this problem) r? `@GuillaumeGomez` ## TODO: - [x] Docs: [Done](https://github.com/rust-lang/rust/compare/e4cdd0c24a994fed354081b5f907680a11f2ddc5..457ed4edb184997d5d6f879c6a220bc4d69ff6fd) - [x] Tests: [Done](https://github.com/rust-lang/rust/compare/2e1b954dc52bf7e5a6e9311394df760db37d383f..4d00c1a7ee5e03d1e78801cc01a85dac08ab603b) [^1]: rust-lang/compiler-team#635 (comment) § Problem [^2]: https://rust-lang.zulipchat.com/#narrow/channel/266220-t-rustdoc/topic/Identifying.20external.20crates.20in.20Rustdoc.20JSON/with/352701211 [^3]: obi1kenobi/cargo-semver-checks#638

Rollup of 3 pull requests Successful merges: - #149043 ( rustdoc-json: add rlib path to ExternalCrate to enable robust crate resolution) - #149134 (std: sys: net: uefi: Implement read_vectored) - #149135 (Constify `residual_into_try_type`) r? `@ghost` `@rustbot` modify labels: rollup

Zalathar · 2025-11-21T03:41:57Z

Failed in rollup in aarch64-msvc-1: #149163 (comment)

@bors r-

aDotInTheVoid · 2025-11-21T09:17:21Z

@bors try jobs=dist-various-1,aarch64-msvc-1

…te-does-it-matter, r=<try> rustdoc-json: add rlib path to ExternalCrate to enable robust crate resolution try-job: dist-various-1 try-job: aarch64-msvc-1

…esolution Historically, it's not been possible to robustly resolve a cross-crate item in rustdoc-json. If you had a `Id` that wasn't in `Crate::index` (because it was defined in a different crate), you could only look it up it `Crate::paths`. But there, you don't get the full information, only an `ItemSummary`. This tells you the `path` and the `crate_id`. But knowing the `crate_id` isn't enough to be able to build/find the rustdoc-json output with this item. It's only use is to get a `ExternalCrate` (via `Crate::external_crates`). But that only tells you the `name` (as a string). This isn't enough to uniquely identify a crate, as there could be multiple versions/features [1] [2]. This was originally proposed to be solved via LukeMathWalker's `--orchestrator-id` proposal (https://www.github.com/rust-lang/compiler-team/issues/635). But that requires invasive changes to cargo/rustc. This PR instead implements Urgau's proposal to re-use the path to a crate's rmeta/rlib as a unique identifer. Callers can use that to determine which package it corresponds to in the language of the build-system above rustc. E.g. for cargo, `cargo rustdoc --message-format=json --output-format=json -Zunstable-options`). (Once you've found the right external crate's rustdoc-json output, you still need to resolve the path->id in that crate. But that's """just""" a matter of walking the module tree. We should probably still make that nicer (by, for example, allowing sharing `Id`s between rustdoc-json document), but that's a future concern) For some notes from RustWeek 2025, where this was designed, see https://hackmd.io/0jkdguobTnW7nXoGKAxfEQ [1]: https://www.github.com/rust-lang/compiler-team/issues/635#issue-1714254865 § Problem [2]: https://rust-lang.zulipchat.com/#narrow/channel/266220-t-rustdoc/topic/Identifying.20external.20crates.20in.20Rustdoc.20JSON/with/352701211

rust-bors · 2025-11-21T11:28:57Z

☀️ Try build successful (CI)
Build commit: a0d82d0 (a0d82d01eacf051b4a3e51b8a2ec059ec6bbd8f3, parent: e22dab387f6b4f6a87dfc54ac2f6013dddb41e68)

aDotInTheVoid · 2025-11-21T13:39:26Z

@bors r=GuillaumeGomez rollup

bors · 2025-11-21T13:39:29Z

📌 Commit 361af82 has been approved by GuillaumeGomez

It is now in the queue for this repository.

Rollup of 7 pull requests Successful merges: - #146978 (Emit error when using path-segment keyword as cfg pred) - #148719 (Allow unnormalized types in drop elaboration) - #148795 (add `rust.rustflags` and per target `rustflags` options to `bootstrap.toml`) - #149028 ([rustdoc] Remove `UrlFragment::render` method to unify `clean::types::links` and `anchor`) - #149043 ( rustdoc-json: add rlib path to ExternalCrate to enable robust crate resolution) - #149098 (Fix error message for calling a non-tuple struct) - #149151 (Add test for importing path-segment keyword) r? `@ghost` `@rustbot` modify labels: rollup

Rollup merge of #149043 - aDotInTheVoid:is-this-real-is-this-out-of-spite-does-it-matter, r=GuillaumeGomez rustdoc-json: add rlib path to ExternalCrate to enable robust crate resolution Historically, it's not been possible to robustly resolve a cross-crate item in rustdoc-json. If you had a `Id` that wasn't in `Crate::index` (because it was defined in a different crate), you could only look it up it `Crate::paths`. But there, you don't get the full information, only an `ItemSummary`. This tells you the `path` and the `crate_id`. But knowing the `crate_id` isn't enough to be able to build/find the rustdoc-json output with this item. It's only use is to get a `ExternalCrate` (via `Crate::external_crates`). But that only tells you the `name` (as a string). This isn't enough to uniquely identify a crate, as there could be multiple versions/features [^1] [^2]. This was originally proposed to be solved via `@LukeMathWalker's` `--orchestrator-id` proposal (rust-lang/compiler-team#635). But that requires invasive changes to cargo/rustc. This PR instead implements `@Urgau's` proposal to re-use the path to a crate's rmeta/rlib as a unique identifer. Callers can use that to determine which package it corresponds to in the language of the build-system above rustc. E.g. for cargo, `cargo rustdoc --message-format=json --output-format=json -Zunstable-options`). (Once you've found the right external crate's rustdoc-json output, you still need to resolve the path->id in that crate. But that's """just""" a matter of walking the module tree. We should probably still make that nicer (by, for example, allowing sharing `Id`s between rustdoc-json document), but that's a future concern) For some notes from RustWeek 2025, where this was designed, see https://hackmd.io/0jkdguobTnW7nXoGKAxfEQ CC `@obi1kenobi` (who wants this for cargo-semver-checks [^3]), `@epage` (who's conversations on what and wasn't possible with cargo informed taking this approach to solve this problem) r? `@GuillaumeGomez` ## TODO: - [x] Docs: [Done](https://github.com/rust-lang/rust/compare/e4cdd0c24a994fed354081b5f907680a11f2ddc5..457ed4edb184997d5d6f879c6a220bc4d69ff6fd) - [x] Tests: [Done](https://github.com/rust-lang/rust/compare/2e1b954dc52bf7e5a6e9311394df760db37d383f..4d00c1a7ee5e03d1e78801cc01a85dac08ab603b) [^1]: rust-lang/compiler-team#635 (comment) § Problem [^2]: https://rust-lang.zulipchat.com/#narrow/channel/266220-t-rustdoc/topic/Identifying.20external.20crates.20in.20Rustdoc.20JSON/with/352701211 [^3]: obi1kenobi/cargo-semver-checks#638

rustbot added the A-rustdoc-json Area: Rustdoc JSON backend label Nov 17, 2025

rustbot assigned GuillaumeGomez Nov 17, 2025

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue. labels Nov 17, 2025

This comment was marked as outdated.

Sign in to view

aDotInTheVoid marked this pull request as draft November 17, 2025 23:48

rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Nov 17, 2025

rust-cloud-vms bot force-pushed the is-this-real-is-this-out-of-spite-does-it-matter branch from c7f4e67 to 2e1b954 Compare November 17, 2025 23:55

run-make-support: re-export rustdoc_json_types

8c569c1

This is useful for allowing writing run-make tests that test rustdoc-json.

rust-cloud-vms bot force-pushed the is-this-real-is-this-out-of-spite-does-it-matter branch from 2e1b954 to 4d00c1a Compare November 19, 2025 19:36

rustbot added the A-run-make Area: port run-make Makefiles to rmake.rs label Nov 19, 2025

rust-cloud-vms bot force-pushed the is-this-real-is-this-out-of-spite-does-it-matter branch 2 times, most recently from e4cdd0c to 457ed4e Compare November 19, 2025 20:53

aDotInTheVoid marked this pull request as ready for review November 19, 2025 20:54

rustbot added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Nov 19, 2025

rustbot removed the S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. label Nov 19, 2025

aDotInTheVoid changed the title ~~rustdoc-json: add rlib path to ExternalCrate to enable robust crate resolution [WIPWIPWIP]~~ rustdoc-json: add rlib path to ExternalCrate to enable robust crate resolution Nov 19, 2025

GuillaumeGomez approved these changes Nov 19, 2025

View reviewed changes

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Nov 19, 2025

Zalathar mentioned this pull request Nov 20, 2025

Rollup of 9 pull requests #149119

Closed

This comment has been minimized.

Sign in to view

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Nov 21, 2025

matthiaskrgr mentioned this pull request Nov 21, 2025

Rollup of 3 pull requests #149163

Closed

bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Nov 21, 2025

rust-cloud-vms bot force-pushed the is-this-real-is-this-out-of-spite-does-it-matter branch from a47dff8 to 2229329 Compare November 21, 2025 09:16

This comment has been minimized.

Sign in to view

rust-cloud-vms bot force-pushed the is-this-real-is-this-out-of-spite-does-it-matter branch from 2229329 to 361af82 Compare November 21, 2025 09:23

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Nov 21, 2025

GuillaumeGomez mentioned this pull request Nov 21, 2025

Rollup of 7 pull requests #149196

Merged

bors merged commit 629a283 into rust-lang:main Nov 21, 2025
11 checks passed

rustbot added this to the 1.93.0 milestone Nov 21, 2025

obi1kenobi mentioned this pull request Nov 23, 2025

Continue resolving cargo-semver-checks blockers for merging into cargo rust-lang/rust-project-goals#104

Open

10 tasks

rustdoc-json: add rlib path to ExternalCrate to enable robust crate resolution #149043

rustdoc-json: add rlib path to ExternalCrate to enable robust crate resolution #149043

Conversation

aDotInTheVoid commented Nov 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

TODO:

Footnotes

Uh oh!

rustbot commented Nov 17, 2025

Uh oh!

This comment was marked as outdated.

GuillaumeGomez commented Nov 18, 2025

Uh oh!

rustbot commented Nov 19, 2025

Uh oh!

aDotInTheVoid commented Nov 19, 2025

Uh oh!

GuillaumeGomez commented Nov 19, 2025

Uh oh!

aDotInTheVoid commented Nov 19, 2025

Uh oh!

bors commented Nov 19, 2025

Uh oh!

This comment has been minimized.

rust-bors bot commented Nov 20, 2025

Uh oh!

aDotInTheVoid commented Nov 21, 2025

Uh oh!

bors commented Nov 21, 2025

Uh oh!

Zalathar commented Nov 21, 2025

Uh oh!

aDotInTheVoid commented Nov 21, 2025

Uh oh!

This comment has been minimized.

This comment has been minimized.

rust-bors bot commented Nov 21, 2025

Uh oh!

aDotInTheVoid commented Nov 21, 2025

Uh oh!

bors commented Nov 21, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

aDotInTheVoid commented Nov 17, 2025 •

edited

Loading