Skip to content

rustdoc-search: adjust packing heuristics to save more space #145637

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

notriddle
Copy link
Contributor

@notriddle notriddle commented Aug 19, 2025

This upgrade causes stringdex to use fewer, larger files with the suffix-only children included more often. This is particularly aimed at reducing the number of files required by many-assoc-items.

The cost of this change is that the browser has to download more data that it isn't using while walking the search tree, because branches of the search tree that it isn't using are in the files it downloads. When I tested against the wordnet dictionary, a search for "indexing" required 12MiB of transfer, where it used to require 8MiB.

#144476 (comment)

@rustbot rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue. labels Aug 19, 2025
@notriddle
Copy link
Contributor Author

@bors try @rust-timer queue

@rust-timer

This comment has been minimized.

@rust-bors

This comment has been minimized.

rust-bors bot added a commit that referenced this pull request Aug 20, 2025
rustdoc-search: adjust packing heuristics to save more space
@rustbot rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Aug 20, 2025
@rust-bors
Copy link

rust-bors bot commented Aug 20, 2025

☀️ Try build successful (CI)
Build commit: b7f830d (b7f830d29248bb15ffccc37d1d5ddb504b80808e, parent: 05f5a58e84a9c3a68586d70bf3d7442c571e379e)

@rust-timer

This comment has been minimized.

@rust-timer
Copy link
Collaborator

Finished benchmarking commit (b7f830d): comparison URL.

Overall result: ❌ regressions - please read the text below

Benchmarking this pull request means it may be perf-sensitive – we'll automatically label it not fit for rolling up. You can override this, but we strongly advise not to, due to possible changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please do so in sufficient writing along with @rustbot label: +perf-regression-triaged. If not, please fix the regressions and do another perf run. If its results are neutral or positive, the label will be automatically removed.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

mean range count
Regressions ❌
(primary)
5.1% [0.5%, 41.2%] 10
Regressions ❌
(secondary)
12.4% [1.2%, 34.2%] 3
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
- - 0
All ❌✅ (primary) 5.1% [0.5%, 41.2%] 10

Max RSS (memory usage)

Results (primary -3.9%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

mean range count
Regressions ❌
(primary)
- - 0
Regressions ❌
(secondary)
- - 0
Improvements ✅
(primary)
-3.9% [-3.9%, -3.9%] 1
Improvements ✅
(secondary)
- - 0
All ❌✅ (primary) -3.9% [-3.9%, -3.9%] 1

Cycles

Results (primary 18.2%, secondary 14.0%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

mean range count
Regressions ❌
(primary)
18.2% [18.2%, 18.2%] 1
Regressions ❌
(secondary)
14.0% [14.0%, 14.0%] 1
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
- - 0
All ❌✅ (primary) 18.2% [18.2%, 18.2%] 1

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 472.158s -> 470.896s (-0.27%)
Artifact size: 378.09 MiB -> 378.09 MiB (-0.00%)

@rustbot rustbot added perf-regression Performance regression. and removed S-waiting-on-perf Status: Waiting on a perf run to be completed. labels Aug 20, 2025
This upgrade causes stringdex to use fewer, larger files with the
suffix-only children included more often. This is particularly aimed
at reducing the number of files required by many-assoc-items.

The cost of this change is that the browser has to download more
data that it isn't using while walking the search tree, because
branches of the search tree that it isn't using are in the files
it downloads. When I tested against the wordnet dictionary,
"indexing" required 12MiB of transfer, where it used to require 8MiB.
@notriddle
Copy link
Contributor Author

@bors try @rust-timer queue

@rust-timer

This comment has been minimized.

@rust-bors

This comment has been minimized.

rust-bors bot added a commit that referenced this pull request Aug 22, 2025
rustdoc-search: adjust packing heuristics to save more space
@rustbot rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Aug 22, 2025
@rust-bors
Copy link

rust-bors bot commented Aug 22, 2025

☀️ Try build successful (CI)
Build commit: e946368 (e946368b895f934127080fbcdab45309179161e0, parent: d20509c2a0c71b60aa2b51566e4d14920e8a1661)

@rust-timer

This comment has been minimized.

@rust-timer
Copy link
Collaborator

Finished benchmarking commit (e946368): comparison URL.

Overall result: ❌ regressions - please read the text below

Benchmarking this pull request means it may be perf-sensitive – we'll automatically label it not fit for rolling up. You can override this, but we strongly advise not to, due to possible changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please do so in sufficient writing along with @rustbot label: +perf-regression-triaged. If not, please fix the regressions and do another perf run. If its results are neutral or positive, the label will be automatically removed.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

mean range count
Regressions ❌
(primary)
0.5% [0.4%, 0.7%] 6
Regressions ❌
(secondary)
2.5% [1.2%, 3.8%] 2
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
- - 0
All ❌✅ (primary) 0.5% [0.4%, 0.7%] 6

Max RSS (memory usage)

Results (secondary -0.2%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

mean range count
Regressions ❌
(primary)
- - 0
Regressions ❌
(secondary)
5.1% [5.1%, 5.1%] 1
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
-2.9% [-3.6%, -2.1%] 2
All ❌✅ (primary) - - 0

Cycles

Results (secondary 2.7%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

mean range count
Regressions ❌
(primary)
- - 0
Regressions ❌
(secondary)
2.7% [2.7%, 2.7%] 1
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
- - 0
All ❌✅ (primary) - - 0

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 468.202s -> 466.275s (-0.41%)
Artifact size: 378.26 MiB -> 378.24 MiB (-0.01%)

@rustbot rustbot removed the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Aug 23, 2025
@notriddle
Copy link
Contributor Author

Okay, that seems like an acceptable tradeoff for the doc_files_count that this PR brings.

r? @GuillaumeGomez

@notriddle notriddle marked this pull request as ready for review August 23, 2025 01:15
@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Aug 23, 2025
@rustbot
Copy link
Collaborator

rustbot commented Aug 23, 2025

These commits modify the Cargo.lock file. Unintentional changes to Cargo.lock can be introduced when switching branches and rebasing PRs.

If this was unintentional then you should revert the changes before this PR is merged.
Otherwise, you can ignore this comment.

@GuillaumeGomez
Copy link
Member

So it reduces the number of files on the disk but increases the network data usage? Not sure it's a good trace. :-/

Also, the previous PR already had up to 50% more instructions, so adding even more on top of it really doesn't seem that great.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
perf-regression Performance regression. S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants