Add `read_buf` equivalents for positioned reads #140459

niklasf · 2025-04-29T13:54:42Z

Adds the following items under the ~~read_buf (#78485)~~ read_buf_at (#140771) feature:

std::os::unix::fs::FileExt::read_buf_at
std::os::unix::fs::FileExt::read_buf_exact_at
std::os::windows::fs::FileExt::seek_read_buf

try-job: x86_64-msvc*
try-job: test-various*
try-job: dist-various*

rustbot · 2025-04-29T13:54:54Z

rustbot has assigned @tgross35.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

tgross35 · 2025-05-01T00:54:44Z

Have these changes been discussed with libs-api at all? Usually changes to unstable API need to be proposed at https://github.com/rust-lang/libs-team/ with the ACP issue template.

This needs some tests as well.

tgross35 · 2025-05-01T01:42:49Z

For the above,
@rustbot author

rustbot · 2025-05-01T01:42:54Z

Reminder, once the PR becomes ready for a review, use @rustbot ready.

Kivooeo · 2025-05-08T04:28:04Z

For consistency and clarity, could we include safety comments on each unsafe block? Even brief notes are helpful to understand the assumptions being made

niklasf · 2025-05-12T23:13:31Z

Thank you both.

Opened an APC that was accepted, but items to be added under a new feature instead of read_buf. Moved to read_buf_at with new tracking issue Tracking Issue for read_buf_at #140771.
Added safety comments also for FFI calls.
Added tests.

@rustbot ready

library/std/src/sys/fd/unix.rs

library/std/src/os/unix/fs.rs

library/std/src/sys/fd/unix.rs

tgross35 · 2025-05-28T22:38:03Z

@bors try

Add `read_buf` equivalents for positioned reads Adds the following items under the ~~`read_buf` (#78485)~~ `read_buf_at` (#140771) feature: - `std::os::unix::fs::FileExt::read_buf_at` - `std::os::unix::fs::FileExt::read_buf_exact_at` - `std::os::windows::fs::FileExt::seek_read_buf` try-job: x86_64-msvc-1 try-job: x86_64-msvc-2

bors · 2025-05-29T01:03:19Z

💔 Test failed - checks-actions

niklasf · 2025-05-29T18:20:14Z

Ready for the next round of review.

The failed test on Windows was an assertion that I had (blindly) added about the seek position after seek_read_buf() with an empty buffer. Turns out it's a noop, but neither the existing seek_read() nor the current underlying Windows API guarantees it (afaict), so I just removed the assertion.

@rustbot ready

tgross35 · 2025-06-10T04:12:12Z

library/std/src/sys/pal/windows/handle.rs

+        // SAFETY: `read` bytes were written to the initialized portion of the buffer
+        unsafe {
+            cursor.advance_unchecked(read);
+        }


Do we need the same handling for ErrorKind::BrokenPipe as with read_buf?

If so, these functions are pretty identical. Maybe we should only have read_buf_at(self, cursor, offset: Option<u64>), and instead of read_buf call read_buf_at(..., None)?

As always, cc @ChrisDenton for Windows changes

It makes sense to me to unify the implementations but pipes aren't seekable so this probably isn't a practical concern. It just avoids unnecessarily repeating ourselves.

I think this error code is only produced on pipes, where read_at and read_buf_at do not make sense in any case.

After weeks, we both respond in the same minute 😆

Will combine them then, after all.

Reverted again. type AnonPipe = Handle makes it a bit awkward to push usage of a deduplicated read_buf_at(..., offset: Option<u64>) all the way down.

library/std/src/os/unix/fs.rs

library/std/src/fs/tests.rs

tgross35 · 2025-06-10T04:17:56Z

Running another try, including a few other unix platforms

@bors2 try

rust-bors · 2025-06-10T04:18:00Z

⌛ Trying commit e8a35d4 with merge 2e522c2…

To cancel the try build, run the command @bors2 try cancel.

Add `read_buf` equivalents for positioned reads Adds the following items under the ~~`read_buf` (#78485)~~ `read_buf_at` (#140771) feature: - `std::os::unix::fs::FileExt::read_buf_at` - `std::os::unix::fs::FileExt::read_buf_exact_at` - `std::os::windows::fs::FileExt::seek_read_buf` try-job: `x86_64-msvc*` try-job: `test-various*` try-job: `dist-various*`

tgross35 · 2025-06-10T04:20:56Z

library/std/src/sys/fd/unix.rs

+crate::cfg_select! {
+    any(
+        all(target_os = "linux", not(target_env = "musl")),
+        target_os = "android",
+        target_os = "hurd",
+    ) => {
+        use libc::pread64;
+    }
+    _ => {
+        use libc::pread as pread64;
+    }
+}


This should move to the top with the other imports (mentioned this in another comment), but actually do we need this remapping? From what I can gather at https://pubs.opengroup.org/onlinepubs/7908799/xsh/read.html and https://linux.die.net/man/2/pread64, pread is the POSIX name and Linux just changed its syscalls.

Sorry, missed the auto-collapsed response earlier. Moved to the top.

Looks like they are identical up to off_t vs. off64_t.

unsafe fn pread(fd: c_int, buf: *mut c_void, count: size_t, offset: off_t) -> ssize_t; unsafe fn pread64(fd: c_int, buf: *mut c_void, count: size_t, offset: off64_t) -> ssize_t;

So it comes down to #[cfg(gnu_file_offset_bits64)], which justifies explicitly picking pread64, I guess? Added a comment to that effect.

tgross35 · 2025-06-10T05:07:11Z

@rustbot author

rust-bors · 2025-06-10T06:47:27Z

☀️ Try build successful (CI)
Build commit: 2e522c2 (2e522c2e8590b7b074d7a35dea51085165d83c67)

tgross35 · 2025-08-13T01:49:07Z

Hey @niklasf gentle ping: this only needs a few small changes if you're able to get it over the line

bors · 2025-08-13T11:15:40Z

☔ The latest upstream changes (presumably #145334) made this pull request unmergeable. Please resolve the merge conflicts.

rustbot · 2025-08-18T19:24:31Z

This PR was rebased onto a different master commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

Adds the following items under the `read_buf_at` feature: - `std::os::unix::fs::FileExt::read_buf_at` - `std::os::unix::fs::FileExt::read_buf_exact_at` - `std::os::windows::fs::FileExt::seek_read_buf`

niklasf · 2025-08-18T20:21:22Z

Back after a while, rebased to resolve conflicts, and ready for the next round of review.

@rustbot ready

rustbot assigned tgross35 Apr 29, 2025

rustbot added O-unix Operating system: Unix-like O-windows Operating system: Windows S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-libs Relevant to the library team, which will review and decide on the PR/issue. labels Apr 29, 2025

This comment has been minimized.

Sign in to view

niklasf closed this Apr 29, 2025

niklasf force-pushed the feature/read-buf-at branch from 74e016c to 2ccb45f Compare April 29, 2025 14:07

niklasf reopened this Apr 29, 2025

rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels May 1, 2025

niklasf mentioned this pull request May 7, 2025

Tracking Issue for read_buf_at #140771

Open

4 tasks

niklasf force-pushed the feature/read-buf-at branch from 2ccb45f to 09ac65a Compare May 12, 2025 21:45

This comment has been minimized.

Sign in to view

niklasf force-pushed the feature/read-buf-at branch from 09ac65a to a22d2b4 Compare May 12, 2025 22:06

This comment has been minimized.

Sign in to view

niklasf force-pushed the feature/read-buf-at branch from a22d2b4 to 59cb58e Compare May 12, 2025 22:30

This comment has been minimized.

Sign in to view

niklasf force-pushed the feature/read-buf-at branch from 59cb58e to 85e1da1 Compare May 12, 2025 22:41

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels May 12, 2025

tgross35 reviewed May 28, 2025

View reviewed changes

library/std/src/sys/fd/unix.rs Outdated Show resolved Hide resolved

library/std/src/os/unix/fs.rs Show resolved Hide resolved

library/std/src/sys/fd/unix.rs Outdated Show resolved Hide resolved

bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels May 29, 2025

niklasf force-pushed the feature/read-buf-at branch from 85e1da1 to 05dcef0 Compare May 29, 2025 18:08

This comment has been minimized.

Sign in to view

niklasf force-pushed the feature/read-buf-at branch from 05dcef0 to e8a35d4 Compare May 29, 2025 18:09

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels May 29, 2025

tgross35 reviewed Jun 10, 2025

View reviewed changes

rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jun 10, 2025

niklasf force-pushed the feature/read-buf-at branch from e8a35d4 to f6dd203 Compare August 18, 2025 19:24

Add read_buf equivalents for positioned reads

bf491a1

Adds the following items under the `read_buf_at` feature: - `std::os::unix::fs::FileExt::read_buf_at` - `std::os::unix::fs::FileExt::read_buf_exact_at` - `std::os::windows::fs::FileExt::seek_read_buf`

niklasf force-pushed the feature/read-buf-at branch 2 times, most recently from bf491a1 to 55ce5c1 Compare August 18, 2025 20:10

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Aug 18, 2025

This comment has been minimized.

Sign in to view

niklasf force-pushed the feature/read-buf-at branch from 55ce5c1 to bf491a1 Compare August 18, 2025 20:31

Add read_buf equivalents for positioned reads #140459

Are you sure you want to change the base?

Add read_buf equivalents for positioned reads #140459

Conversation

niklasf commented Apr 29, 2025 • edited by tgross35 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rustbot commented Apr 29, 2025

Uh oh!

This comment has been minimized.

tgross35 commented May 1, 2025

Uh oh!

tgross35 commented May 1, 2025

Uh oh!

rustbot commented May 1, 2025

Uh oh!

Kivooeo commented May 8, 2025

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

niklasf commented May 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tgross35 commented May 28, 2025

Uh oh!

bors commented May 29, 2025

Uh oh!

This comment has been minimized.

niklasf commented May 29, 2025

Uh oh!

tgross35 Jun 10, 2025

Choose a reason for hiding this comment

Uh oh!

tgross35 Jun 10, 2025

Choose a reason for hiding this comment

Uh oh!

ChrisDenton Aug 18, 2025

Choose a reason for hiding this comment

Uh oh!

niklasf Aug 18, 2025

Choose a reason for hiding this comment

Uh oh!

niklasf Aug 18, 2025

Choose a reason for hiding this comment

Uh oh!

niklasf Aug 18, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

tgross35 commented Jun 10, 2025

Uh oh!

rust-bors bot commented Jun 10, 2025

Uh oh!

tgross35 Jun 10, 2025

Choose a reason for hiding this comment

Uh oh!

niklasf Aug 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tgross35 commented Jun 10, 2025

Uh oh!

rust-bors bot commented Jun 10, 2025

Uh oh!

tgross35 commented Aug 13, 2025

Uh oh!

bors commented Aug 13, 2025

Uh oh!

rustbot commented Aug 18, 2025

Uh oh!

niklasf commented Aug 18, 2025

Uh oh!

This comment has been minimized.

Uh oh!

Add `read_buf` equivalents for positioned reads #140459

Add `read_buf` equivalents for positioned reads #140459

niklasf commented Apr 29, 2025 •

edited by tgross35

Loading

niklasf commented May 12, 2025 •

edited

Loading

niklasf Aug 18, 2025 •

edited

Loading