`Box` in custom allocator fails even with `-Zmiri-tree-borrows`

[js2xxx]

I'm writing an arena-like memory allocator that uses pointer arithmetic to calculate the associated bin address of the allocated pointer. Some of the structures look like this:

Example on the playground

When I tested my allocator with miri, it told me some undefined behavior occurred.

At first, I thought it was just a duplicate of #2104 and rust-lang/unsafe-code-guidelines#402 when I ran the example above with -Zmiri-stack-borrows, so I followed the suggestions in the comments there and switched to -Zmiri-tree-borrows.

However, when switched to -Zmiri-tree-borrows, the error stops happening if I use Vec::with_capacity_in(1, alloc), but occurs again for a different reason if I use (Box::new_in([1], alloc) as Box<[u32], A>).into_vec() (commented out in the example above), which indicates something wrong with Box once again.

The output of miri says:

error: Undefined Behavior: read access through <1659> is forbidden
   --> ~/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/cell.rs:517:18
    |
517 |         unsafe { *self.value.get() }
    |                  ^^^^^^^^^^^^^^^^^ read access through <1659> is forbidden
    |
    = help: this indicates a potential bug in the program: it performed an invalid operation, but the Tree Borrows rules it violated are still experimental
    = help: the accessed tag <1659> is a child of the conflicting tag <1526>
    = help: the conflicting tag <1526> has state Disabled which forbids this child read access
help: the accessed tag <1659> was created here
   --> main.rs:58:5
    |
58  |     (Box::new_in([t], a) as Box<[T], A>).into_vec()
    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
help: the conflicting tag <1526> was created here, in the initial state Reserved
   --> main.rs:58:6
    |
58  |     (Box::new_in([t], a) as Box<[T], A>).into_vec()
    |      ^^^^^^^^^^^^^^^^^^^
help: the conflicting tag <1526> later transitioned to Disabled due to a foreign write access at offsets [0x0..0x8]
   --> main.rs:37:9
    |
37  |         self.top.set(delta as usize);
    |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    = help: this transition corresponds to a loss of read and write permissions
    = note: BACKTRACE (of the first span):
    = note: inside `std::cell::Cell::<usize>::get` at ~/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/cell.rs:517:18: 517:35
note: inside `<MyAllocator as std::alloc::Allocator>::deallocate`
   --> main.rs:50:28
    |
50  |             println!("{}", this.top.get());
    |                            ^^^^^^^^^^^^^^
    = note: inside `<&MyAllocator as std::alloc::Allocator>::deallocate` at ~/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/alloc/mod.rs:392:18: 392:50
    = note: inside `<alloc::raw_vec::RawVec<i32, &MyAllocator> as std::ops::Drop>::drop` at ~/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/alloc/src/raw_vec.rs:532:22: 532:56
    = note: inside `std::ptr::drop_in_place::<alloc::raw_vec::RawVec<i32, &MyAllocator>> - shim(Some(alloc::raw_vec::RawVec<i32, &MyAllocator>))` at ~/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ptr/mod.rs:515:1: 515:56
    = note: inside `std::ptr::drop_in_place::<std::vec::Vec<i32, &MyAllocator>> - shim(Some(std::vec::Vec<i32, &MyAllocator>))` at ~/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ptr/mod.rs:515:1: 515:56
    = note: inside `std::ptr::drop_in_place::<(std::vec::Vec<i32, &MyAllocator>, std::vec::Vec<i32, &MyAllocator>)> - shim(Some((std::vec::Vec<i32, &MyAllocator>, std::vec::Vec<i32, &MyAllocator>)))` at ~/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ptr/mod.rs:515:1: 515:56
    = note: inside `std::mem::drop::<(std::vec::Vec<i32, &MyAllocator>, std::vec::Vec<i32, &MyAllocator>)>` at ~/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/mem/mod.rs:992:24: 992:25
note: inside `main`
   --> main.rs:66:5
    |
66  |     drop((a, b));
    |     ^^^^^^^^^^^^

note: some details are omitted, run with `MIRIFLAGS=-Zmiri-backtrace=full` for a verbose backtrace

error: aborting due to 1 previous error

It seems a Reserved region is affected by foreign writing access and thus becomes Disabled. I wonder what "magic" Box puts in here, and whether there is some way to solve it (or get rid of it).

[RalfJung]

This is odd, the problem is with accessing the allocator state itself, not the memory that stores the data. The backtraces are not very helpful since the actual action is in the standard library. Something Box/Vec do with the allocator itself makes Miri unhappy.

Note that there is an unrelated UB issue here

        let delta = unsafe { next.byte_offset_from(self.memory.get()) };

This is UB if next is out-of-bounds.

[RalfJung]

However, when switched to -Zmiri-tree-borrows, the error stops happening if I use Vec::with_capacity_in(1, alloc), but occurs again for a different reason if I use (Box::new_in([1], alloc) as Box<[u32], A>).into_vec() (commented out in the example above), which indicates something wrong with Box once again.

I can't reproduce this locally: when I run your file with -Zmiri-tree-borrows, it seems to work fine.

$ ./miri run alloc.rs -Zmiri-tree-borrows
8
8
8

EDIT: oh your code is the non-failing one. That's confusing...

[RalfJung]

This line is the culprit:

let their_alloc = ptr.as_ptr().map_addr(|addr| addr & !127).cast::<Self>();

Even with Tree Borrows, you can't just "get out" of the Box and access the surrounding memory again. ptr here is a unique pointer to a Box<i32> -- emphasis on "unique pointer". Tree Borrows is fine with using this pointer for more than just that i32, but it is still unique in everything it touches. And yet here you are using it in a way that aliases with the original my_alloc pointer or with the other Box.

This seems pretty fundamental; if we want to assume that Box is unique then we can't just use a Box pointer (or things derived from it) in aliasing ways like that.

[RalfJung]

What I don't understand is why you use ptr here to get to top. You have self available which should be a perfectly good way to access that same memory.

[js2xxx]

Thanks for your reply!

What I don't understand is why you use ptr here to get to top. You have self available which should be a perfectly good way to access that same memory.

Here is a more realistic example with some explanatory comments: Rust playground. In short, finding the corresponding bin is somewhat expensive in my real implementation, so I chose to use pointer arithmetics.

This seems pretty fundamental; if we want to assume that Box is unique then we can't just use a Box pointer (or things derived from it) in aliasing ways like that.

I think the uniqueness of Box pointers is too strong currently, which should just cover the actual storage place of its data (such as the 4 bytes of a u32) in space, and just cover the lifetime between the allocation and deallocation of its memory in time. Could it still be sound being modified this way? Or could it be actually implemented in pratice?

Note that there is an unrelated UB issue here

Oops XD. The newer example has fixed it.

[RalfJung]

I think the uniqueness of Box pointers is too strong currently, which should just cover the actual storage place of its data (such as the 4 bytes of a u32) in space, and just cover the lifetime between the allocation and deallocation of its memory in time. Could it still be sound being modified this way? Or could it be actually implemented in pratice?

There's no way to have a pointer that's only unique for some of memory, that would defeat the entire purpose of noalias (being able to reorder accesses without knowing much about them).

Making uniqueness temporary in scope is possible, but here this doesn't really help. Consider a function like:

fn foo(a: Box<i32, MyAlloc>, b: Box<i32, MyAlloc>) {
  drop(a); drop(b),
}

LLVM does limit the scope of the noalias here (more so than Stacked/Tree Borrows), and it's the entire function. And yet the custom allocator destructor will do aliasing accesses while the function runs. LLVM doesn't even know that there is a "deallocation" here, calling MyAlloc::deallocate is just a regular fn call. There's no way to say that this magically ends the scope of some aliasing requirements (which ones would that even be, and how would we know the parent scope).

I think the only way to allow such accesses is to not have noalias to begin with.

[js2xxx]

LLVM doesn't even know that there is a "deallocation" here, calling MyAlloc::deallocate is just a regular fn call.

Things being it here, why not give Allocate::deallocate some special treatment just like foreign functions? If my memory is correct, the noalias rule for Box pointers will end at a foreign function call, such as libc::free.

[RalfJung]

No there is no such special treatment. Foreign functions are usually "unknown" to the compiler so it has to be very conservative, but then Rust supports cross-language LTO so e.g. inlining C functions into Rust may actually happen.

malloc and free are special magic functions that likely do need extra compiler treatment (different from your average foreign function); the exact shape and rules for that are unknown at this time. rust-lang/unsafe-code-guidelines#442 discusses some of that. But that's all specifically for the global allocator. What Box<T, A> does is just call some functions on A; so far there's no magic going on here whatsoever. Maybe there should, I don't know -- I worry it might cause more problems. What causes issues for you is making the Box type magic (with noalias) without also making the allocator functions magic.

[js2xxx]

Okay I get it now. Seems like this issue cannot be resolved for a few days. I'll leave this open for now.