no_std + alloc feature selection

[wez]

I'd like to make use of futures in some embedded (arm) and separately in some kernel (macOS) code. In both of these places the code is no_std but implements the allocator api such that alloc::arc::Arc and alloc::boxed::Box are available.

I'm relatively new to rust so I may be wrong, but it appears to be impossible from outside futures-rs to replicate a couple of things that would unlock using futures in these situations:

• the ArcWrapped UnsafeNotify implementation to avoid losing ones mind trying to implement notify handles
• impl Future for Box which appears to be necessary to implement an executor like this one: https://github.com/alexcrichton/futures-rs/blob/2b3f0f0cc6b60f2659092dd3f2f88844c3f4bd5b/tests/support/local_executor.rs#L29

If there were a crate level feature selection to enable Box and Arc related conveniences and do so in terms of alloc rather than the std counterparts, I think it would make it significantly easier to integrate futures in these environments.

I'm also interested in guidance on how to achieve this without modifying futures-rs.

(I'd love to try futures-await in these environments too, while you're thinking about this!)

[alexcrichton]

Thanks for the report! Unfortunately though alloc isn't a stable feature of the standard library (e.g. it's not a stable crate to link against) so there's not quite a compilation target for this crate to use on stable, so I'd be hesitatnt to add support. We could perhaps, however, look to get a stable slice of the standard library for this!

[wez]

Thanks; that makes sense re: alloc.

I did manage to solve my immediate need by not using the Spawn/executor stuff; I realized that I can just poll all of the Future/Stream instances directly from the main loop of the MCU. It's not quite as efficient as the notify handle stuff would allow, but the cardinality of these things is typically very low in the embedded world and it is no worse than the pattern of just polling everything in a loop that is prevalent in a lot of embedded code.

I'd love to try a no_std futures-await, but at least I have a decent futures foundation to build on top of now :-)

[alexcrichton]

Oh so actually futures-await is probably totally compatible with no_std, I think it just needs to disable the feature when depending on the futures crate actually!

[carllerche]

It looks like the outcome of this is "no action" for futures-rs. I'm going to close the issue. Let me know if I am wrong.

[jrobsonchase]

Since futures-preview is nightly-only these days, would it be possible/desirable to take another look at this? I've got the same usecase as OP and would love to not need to resort to an extra struct FutureBox<F: Future>(Box<F>); wrapper just to get an UnsafeFutureObj impl.

[taiki-e]

I'm working on this now (branch, changes).

I think we probably need to solve the following two things:

• Is there a way to import items of alloc crate ergonomically?
• Currently, both futures_api and alloc features are unstable. It may be preferable to wait until either one stabilizes.

[jrobsonchase]

Is there a way to import items of alloc crate ergonomically?

For this, I've it done by creating a top-level module that re-exports the std/alloc things, and then any sub-modules can simply use crate::alloc_things::collections::HashMap; (for example) without having to worry about which of std or alloc is actually in use.

#![cfg_attr(feature = "alloc", feature(alloc))]
#![cfg_attr(not(feature = "std"), no_std)]

#[cfg(feature = "alloc")]
extern crate alloc;

#[cfg(any(feature = "alloc", feature = "std"))]
mod alloc_things {
    #[cfg(all(not(feature = "std"), feature = "alloc"))]
    pub use alloc::*;

    // pulls in more than necessary, but you get the idea
    #[cfg(all(feature = "std", not(feature = "alloc")))]
    pub use std::{*, prelude::v1 as prelude};
}

#[cfg(any(feature = "std", feature = "alloc"))]
mod sub {
    use crate::alloc_things::{prelude::*, sync::Arc};

    // More stuff
}

Currently, both futures_api and alloc features are unstable. It may be preferable to wait until either one stabilizes.

Why is that? It seems to me that it doesn't make much difference if just one or both is unstable since either condition will require a nightly compiler. It'll only work on stable if both are stabilized. Is it just a matter of chasing nightly changes and reducing churn?

[taiki-e]

For this, I've it done by creating a top-level module that re-exports the std/alloc things, and then any sub-modules can simply use crate::alloc_things::collections::HashMap; (for example) without having to worry about which of std or alloc is actually in use.

In the case of feature = std we can import from std all, but in case of any(feature = "std", feature = "alloc") we need to import from both core and top-level module.

I don't care much about myself, but I believe that if we have a more simple way to import it, we will be able to advance alloc + no_std support more smoothly (If other contributors do not care about this, it is okay to proceed as it is).

In the case of feature = std:

use std::any::Any;
use std::boxed::Box;
use std::cell::UnsafeCell;
use std::fmt;
use std::mem;
use std::ops::{Deref, DerefMut};
use std::pin::Pin;
use std::sync::Arc;
use std::sync::atomic::AtomicUsize;
use std::sync::atomic::Ordering::SeqCst;

In the case of any(feature = "std", feature = "alloc"):

use core::any::Any;
use core::cell::UnsafeCell;
use core::fmt;
use core::mem;
use core::ops::{Deref, DerefMut};
use core::pin::Pin;
use core::sync::atomic::AtomicUsize;
use core::sync::atomic::Ordering::SeqCst;
use crate::alloc_things::sync::Arc;
use crate::alloc_things::boxed::Box;

I'm thinking of using shim crate (alloc-shim is my experiment to solve this).

use alloc::any::Any;
use alloc::boxed::Box;
use alloc::cell::UnsafeCell;
use alloc::fmt;
use alloc::mem;
use alloc::ops::{Deref, DerefMut};
use alloc::pin::Pin;
use alloc::sync::Arc;
use alloc::sync::atomic::AtomicUsize;
use alloc::sync::atomic::Ordering::SeqCst;

Also, If we set the crate feature so that std depends on alloc and activate the alloc feature-gates only when 'std' is not specified, I think that #[cfg] can be made shorter (I've never seen this with other crates...).

[features]
default = ["std"]
std = ["alloc"]
alloc = []

#![cfg_attr(all(feature = "alloc", not(feature = "std")), feature(alloc))]

#[cfg(all(feature = "alloc", not(feature = "std")))]
extern crate alloc;

#[cfg(feature = "alloc")]
mod alloc_things {
    #[cfg(all(feature = "alloc", not(feature = "std")))]
    pub use alloc::*;

    // pulls in more than necessary, but you get the idea
    #[cfg(feature = "std")]
    pub use std::{*, prelude::v1 as prelude};
}

// It is simpler than #[cfg(any(feature = "std", feature = "alloc"))].
#[cfg(feature = "alloc")]
mod sub {
    use crate::alloc_things::{prelude::*, sync::Arc};

    // More stuff
}

[Nemo157]

I've found the easiest case is to import everything from the lowest crate they're available in. If you scan the source you'll see that core is always used where possible.

---

One reason not to have std imply alloc is that I would personally gate the alloc feature behind the nightly feature to require users to opt in to the instability

#[cfg(all(feature = "alloc", not(feature = "nightly")))]
compile_error!("The `alloc` feature requires the `nightly` feature active to explicitly opt-in to unstable features");

An alternative to having std imply alloc would be to have a build.rs set a cfg flag when either is active, something like

if env::var("CARGO_FEATURE_STD") || env::var("CARGO_FEATURE_ALLOC") {
    println!("cargo:rustc-cfg=alloc");
}

then use #[cfg(alloc)] to test if allocation is supported.

---

One way to just have the "alloc" crate available with both features is to just inject it or std into the extern prelude using the same name (I'm pretty sure they have identical APIs):

#[cfg(feature = "alloc")]
extern crate alloc;
#[cfg(all(feature = "std", not(feature = "alloc"))]
extern crate std as alloc;

[jrobsonchase]

I'm thinking of using shim crate (alloc-shim is my experiment to solve this).

Oh hey, we had same idea 😄

I've found the easiest case is to import everything from the lowest crate they're available in. If you scan the source you'll see that core is always used where possible.

Fully agree with this! In cases where no_std is a goal, it makes the most sense to start with the lowest common denominator.

[taiki-e]

One reason not to have std imply alloc is that I would personally gate the alloc feature behind the nightly feature to require users to opt in to the instability

That makes sense.

An alternative to having std imply alloc would be to have a build.rs set a cfg flag when either is active

Great, I will use this.

---

I've found the easiest case is to import everything from the lowest crate they're available in. If you scan the source you'll see that core is always used where possible.

One way to just have the "alloc" crate available with both features is to just inject it or std into the extern prelude using the same name (I'm pretty sure they have identical APIs):

I wrote both of them (1, 2). Except that prelude::v1::* can not be used, I think the second one is easier to write.