rt: Runtime refinements

[carllerche]

Summary

Based on experience gained in 0.2, there are a few aspects of the Runtime that can use improvements.

• Remove basic_scheduler and threaded_scheduler variants from the public API.
• Automatically use the basic_scheduler when core_threads is set to 0, or 1.
  • When core_threads is set to 1, use basic_scheduler and spawn it on a background thread.
  • When core_threads is set to 0, use basic_scheduler on the current thread.
• Switch Runtime::block_on to take &self and remove tokio::runtime::Handle.
• #[tokio::main] always uses the threaded scheduler and requires the rt-threaded scheduler.
  • Using #[tokio::main] panics without rt-threaded.
  • Using #[tokio::main(core_threads = 1)] results in a build failure.

Remaining

• Initial pass rt: Refactor Runtime::block_on to take &self #2782
• Support concurrent calls to block_on. Refactor basic_scheduler/shell to embed the block_on implementation within their own structs rather than within Runtime::block_on. This means making those block_on's take &self not &mut self. Improve stealing the root parker in situations where it gets reset back into the mutex and another thread needs to now drive it.
• Refactor runtime::Builder to remove threaded_scheduler/basic_scheduler and refactor core_threads to do the behavior described in rt: Runtime refinements #2720.
• For new_single_thread should spawn basic on a bg thread
• Refactor #[tokio::main] as described in rt: Runtime refinements #2720.
• Review runtime/macro docs to ensure consistency

Details

Removing basic_scheduler and threaded_scheduler

The details of which scheduler to pick isn't really what matters to the end-user. We already have core_threads to configure how many threads should be used to schedule tasks. The scheduler implementation can be selected internally based on the number of threads used.

Currently, basic_scheduler does not spawn a dedicated runtime thread. Instead, the current thread is used to schedule tasks. When block_on is called, the runtime starts running tasks and when block_on completes, the runtime is paused. This behavior can be gained again by setting core_threads = 0.

Using core_threads = 0 to signify running on the current thread might be a bit confusing. However, running the scheduler on the current thread is, in effect, requesting no runtime threads. This is also a somewhat specialized use case. Most users will use the threaded scheduler with default configuration settings.

Switch Runtime::block_on to take &self

The reason Runtime::block_on takes &mut self is to support the core_threads = 0 use case. When there are no runtime threads, the runtime is executed "in-place". The &mut self argument enforces that a single caller may "enter" the runtime so it is clear which thread executes the runtime.

However, there are valid cases where multiple threads may want to potentially share the responsibility of executing the runtime. In this case, the user is responsible for managing synchronization. Instead, Tokio can provide this functionality out of the box. When running with core_threads = 0 and calling block_on concurrently from multiple threads, the first thread that enters the runtime becomes responsible for executing the runtime. Other threads just block on the supplied task.

Making this change also has the property of removing the need for having a separate runtime::Handle. The Handle type was added because Runtime is !Sync. This change would make Runtime sync and threads could just use functions on Runtime instead of Handle.

Additional consideration must be taken to support concurrent calls to block_on when core_threads == 0. Consider the following:

• Thread A calls block_on and takes the responsibility of driving the runtime
• Thread B calls block_on and uses a regular thread parker.
• Thread A's future completes and no longer drives the runtime.

At this point, thread B is waiting for its future to complete, but nothing is driving the runtime anymore, so the future will never unblock.

To handle this, when the thread that is driving the runtime completes the call to block_on, another blocked thread must be notified to start driving the runtime.

This can probably be implemented using a SyncParker<T: Park> utility that handles the hand off and implements Park for &Self.

#[tokio::main] always uses the threaded scheduler and requires the rt-threaded scheduler.

The behavior of #[tokio::main] changes based on feature flags that are enabled. When rt-threaded is not included, #[tokio::main] uses the basic scheduler. When it is enabled, the threaded scheduler is used. In general, changing behavior based on feature flags is not great. Feature flags are intended to be additive.

Additionally, Runtime::new() is only defined with rt-threaded.

Instead, #[tokio::main] will require rt-threaded and always have the same behavior. If rt-threaded is not the program will fail to build. The main macro can stll be used without the rt-threaded feature flag enabled, but core_threads will need to be set to 0 or 1. In this case, the basic scheduler is used.

The goal is to ensure consistent behavior regardless of which feature flags are selected.

Open question

Should #[tokio::main] / Runtime::new() be available if io-driver and time feature flags are not enabled? I.e. should the default functions be available when all feature flags are not enabled? I'm leaning towards allowing #[tokio::main] being available w/ only rt-threaded and not require 'io-driver/time`.

Refs: #2698

[hawkw]

• Using #[tokio::main] panics without rt-threaded.

Couldn't we just only expose main if rt-threaded is enabled? We can still expose other macros without the rt-threaded feature.

[LucioFranco]

Can we still provide a single threaded macro too? Otherwise this looks good.

Should #[tokio::main] / Runtime::new() be available if io-driver and time feature flags are not enabled? I.e. should the default functions be available when all feature flags are not enabled? I'm leaning towards allowing #[tokio::main] being available w/ only rt-threaded and not require 'io-driver/time`.

I am curious what the use-case here would be with no io driver or timer?

[hawkw]

I am curious what the use-case here would be with no io driver or timer?

IMO, yeah, it seems pretty reasonable to require users who want the shell runtime to have to actually go through the builder API...it's not a common configuration or what most folks are going to want out of the box.

I'm open to being convinced otherwise, of course.

[LucioFranco]

We also likely want to remove runtime::Builder::num_threads in this release.