RFC 0002: MT Execution Contexts

[ysbaddaden]

No description provided.

[beta-ziliani]

Preview: https://github.com/crystal-lang/rfcs/blob/rfc-0002-mt-execution-contexts/text/0002-execution-contexts.md

[straight-shoota]

The distinction between execution context and scheduler could need a bit refinement. There is definitely some overlap in functionality, just by comparing the API. I guess execution contexts might take over some features of the current scheduler?

[RX14]

Absolutely lovely, well-written proposal. I completely agree with the design intent here, and only have a few—mostly overlapping—comments about event loops and the default context. The vast majority of this design is exactly what I would like to see in crystal.

[RX14]

This proposal supports creating multiple execution contexts, let the application configure it's own EC and start fibers in it if required. That allows all the complexity the app needs when configuring the context the application actually runs in, because it's initialized by the application. The root context does not have to be well-used by the application.

[ysbaddaden]

@RX14 Tell me if I'm wrong, the plan would be:

Crystal 1

• introduce EC with ST and MT;
• deprecate same_thread argument;
• same_thread: false is NOOP;
• ST accepts same_thread: true (always true anyway);
• MT raises on same_thread: true (new API, no breaking change);
• default EC is ST (no breaking change);
• consider a -Dmt flag to force default EC to be MT (?);

Crystal 2

• remove deprecated same_thread (breaking change);
• default EC becomes MT (breaking change).

Isolated context

I think I see that context for UI loops only, and want to prevent blocking behaviors, but there's nothing wrong with doing blocking calls in other use cases, and using the event-loop normally is fine. Still, spawning a fiber without an explicit context should either raise or the default context should be configured (as you suggest):

abstract class ExecutionContext
  class Isolated < ExecutionContext
    def initialize(name : String, @spawn_context : ExecutionContext? = nil, &)
      @thread = Thread.new(name) { yield }
    end

    def spawn(**args, &) : Fiber
      if ctx = @spawn_context
        ctx.spawn(**args) { yield }
      else
        raise RuntimeError.new("Can't spawn in isolated context (need a spawn context)")
      end
    end
  end
end

mt = ExecutionContext::MultiThreaded.new
ui = ExecutionContext::Isolated.new("GTK", spawn_context: mt) { Gtk.main }

Instead of raising, the spawn context could be the default EC.

[ysbaddaden]

@RX14 I applied your suggestions to the RFC.

[RX14]

@ysbaddaden I think -Dmt is probably not necessary, and the exact implementation plan for crystal 2 is best left deferred until there's operational experience, but I agree on everything else.

I envision the root execution context being MT or ST a moot point, because every well-architected app has a single App.run line at the top-level and converting that top-level code to be spawning a MT context and waiting for that fiber should be a one-liner if we have the right helper methods in place.

If we all agree, maybe we can start on the other 90% of the RFC: bikeshedding naming. I like ExecutionContext::Parallel, because I don't like the idea of implementation details (threads) leaking into the name.

[ysbaddaden]

@RX14 The mt flag may not be necessary in Crystal v1, as the default context could be MT:1 and resized on demand (still no breaking change). I'll still push for MT:N to be the default in Crystal v2. Execution contexts are a mean to further control the parallelism in very specific cases, not the end solution. I believe developers shouldn't have to care about it until you have to.

I wouldn't bikeshed the namings just yet. As I'm experimenting with the types, I feel that the difference is getting thinner and thinner. In fact, Kotlin only has a single scheduler implementation, and a couple constructors to start execution contexts with 1 (ST) or many threads (MT).

I'm also struggling with the inheritance: EC::MT < EC makes sense, but so does EC::MT::Scheduler < EC as we want EC.current to point to the current MT scheduler running on the thread, not the shared MT context (it's easier to reach the context from the scheduler).

[crysbot]

This pull request has been mentioned on Crystal Forum. There might be relevant details there:

https://forum.crystal-lang.org/t/crystal-multithreading-support/6622/8

[ysbaddaden]

I forgot again but MT:1 would break spawn(same_thread: true) in Crystal 1. It's a NOOP without the preview_mt flag but the parameter was still exposed to the public API 😭

[straight-shoota]

I think we can accept breakage with same_thread: true. It only works with preview_mt which is explicitly a preview feature. There should be no expectations on compatibility in a setting outside of preview_mt.

[yxhuvud]

I like this, I like this a lot.

[yxhuvud]

I think data and especially execution locality could show up on the negative side as well, as the round robin take away a lot of programmatic control of data locality as well. It is .. possible.. to manually schedule fibers to dedicated threads but it really is not the way it currently is meant to be used.

[yxhuvud]

I'm in the 'let the event loop decide if it want to be instantiated on a thread, execution level or global level' camp. How that would look API-wise I'm less sure - especially not if dynamic amount of threads in a context is to be supported.

This might be complex: I don’t think we can share a libevent across event bases

From what I have gathered from libevent docs, it is possible, but would necessitate a lot more synchronization when io happens (*), so it is probably slower.

But yes, it is complex. Windows, and its weird file handles says hi. Each open file handle is specific for each instance of whatever it uses, so it needs to be only one global one event instance there.

• we already enable some structures for thread safety but then create separate bases for each thread anyhow, IIRC. It was quite a while since I looked at it. I think we can remove that enabling without danger - they should really only be used when actually reusing a libevent base between threads). We don't use the specialized mt safe functions libevent that make use of it.

[yxhuvud]

Allowing a dynamic amount of threads requires more synchronization and complexity than having a static amount. While it sounds nice to be able to adjust, it probably warrants its own separate class. Making certain all threads are in a waiting state before starting to actually queue stuff allows a bunch of simplifications with less mutexes and risks a lot fewer possible race conditions too.

[ysbaddaden]

Enqueue doesn't have to need much sync. Go pushes to a bounded local queue (per scheduler) with overflow to a global queue; threads can be started at any time: when they reach the run loop they will grab a batch of fibers from the global queue or steal from another scheduler. Stopping ain't more complex, schedulers aren't tied to a specific thread, the thread detaches the scheduler and returns itself to the thread pool.

The complexity is more in when to start / stop a thread.

[ysbaddaden]

And that complexity pushes for it to become a "future evolution".

[Blacksmoke16]

As someone who isn't familiar with this stuff at all, my random question is:

Do we need to do anything in relation to like how Intel has P and E cores now? Like as a way to signal to the OS's thread scheduler that a fiber should have a preference on where it runs? Or is that something the OS itself handles somehow?

[ysbaddaden]

@Blacksmoke16 From what I read specifying a thread priority can hint the OS to schedule the thread on a big (efficient) or little (power) core. We can also set a thread affinity to a given core, but we must detect the core type beforehand.

[crysbot]

This pull request has been mentioned on Crystal Forum. There might be relevant details there:

https://forum.crystal-lang.org/t/upcoming-release-1-15-0/7537/1

[ysbaddaden]

The current constructors are rather complex and inconvenient to type, and pushing ExecutionContext under the Fiber namespace didn't help. I mean:

ctx = Fiber::ExecutionContext::Isolated.new(name) { }
ctx = Fiber::ExecutionContext::SingleThreaded.new(name)
ctx = Fiber::ExecutionContext::MultiThreaded.new(name, size: 1..4)

I'm wondering if we could have simpler helpers. For example (thinking out loud, don't take things for granted):

ctx = Fiber.start_isolated(name) { } # or #spawn_isolated ?
ctx = Fiber.start_concurrent_context(name)
ctx = Fiber.start_parallel_context(name, size: 1..4)

I'm not sure it's really better. Maybe a bit more explicit thanks to the start_ prefix?

The naming might also indicate that maybe the ST and MT classes should be named Concurrent and Parallel of SingleThreaded and MultiThreaded?

[crysbot]

This pull request has been mentioned on Crystal Forum. There might be relevant details there:

https://forum.crystal-lang.org/t/upcoming-release-1-16-0/7883/1

[ysbaddaden]

I updated the phrasing and names of the proposed execution contexts.

Reading the RFC again, I believe that its purpose, to propose a new API to achieve parallelism in Crystal, can be considered done, even though the feature isn't fully released.

I propose to archive RFC 0002 and to start a new one with details on the actual execution contexts (Concurrent, Parallel, Isolated), what guarantees they bring and how they differ, and what still needs to be done: resize, shutdown, thread pool and detach on syscall, mostly.

[straight-shoota]

thought: I'm still not sure if it's even worth having a separate concurrent context when there's not much practical benefit over a parallel context with max=1.

[ysbaddaden]

Well, while ST is fixed to at most 1 thread at any time, MT:1 could grow to MT:N under the hood.

[straight-shoota]

But not on its own, only if you rescale the context. Or am I missing something?

[ysbaddaden]

It must be an explicit call, sure, but it can happen, and a compilation error, or at least a runtime exception, is better than running into segfaults. That doesn't mean that a Concurrent contexts needs a specific implementation, but that it needs to make sure that the context can NEVER be resized.

This guarantee alone seems enough to warrant the existence of the context.