Saturday, January 1, 2011

Coroutines, futures and actors

I've been reading up on Scala and Erlang lately - I'd like to try implementing coroutines with support for multiple threads in Impulse to take advantage of multiple cores.  This is how I understand the current languages/frameworks that are related to concurrency:
Erlang            =>    Extremely light preemtive(*)
                        coroutines + async IO +
                        multithreaded + distributed

Io                =>    Coroutines + async IO

Stackless Python  =>    Coroutines + blocking IO
+ Tasklets

Scala             =>    Event callbacks or OS threads
                        (to have continuations soon?)

Twisted, Node.js  =>    Event callbacks
(*) VM instruction preemtion, as in yielding after N number of reductions (function calls). Native C bindings can still block.

Event callbacks are stateless, which means nested loops need to handle state explicitly. Non-preemtive coroutines means that a "while (true);" can block all other coroutines. Blocking IO means that a IO call will block all other coroutines.

So apart from making every value immutable, I assume I could give each actor its own global environment, ensuring that it doesn't have access to objects that could be running in a different thread. I believe in Scala you just have to follow convention and not access mutable state?

Using futures is a great way to write your programs in a data-flow style. But without immutable state, even with cooperative coroutines, values
from change from underneath you. Io allows any object to become a temporary actor by sending asynchronous methods that return a future - but in a multi-threaded environment, there doesn't seem to be enough control without moving the object to it's own global environment (which seem pretty tricky).

So, would a combination of non-shared state between actors, coroutines, non-blocking IO, futures and one thread per processor with a scheduler do the trick? Example in Io:
calc = Actor clone do (
  product = method (value,
    range(1, value) reduce(*)
  )
)

value = calc product(5)
// do other stuff
writeln(value + 10)