UPDATE 2013.05.14: Edited to tweak various errors and to add some variations at the end that I prefer.

Today: by-reference vs copying closures

Today we offer three different kinds of closures (&fn, @fn, and ~fn), but these closures can really be divided into two basic categories: by-reference and copying closures. A by-reference closure is the usual kind: it is allocated on the stack and has full access to the variables in the creating stack frame. It can read them, write them, and borrow them. These are used with for loops and the like.

Copying closures, on the other hand, are somewhat different. They are not tied to any particular stack frame. Instead, they copy the current values of the variables which they close over into their environment (like all the default Rust copies, this is a shallow copy, so if the value being closed over contains ~ pointers, it will no longer be accessible from the creator). These closures are used primarily as task bodies and for futures. There are some scattered uses of @fn closures in the compiler but as far as I can tell they are all legacy code that should eventually be purged and rewritten to use traits (i.e., the visitor, the AST folder).

Loosely speaking, a &fn closure is by-reference and @fn and ~fn closures are copying closures. But this is not strictly true. In fact, an the &fn type can be either a by-reference closure or a copying closure, because you are permitted to borrow a @fn or ~fn to a &fn. So the type in isolation does not tell you whether a closure is by-reference or not. In fact, there is no explicit indication at all—instead, when you create a closure today (i.e., with a |x, y| ...) expression, the compiler infers based on the expected types whether this should be a by-reference closure or a copying closure. Because the semantics of these two vary greatly, I find this potentially quite confusing and unfortunate.

Tomorrow (perhaps): closures and procedures

In general, I would prefer to draw a starker line between copying and by-reference closures. I propose to use the term closure to refer only to by-reference, stack-allocated closures. We could then use another term, perhaps procedure, to refer to the copying closures. This would mean that our type hierarchy would look like:

T = S               // sized types
  | U               // unsized types
S = fn(S*) -> S     // closures (*)
  | &'r T           // region ptr
  | @T              // managed ptr
  | ~T              // unique ptr
  | [S, ..N]        // fixed-length array
  | uint            // scalars
  | ...
U = [S]             // vectors
  | str             // string
  | Trait           // existential ("exists S:Trait.S")
  | proc(S*) -> S   // procedures (*)

This chart is basically the same as the one you will find in the dynamically sized types post from before with one crucial difference: closure types have been split from procedures, and closure types have moved into the category of sized types, meaning that you no longer write an explicit sigil when you use one. This is because the representation of a closure would always be a pair of a borrowed pointer into the stack and a function pointer: the type has a fixed size (two words) and requires no memory allocation.

I have chosen to leave procedures as unsized, since a procedure must allocate memory on the heap, and this allows the user to select which heap is used; in earlier drafts of this idea, I had modified procedures to implicit use the exchange heap, meaning that a type like proc() always represented an exchange heap allocation. But I think it’s more consistent to have that type be written ~proc, and it maintains the general Rust invariant “you don’t have allocation unless you see a sigil”.

UPDATE: bstrie on IRC asked about fn items, which never have any environment. As today, these would continue to be coercable to either a closure or a procedure.

Closure and procedure expressions

Closures would still be created with the form |x, y| expr. Procedures would be created using the keyword proc: proc(x, y) expr. If desired, we could integrate procedures into do using some syntax like one of the following, depending on whether we wish to make the sigil explicit:

do spawn proc { ... }   // sigil inferred

do spawn ~proc { ... }  // sigil explicit

Closure and procedure types in more detail

The full function or procedure type would look something like this ([] indicates optional content):

 [once] (fn|proc) [:['r] [Bounds]] <'a...> (S*) -> S
 ^~~~~^           ^~~~~~~~~~~~~~~^ ^~~~~~^ ^~~^    ^
   |                     |            |     |      |
   |                     |            |     |  Return type
   |                     |            |    Argument types
   |                     |          Bound lifetime names
   |               Lifetime and trait bounds
Onceness

Here the “onceness” indicates whether the closure/procedure can be called more than once. The “lifetime and trait bounds” indicate constraints on the environment. The lifetime bound 'r indicates the minimum lifetime of the variables that the closure/procedure closes over, and the “bounds” (if any) would give bounds on the types of those variables. Finally, you have the argument and return types.

If omitted, the default bounds for a closure would be a fresh lifetime and no type bounds. The default bounds for a procedure would be the static lifetime and Owned.

Use cases

Let’s look briefly at the use cases I listed before.

Higher-order and once functions

Typical uses for higher-order and once functions look much the same as before, but minus a sigil.

impl<T:Sized> for [T] {
    pub fn map<U:Sized>(f: fn(&T) -> U) -> ~[U] { ... }
                        // ^~~~~~~~~~~
}

impl<T:Sized> for Option<T> {
    // `each` on an option type can only execute at most once:
    pub fn each(f: once fn(&T) -> bool) -> bool { ... }
                // ^~~~~~~~~~~~~~~~~~~
    }
}

For contrast, these are &fn(&T) -> U and &once fn(&T) -> bool today.

Sendable functions and sendable once functions

Here is an example of a sendable once function:

fn spawn(f: ~once proc()) {...}
         // ^~~~~~~~~~

As we saw before, one would write one of the following to call this function:

do spawn proc { ... }
spawn(proc { ... })

Creating a future would look like future(proc expr) (vs future(|| expr) today).

Const closures

One could still use const closures to achieve lightweight parallelism:

impl<T:Sized> for [T] {
    pub fn par_map<U:Sized>(f: fn:Const(&T) -> U) -> bool { ... }
                            // ^~~~~~~~~~~~~~~~~
}

However, I have been thinking that we’ll have to be careful here, we need some way to guarantee that the closure does not move from its environment and then replace the moved value. Today this is illegal, but if we can prevent closures from recursing (which we must do anyhow) then we could make such moves legal, and it would be useful sometimes. On simple solution is to stay that if the closure type has a Const bound, moves are illegal, but it’s a bit…ad-hoc, since the bounds are only supposed to be constraining the types of the variables that are closed over. Still, it might be good enough.

Sendable const functions and combinators

As I argued before, I think these are not important use cases, but with procedures they actually work out fine (though not with variations 2 and 3 below). A sendable const function can be expressed with the type ~proc:Owned+Const(), which is complex, but then it is a complex idea. Combinator types would likely look like @proc or @proc:'r, in the case where the combinator closes over borrowed data.

Variation #1: Leaving procedures out of the core language

In fact, I think proc types need not be built into the language, you could model them with traits, though you’d probably want a macro like proc!(...) for defining the proc body. This would also mean the procedures can’t be used with do form.

Variation #2: Limit procedures to execute once

I don’t know of any (good) uses cases for non-once procedures. I think they should just always be once. This would mean that the only closure types that are commonly needed would be:

1. fn(T) – normal higher-order functions
2. once fn(T) – higher-order functions that execute at most once
3. ~proc(T) – procedures

Because procedures can always be desugared into a struct and a trait, this would not lose no expressiveness.

Variation #3: Limit procedures to execute once and use exchange heap

For maximum streamlining, we could make proc implicitly use ~, in which case it would be written:

1. fn(T) – normal higher-order functions
2. once fn(T) – higher-order functions that execute at most once
3. proc(T) – procedures

These types read pretty well, I think.

Summary

I have long been unsatisfied with the implicit and confusing divide between “by reference” and “copying” closures. Splitting them into two concepts seems to address a lot of issues and be an overall win to me.

Just write &mut fn()

Maybe it’s not so bad. It is advertising the possibility that the closure may mutate its environment. This would mean that while &fn() is a valid type, it is a type that does not permit the function to be called, much as &&mut (pointer to a mutable borrowed pointer) does not permit the mutable borrowed pointer to be used.

At first I was thinking that there is also a valid interpretation for &fn, meaning a function that does not mutate the variable in its environment, but then I realize that per the DST proposal any &mut fn could be borrowed to &fn, and so that would not be sound.

Remove everything but borrowed closures

We could just only have borrowed closures. The type would be written fn[:bounds]() or once fn[:bounds](). There’d be no need to notate the kind of environment pointer: it’s always a borrowed pointer. All other uses of closures would be expressed using traits and impls.

Mainly this means that code which spawns traits would get somewhat verbose, because you would need to create a struct or some other type to capture all of the upvars. For larger tasks, this is not a big deal, but for some code it could be rather annoying. I imagine futures in particular would become much more verbose; enough so as to be nearly unusable.

On the upside, there’d be no more confusion about whether a closure copies its environment or not (no, it never does). Closure types would be simpler (no need to worry about sigils). You’d write fn() or once fn() in all but the most esoteric cases. The code to manage closures would become much simpler.

Add a new keyword for what is now called an “owned closure”

This is basically the fn~ solution with another name. Rather than writing fn~ to indicate a closure value that owns its environment, we could write proc (for procedure) or something like that. This avoids the annoying “sigil after the name”, at the cost of a new keyword.

Procedures could probably always be single-shot (that is, once). Almost all use cases for them (futures, tasks, etc) are single-shot, and the others could probably be accommodated with traits instead. But we could also distinguish between a proc and a once proc if we wanted.

Procedures would probably be less interoperable with functions, since the name does not particularly suggest interoperability. For example, I imagine you could not use a proc where a fn is expected. I don’t know of any time that this is actually important.

Using a different name also helps to draw a clear line between between “closures” (which reference the variables in the stack frame that created them) and “procedures” (which copy out from that stack frame). I personally would prefer to designate procedures with a different syntax, e.g., proc(x, y) { ... } in place of |x, y| ..., but this is not necessary (as an aside, I had hoped to write some today about why I think our current use of || to designate any kind of closure is troublesome and should be changed, before I realized that we’d have to address this problem I’m thinking over instead).

More ideas?

Ok, that’s most of the more radical ideas I’ve had so far. I’ll have to keep thinking on it.

My solution for this was to make the type system treat a &fn() value the same way it treats &mut T pointers: they would be non-copyable, and when you invoke them, that would be effectively like a “mutable borrow”, meaning that for the duration of the call the original value would become inaccessible. So in short the type system would guarantee that when you call a closure, that same closure is not accessible from any other path in the system, just as we now guarantee that when you mutate a value, that same value is not accessible from any other path in the system.

This is all well and good, and I think this treatment would be largely invisible to the user under common access patterns. However, it does not play well with the proposal for dynamically sized types, because under this proposal all things written &T must behave the same, no matter what T is. This is in fact the whole point of the proposal! But here I want to treat &fn specially.

I’ve been pondering various solutions this morning. I have come up with two possible avenues:

1. Instead of writing &fn() you could write &mut fn(). This is perhaps the “principled” solution, but I consider it rather a non-starter. Writing &fn() for a closure is…tolerable, but &mut fn() is not. It’s verbose and it seems sort of nonsensical (although there is some logic to it, when you consider that calls to the function may mutate the environment and so forth).

2. We go back to the older notation and move sigils for closures after the fn. This actually has some notational perks. For example, rather than writing &fn() we can just write fn() (if there is no sigil, we can default to &). On the minus side, a sendable closure would be written fn~()—but, then again, under the dynamically sized types proposal, sendable closures were going to be written ~fn:Owned(), so is fn~() really so bad?

More details after the fold.

OK, let’s dig into the details a bit more. As anyone who has been following my blog posts probably knows by now, there are many, many use cases for closures. I want to dive into the use cases that are on my mind and elaborate on them. I also want to take this case to write up a bit more thoroughly how I think closures should work, including a few unrelated issues.

Syntax and use cases

Here is a list of use cases to be accommodated:

1. “Higher-order functions”: simple functions like map, fold and so forth. By far the most common use case.
2. “Once functions”: functions that can only execute once. This means that they can move values out of their environment.
3. “Sendable functions”: functions that can be sent between tasks. This means that they only close over “sendable” values (no garbage-collected data or borrowed pointers).
4. “Sendable once functions”: sendable functions that can only execute once. This is what a task body will be.
5. “Const functions”: functions that do not close over mutable state. We don’t make much use of this yet, but I plan to do so in order to achieve lightweight fork-join parallelism a la PJS.

The use cases above seem to me to be the “bread and butter” cases that will arise frequently. I will go over the syntax and give an example for each of those use cases shortly. Interestingly, I think that all of them actually read reasonably well if the sigils are moved after the fn keyword, and in some cases the examples read much better.

However, there are two additional use cases that I have considered in the past which I left out. These use cases become significantly harder to read under the new proposal (though they were always hard to read). Interestingly, I realized while writing this blog post that I think these use cases are no longer terribly important, since both of them can be expressed equally well using objects instead of closures, as I will explain shortly. The two use cases are:

1. “Sendable const functions”: functions that can be sent between tasks and do not close over mutable state. You could safely share such functions between tasks in an ARC (atomically referenced counted container) and execute them multiple times in parallel.
2. “Combinators”: combinator libraries create and return closures that closure over their arguments, which may include borrowed values.

Higher-order functions

Here is an example of a simple higher-order function (with the closure type highlighted):

impl<T:Sized> for [T] {
    pub fn map<U:Sized>(f: fn(&T) -> U) -> ~[U] { ... }
                        // ^~~~~~~~~~~
}

For contrast, this is &fn(&T) -> U today.

Once functions

Here is an example of a higher-order function that executes at most once:

impl<T:Sized> for Option<T> {
    pub fn each(f: once fn(&T) -> bool) -> bool { ... }
                // ^~~~~~~~~~~~~~~~~~~
    }
}

For contrast, this is &once fn(&T) -> U today.

Sendable functions and sendable once functions

Here is an example of a sendable once function:

fn spawn(f: once fn~()) {...}
         // ^~~~~~~~~~

The ~ after the fn tells the type system that the environment for this function is allocated using an owned pointer. It also implies a default bound of Owned. The once tells the type system that the function will only execute once.

For contrast, this is ~once fn() today.

Const functions

Here is an example of how I would use a const function to achieve lightweight parallelism:

impl<T:Sized> for [T] {
    pub fn par_map<U:Sized>(f: fn:Const(&T) -> U) -> bool { ... }
                            // ^~~~~~~~~~~~~~~~~
}

This is a parallel map function. It is similar to the regular map except that its iterations execute in parallel. As a consequence, it demands a fn:Const rather than a fn—the Const bound specifies that all the environmental state must be immutable. This is exactly the “patient parent” or “parallel closures” model that is used in PJS and described in this HotPar paper I wrote.

For contrast, this is &fn:Const() today.

Sendable const functions

Sendable const functions are one of the two cases that I said would become less attractive under the new proposal. They would look something like fn~:Const (vs ~fn:Const today). The newer syntax works and should be available, but it’s hard to read, due I think to the juxtaposition of ~ (which specifies the kind of pointer used for the environment) and the : that begins the bound specifier :Const. If this use case were important, I might be worried that the syntax is too ugly, but when I tried to come up with an example for where this use case would be needed, I realize that time has left the use case behind to some extent.

The primary use case for a sendable const function initially was to allow hashtables to be placed in ARCs—the reason for this was that a HashMap requires closures for for computing the hash function of its argument, and those to share the hashmap (and perform parallel lookups) we had to be sure that the closures would not mutate any state. However, this is somewhat outdated, because hashing and equality comparison today is based on traits rather than closures.

Now, using traits is somewhat limited, because due to coherence it means that any one type can only be hashed in one way, and sometimes you would like to have specialized hashing for specific circumstances. But these use cases can easily be accommodated in three ways:

1. Using newtyped keys (struct MyKey(key)) and defining different implementations for the hashing and equality traits on MyKey.
2. If a newtyped key is not acceptable, you can write a hash table that takes a simple function pointer (extern "Rust" fn) rather than a closure. Function pointers carry no state, but state is rarely needed for equality comparisons.

3. If you really need state, then you can write a specialized trait in lieu of a closure:

   trait HashFuncs<K> {
       fn hash(&self, k: &K) -> uint;
       fn eq(&self, k1: &K, k2: &K) -> bool;
   }
   

   Now your hashtable can either take a ~HashFuncs object to use for hashing and equality comparison or, if you wish to avoid dynamic dispatch for performance reasons, you can parameterize your hashtable type by the instance of HashFuncs that it should use:

   struct MyHashMap<K,V,F:HashFuncs<K>> {
       f: F,
       ...
   }
   

Combinators

General purpose combinators are the other case that (might) get less attractive. This is less clear cut. The idea of a combinator library is that you have functions that return functions, and then you can compose these functions into bigger functions. The most common example is a parser combinator, which is a simple way to create inefficient and buggy parsers (ok, that’s unfair, but I couldn’t resist; I’ve had some bad experiences trying to scale up parser combinators—truth is, they are super nice to work with, at least until things go wrong).

Anyway, a typical parser combinator library would begin with a primitive like the following:

fn expect(c: char) -> fn@(&mut ParseState) -> Result<(), Err> { ... }
                   // ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Note that the function returns a closure. We used fn@ because this closure must be allocated on some heap in order for us to return it, and because using the type fn@ (vs say fn~) would allow us to close over managed and other task-local data. So far, I think this example works out fine.

Where things get more complex is if we want to close over borrowed pointers. For example, imagine an expect function that takes a slice:

fn expect_string<'a>(s: &'a str)
                     -> fn@:'a(&mut ParseState) -> Result<(), Err> {...}
                     // ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Here the type system will require that the lifetime 'a of the input slice s appear in the resulting function type, so that it can be sure that the function is not used after the slice is no longer valid. This makes the type more complicated: fn@:'a (vs the also-not-especially-intuitive notation of @'a fn today).

Of course, one could address this problem by having expect_string take a ~str or @str instead of a borrowed string, but in some use cases borrowed pointers may perfect sense. For example, I had once thought to use this pattern to create a combinator library for expressing iteration primitives like enumerate and so forth (similar, experimental work is now underway in the iter module).

Interestingly, just as with sendable const closures, objects and traits can provide an alternative that is ultimately (I think) a better and more readable design anyway. We could rewrite the return type from a closure into a trait:

trait Parser<R> {
    fn parse(&mut ParseState) -> Result<R,Error>;
}

fn expect(c: char) -> @Parser<()>;
fn expect_string<'a>(s: &'a str) -> @Parser:'a<()>;

Here in the expect_string case I have taken advantage of the fact that object types will also carry bounds similar to closure types. An advantage of this design is that using a trait allows the Parser objects to carry more methods as well.

If we were to extend the example to include an actual combinator, I imagine it would look something like this:

fn or<'a, R>(p1: @Parser:'a<R>, p2: @Parser:'a<R>) -> @Parser:'a<R> {...}

Of course, for maximum efficiency, one would avoid using object types altogether. Then you would just implement Parser directly on the char and &str types, and perhaps write the or combinator like so:

struct or<P1,P2>(P1, P2);

impl<R,P1:Parser<R>,P2:Parser<R>> Parser for or<P1,P2> {
  fn parse(&self, state: &mut ParseState) -> Result<R,Error> {
    let (ref p1, ref p2) = *self;
    state.try(); // (*)
    match p1.parse(state) {
      Ok(r) => { state.confirm(); Ok(r) }
      Err(_) => { state.backtrack(); p2.parse() }
    }
  }
}

// (*) Here you see my imperative roots. A true functional
// programmer would not use in-place mutation here but rather
// clone and return a new parser state.

Summary

Another long post mostly targeted at rust devs and myself. Sorry about that. I think the bottom line is that we should move sigils for closures and have them appear after the fn keyword. This makes me sad, because this is how things used to be, and in fact one of the main goals of the dynamically sized types (DST) proposal was to move the sigils in closure types in front. But of course soundness comes first, and I think the general wins of the DST proposal (consistent behavior for all &T, @T, ~T etc) outweigh the need to write fn~ on occasion (I don’t really see much use for fn@).

There is also one final solution I didn’t mention in my initial paragraphs. We could adopt the “principled” solution of using &mut for closures but change the way we notate &mut. I have largely avoided thinking about because I want to avoid destabilizing syntax changes. However, I have toyed around occasion with an idea for reorganizing our types to emphasize ownership and de-emphasize mutability, which goes in this direction. I may indulge myself and write it up at some point. Still, I largely consider this a non-starter.

Adopting the “move sigils in back” proposal does have another casualty, though. There has been some talk of figuring out ways to make @ and ~ less special (as in, allowing user-defined pointer types like RefCounted<T> that are on equal footing). The DST proposal is clearly a step in that direction. Moving the sigils backwards on fn types is, well, a step backward, because closures would always be allocated using a limited set of allocators (stack, ~, or @).

In an odd way, finding this interaction makes me feel good. I’ve been concerned that the DST proposal seemed too easy, which meant we weren’t thinking hard enough about it. But there is another reason as well: I have also been concerned that closure types were becoming a bit too… special, particularly with regard to copyability. Basically I’ve been concerned that although the syntax for a borrowed closure was &fn, borrowed closures didn’t really behave like & pointers—without the DST proposal, this was something that we could safely enforce as part of the type system, but it’s still confusing for users. So I think the DST proposal forces us to be more honest, and that’s a good thing all around.

In the spirit of “no stone left unturned”, I thought I’d write out this idea. At first I thought this was a rather futile exercise, since any large changes to Rust have to pass a pretty high bar at this point, but now that I’ve thought the idea through, I think it has a lot of merit and is worth considering.

A change to representation

For the purposes of simplicity, I will focus on vector types in this blog post, though I think that many of the same considerations apply to other types like closure and trait types (as well as strings, but those are really just newtyped vectors to the compiler).

In the compiler today, both a ~[T] and an @[T] are represented as a Box<Vector<T>>* where the Box and Vector types are defined as follows (here N is the length of the vector, which naturally is not known until runtime):

template<class T>
struct Box {
    type_descriptor_t *type_desc;
    ...
    T payload;
}

template<class T>
struct Vector {
    unsigned length;
    T[N] elements;
}

(The fact that ~[T] uses a box is not actually necessary, it was done as part of the early work on tracing GC and will eventually be undone, at least for those cases where the type T does not itself include managed pointers)

However, today, a slice &[T] is represented quite differently. It is in fact a Slice<T> type, where Slice is defined as follows:

template<class T>
struct Slice {
    T* elements;
    unsigned length;
}

The reason for this is that we wish a slice to be a subset of another vector, which is enabled by this two-word representation.

What I’d like to do is to use two words for all vectors. Therefore, the layout for ~[T] and @[T] will be:

template<class T>
struct Vector {
    Box<Elements<T>>* elements;
    unsigned length;
}

template<class T>
struct Elements {
    T[N] elements;
}

What does this new representation buy us?

Notice that, apart from the box header, this means that a ~[T] or a @[T] is in fact a valid slice. This is exactly like any other ~T or @T pointer, which has the same format as a &T pointer but for the box. This is actually quite similar to how we handle object types (@Trait vs &Trait) and closure types (@fn(), &fn()).

This means that we can define our Rust type hierarchy as follows:

T = S            // sized types
  | U            // unsized types
S = &'r T        // region ptr
  | @T           // managed ptr
  | ~T           // unique ptr
  | [S, ..N]     // fixed-length array
  | uint         // scalars
  | ...
U = [S]          // vectors
  | str          // string
  | Trait        // existential ("exists S:Trait.S")
  | fn(S*) -> S

Note that I have divided the types into two groups. Sized types indicate values whose size is known to the compiler. Unsized types represent values whose size is not known the compiler (this terminology is somewhat imprecise; unsized values do in fact have a size, but it is not known until runtime). Note that unsized types are generally only legal behind a pointer; that is, you can’t have a type like ~[[int]], which would be an array of arrays, where each subarray could have a different size. You could have ~[~[int]]—an array of pointers to arrays—or ~[[int, ..4]], an array of fixed-length arrays of size 4.

Pointers to values of unsized type (e.g., @U, &U) are “fat” pointers, meaning that at runtime they are represented by a pair ((pointer, meta)). The first word is a pointer to the data, and the second word (meta) is some kind of descriptor that indicates what size the data has. The exact nature of this descriptor will change depending on the type U, but there is always something there (for vectors, the meta value is just a length; for objects, it’s a vtable; etc). Standard pointer operations (notably borrowing) are applied to the pointer portion of this pair but leave the meta portion intact.

Writing generic code in the face of unsized types

Using this definition of types means that we can write and compose generic impls that operate over types like @T, ~T, and [T], instead of writing impls, like the following:

impl<T:ToStr> ToStr for @T {
    fn to_str(&self) -> ~str {
        let @ref v = *self;
        fmt!("@%s", v.to_str())
    }
}

impl<T:ToStr> ToStr for ~T {
    fn to_str(&self) -> ~str {
        let ~ref v = *self;
        fmt!("~%s", v.to_str())
    }
}

impl<T:ToStr+Sized> ToStr for [T] {
    fn to_str(&self) -> ~str {
        let mut result = ~"";
        let mut prefix = "";
        result.push_char('[');
        for self.each |v: &T| {
            result.push_str(prefix);
            result.push_str(v.to_str());
            prefix = ",";
        }
        result.push_char(']');
    }
}

This replaces the impls we must write today, which would be over ~T, @T, ~[T], @[T] (and &T and &[T], typically, but I didn’t include those in the above example).

However, there is a catch. The compiler must ensure that unsized types do not appear in illegal locations. For example, we cannot have a local variable of unsized type, because that would require an unknown amount of stack space. Similarly, we cannot have a vector whose elements are unsized. In fact, this is visible in the previous code snippet: if you look carefully at the impl for [T], you will see that the type T is declared with a bound Sized:

impl<T:ToStr+Sized> ToStr for [T] { ... }

This indicates that the type T must be a sized type.

In practice, I suspect we wouldn’t have to write the Sized bound very often. This is because the traits Copy and Clone must extend Sized, since they return a new instance of the receiver, and you can’t return an unsized type (note that functions must take sized arguments and return sized values). Today, most generic functions fall into two categories: those that copy values around, and those that manipulate them solely by reference. The former would require a Sized bound, but then they also require a Copy bound, which implies Sized. The latter do not require Sized at all.

In summary

In summary, I think we can have our cake and eat it too. If we change the representation of vectors and slices, we can have composable types and all the efficiency and flexibility of the current system. The price is that we must distinguish “sized” from “unsized” type parameters. I argue that this is likely to be a minor cost, since most of the time parameters that would require a Sized bound will already have a Copy or Clone bound anyhow. I think that’s pretty exciting, since the non-composability of vector types has always seemed like a language wart in the making.

From my point of view, the subset below includes basically all the JavaScript syntax that I ever use. There are two primary limitations that I think people will encounter in practice:

1. The fact that mutating shared state boots you out of parallel mode. This restriction is (I think) easy to understand, but it will often take some restructuring to obey it.
2. The fact that strings and many native objects (regular expressions, DOM objects, etc) are currently not supported. I expect we’ll improve the support for strings in the near term and also for some native objects, but for others—notably DOM—it will be very challenging to make things work in parallel, due to the many complex implementation details.

OK, let’s get to the details. In the list that follows, we often refer to a plain old JavaScript object (POJO), which means an object defined in JavaScript, not a built-in object. It should be created either with a literal ({...}, [...]) or a new C expression where C is a user-defined function. I’ve also written bug for cases where the current implementation is more limited than it should be.

• a: Variable access
  • You should be able to access any variable in scope, so long as you do not use with or eval. I think this all works fine today, though there may be errors in the implementation today relating to infrequently used access patterns, such as "use strict".
• a.b: Property access
  • If a is a POJO and b is a data property
    • no getters (should work someday)
• a[e]: Element access
  • If a is a POJO or a TypedArray
• a + b, a - b, etc: Binary operators
  • for any primitive values (someday we should be able to support all objects)
  • bug: today, only works if a and b are both numbers or bools (in any combination)
• a === b, a !== b: Strict equality and unequality
  • always works
• a == b, a > b, etc: Loose relational operators
  • works if a and b are both numbers or bools (in any combination)
• a[i] = b: Numeric property assignment
  • if a is a POJO or TypedArray owned by current task, and i <= a.length (no holes—not yet, anyway)
  • bug: today, we must successfully predict that a will be an array or typed array. In practice, this preciction is reasonably successful, but problems arise when the same function is called many times from different contexts.
• a.e = b or a["e"] = b: String property assignment
  • if a is a POJO owned by the current task
  • bug: today, we must successfully predict the offset of e within a. In practice, this prediction often fails, as the code must be written very, very carefully for it to work.
• {...}, [...], new C(...): object literals and creation
  • for new C(...), C must be a JavaScript function
• f() and a.m(...): Function and method calls
  • If the function being called is a user-implemented function, or one of the functions in the following list:
    • higher-order functions like map, reduce, etc
    • parallel higher-order functions like pmap, preduce, etc
    • Array.push (presuming the receiver is writable)
      • bug: today only a[a.length] = e works, not a.push(e)
    • Math.*
      • bug: today, most but not all Math functions work, and only if we predict the function that will be called and are able to inline it. In practice, this prediction is almost always successful.
    • (more to come)
• function(a, b) { ... } or (a, b) => { ... }: closure creation
  • this should basically always work, I think
  • bug: the => syntax doesn’t work yet in parallel execution, I don’t believe
• if, while, etc
  • works fine

One caveat I should point out in the current implementation: even if you stick to the above subset, it is possible that some parallel iterations will abort, generally because of mispredicted types or other transient errors. But I consider a parallel abort to be ok if the engine will eventually stabilize and all subsequent runs will be successful. This is the same as what happens with JIT engines, which often generate code that is later invalidated and recompiled.

The problem stems from the fact that, if you are clever, you can get a stack closure to recurse (that is, to call itself again with the same environment). This would mean that while the stack closure has a new set of local variables, the variables it inherits from its environment are the same. When the borrow checker was first written, this was not true, but it is now, since closures were generalized in the meantime.

Executive Summary

My proposed fix is a change that will guarantee statically that stack closures cannot recurse (that is, cannot call themselves with the same environment). I’ll go into the details of the problem and my proposed fix in the post, but I wanted to start by briefly summarizing what the effects would be on end-users.

• Almost all if not all existing higher-order functions would still work fine. In particular, you can call &fn closures normally and pass them as a parameter to another function and so on. What would be a little more subtle is storing &fn closures into data structures; it’d still be supported, but some patterns that would be legal today would become illegal.

• The “liveness” pass, which checks that all variables are initialized, can be generalized to permit closures to move out from local variables so long as they move a new value back in. This means that foldl can be implemented without copies:

  function foldl<A,B,I:Iterable<B>>(a0: A,
                                    iter: &I,
                                    op: &fn(A,&B) -> A) -> A {
      let mut result = a0;
  
      // Here I am deliberately desugaring the `for` syntax,
      // because I want to emphasize that this is a closure:
      iter.each(|b| {
          // Note: A is not copyable, therefore this call
          // *moves* from result into `op()` and then restores
          // it. This did not used to be legal because we are
          // executing in a closure, and we were afraid
          // that `op` might in fact somehow recurse, in which
          // case it would find that `result` is uninitialized.
          result = op(result, b);
          true
      })
  }
  

What would not work would be:

• Passing the same &fn closure as a parameter more than once, or calling a &fn closure with itself as an argument (no Y combinators).

• Making a struct S that contains &fn closures and then calling those closures via an @S or &S pointer, like so:

  struct S {f: &fn()}
  fn foo(s: &S) { s.f() } // would be illegal
  

  You could write this code using an &mut S pointer, though:

  struct S {f: &fn()}
  fn foo(s: &mut S) { s.f() } // would be illegal
  

  The reason for this is that &mut pointers are non-aliasable. Similar rules arise in the revised borrow checker I’ve been working on for various corner cases where aliasing is a concern. I’ll have a post on that at some point too.

What is the problem, anyway?

Here is an example of an unsound function:

struct R<'self> {
    // This struct is needed to create the
    // otherwise infinite type of a fn that
    // accepts itself as argument:
    c: &'self fn(&R)
}

fn innocent_looking_victim() {
    let mut vec = ~[1, 2, 3];
    conspirator(|f| {
        if vec.len() < 100 {
            vec.push(4);
            for vec.each |i| {
                f.c(&f)
            }
        }
    })
}

fn conspirator(f: &fn(&R)) {
    let r = R {c: f};
    f(&r)
}

What happens when you run this function is that the vector vec is pushed to while it is also being iterated over, which is supposed to be impossible. The root cause of this problem is that the borrow checker generally assumes that &fn closures do not recurse (which, when it was first written, was true). Because of this, the closure f which is passed to conspirator is permitted to freeze vec, because it looks to the borrow checker like it can track all the possible aliases of vec and it sees that this action is ok. But the borrow checker is of course mistaken here, since the closure f is passed to itself as an argument, and thus there is an alias of vec, capured in the closure environment.

The problem lies in the &fn closures, which effectively create implicit references to the data they capture. I tried to make up an example showing what that function looks like if state is passed explicitly, but due to the problem of recursive types it is quite tedious, so I’m going to, um, leave it as an exercise to the reader.

Anyhow, my solution has two parts:

1. Modify the borrow checker to treat these implicit references just like any other reference in the borrow checker. Basically the model should be that when a stack closure with lifetime 'a is created, its contents are opaque to the creator, except that any data which it references is considered borrowed for the lifetime 'a. The type of borrow will depend on how the variable is used (for example, is it read? mutated? borrowed from within the closure?). In the case above, the variable vec would be borrowed mutably, since it is pushed to.
2. Guarantee that closures cannot recurse, because otherwise we’d have to treat every upvar as potentially aliased, which would make most programs illegal.

Let’s look at those changes in more detail.

Modifications to the borrow checker

The basic idea would be to examine the body of each closure as we are conducting the borrow check to examine what free variables it references and how. This is fairly straightforward to do: the borrow checker conducts a walk of the AST already to find all the functions it must check, so basically what we would do is to analyze functions on the way up the tree. So we would analyze each closure first, assuming it has total access to the upvars of the parent. We would then compute a list of the upvars that the closure borrowed and what level of access it required. In the parent fn, when we find a closure expression, we would not examine the body of the closure but rather just treat it as taking out loans that persist for the lifetime of the closure. This is very similar to what we have to do for once fns and also what we do for moves (I guess that’s another post, though).

Guaranteeing closures cannot recurse

The idea here would be to make all &fn closures non-copyable. Basically this would mean the only copyable closure type would be an @fn:

• ~fn is non-copyable
• ~once fn is non-copyable
• &fn will become non-copyable
• &once fn is non-copyable
• @fn is copyable
• @once fn is non-copyable

At first I thought that @once fn was not necessary, but in fact it is potentially useful for combinator libraries and the like, as it allows you to return a fn that can move out of its environment.

For fun, let’s review the full closure type specification from a previous blog post, modernized somewhat and taking these changes into account.

(&'r|~|@'r) [unsafe] [once] fn [:K] (S) -> T
^~~~~~~~~~^ ^~~~~~~^ ^~~~~^    ^~~^ ^~^    ^
   |          |        |        |    |     |
   |          |        |        |    |   Return type
   |          |        |        |  Argument types
   |          |        |    Environment bounds
   |          |     Once-ness (a.k.a., affine)
   |        Effect
Allocation type and lifetime bound

One part I had hoped to remove was the environment bounds, but I think they are still necessary. The only real use case for this is :Const, which would be a way of saying that the closure only closes over deeply immutable data. This enables parallelism in various ways (putting closures in ARCs, fork-join parallelism a la PJS, etc). Conceivably we could also support :Clone, which would permit closures to be cloned, but we’d need some magic support in trans (code which, admittedly, mostly exists) to make that work.

Some musings on orthogonality or lack thereof

It annoys me that the rules for closures feel… one-off. I considered briefly if we were not categorizing things correctly. To some extent, the answer is clearly yes: there are many partly orthogonal characteristics of closures (once-ness, type of pointer used to reference its data, kinds of loans it requires, etc). Ultimately, we are trying to boil this down into a relatively small set of types that covers all important use cases.

A similar phenomena occurs with & and &mut: there are really two characteristics, aliasability and mutability, and we have joined them together, such that & references are immutable and aliasable and &mut are mutable and unaliasable. This is typically what you want, but there are rare occasions where you must use &mut solely for its non-aliasable nature and not because of mutability. In particular, if you want access to other non-aliasable things, such as other &mut pointers or (per this post) &fn closures.

In the beginning of the post I wrote that the following example will not work:

  struct S {f: &fn()}
  fn foo(s: &S) { s.f() } // would be illegal

The reason for this has to do precisely with the fact that &S pointers are always aliasable. Hence we could not permit s.f to be called because we can’t guarantee that there are no aliases to s lurking around, and thus creating aliases to s.f. You could fix this program by using &mut:

  struct S {f: &fn()}
  fn foo(s: &mut S) { s.f() } // legal

In this case, we don’t care about mutability, we do care about uniqueness.

I was debating for a time whether to suggest adding more facets to the various types. For example, one could imagine &, &alias, &mut, and &mut alias. But I think that ultimately, this is a bad idea. For one thing, the correct aliasability default varies, so you’d probably want something like &, &noalias, &mut, &mut alias. The type system ultimately feels more complex, with many branches (case in point: see the full closure type specification above!).

Background: Issue #5656 and the case of the many cursors

To explain the problem I encountered, consider this running example (written as one would write it after my patch):

struct OwnedCursor<T> {
    buffer: ~[T],
    position: uint
}

pub impl<T> OwnedCursor<T> {
    fn get<'c>(&'c self) -> &'c T {
        &self.buffer[self.position]
    }

    fn move(&mut self, by: int) -> bool {
        if (by > 0) {
            self.position += by as uint;
        } else {
            self.position -= by as uint;
        }
        self.position < self.buffer.len()
    }
}

This defines a type OwnedCursor that owns a vector and a position within that buffer. The type offers two methods, get() and move(). The method definitions themselves should be fairly self-explanatory. get() returns a pointer to the current item and move() modifies the current position.

What is interesting is the lifetimes on the function get(). The signature indicates that it takes a borrowed pointer to an OwnedCursor with lifetime 'c and returns a pointer with that same lifetime. In other words, the &'c self declaration means that the method get() is roughly equivalent to a function written as follows:

fn get<'self, T>(self: &'c OwnedCursor<T>) -> &'c T {
    &self.buffer[self.position]
}

The reason that we can say that the returned value has the same lifetime as the input is that (1) we know that the pointer self will be valid for the entirety of the lifetime 'c and (2) self is an immutable pointer, so the field buffer will not be mutated. So we can say that, for the lifetime 'c, the OwnedCursor object will not be freed and it is immutable, therefore we can take a pointer into self.buffer and know that this memory is also valid.

Now let’s suppose that we wanted to develop many kinds of cursors. We might introduce a generic trait and convert the impl to use it:

trait Cursor<T> {
    fn get<'c>(&'c self) -> &'c T;
    fn move(&mut self, by: int) -> bool;
}

impl<T> Cursor<T> for OwnedCursor<T> {
    // as before
}

Now we can introduce a second kind of cursor, one that doesn’t own the vector that it iterates over:

struct BorrowedCursor<'b, T> {
    buffer: &'b [T],
    position: uint
}

impl<'b, T> Cursor<T> for BorrowedCursor<'b, T> {
    fn get<'c>(&'c self) -> &'c T {
        &self.buffer[self.position]
    }

    fn move(&mut self, by: int) -> bool {...}
}

This definition is very similar, except that the type and impl are parameterized by a lifetime 'b, representing the lifetime of the buffer. Everything seems fine, but when I tried running this example through the compiler with my patch, I encountered a type error on the get() routine. The compiler reported that the lifetime of &self.buffer[self.position] was not 'c but rather 'b—the lifetime of the buffer, and so the return type of the function was invalid.

Unfortunately, the compiler is quite correct! The function signature states that we return a pointer with the lifetime 'c, but here 'c is the lifetime of the cursor. Our pointer is a pointer into the buffer, so it will have the lifetime 'b.

In the original OwnedCursor type, the buffer was owned by the cursor, so the result had identical lifetimes. But in the case of BorrowedCursor, the lifetime of the buffer is not tied to the cursor object, which after all is only borrowing the buffer.

Of course, it’s kind of nonsensical to have a cursor that outlives the buffer it’s working with. But there is in fact nothing in the type system that would prevent you from doing that—it’s just that once your buffer became invalid you would be prevented from actually using the cursor anymore.

The solution

This problem was quite vexing at first. The example I just gave is perfectly reasonable and it really should work (and, in fact, the old system I am attempting to replace allowed it, though in a kind of roundabout and possibly unsound way). The solution I decided on was just to formalize the common sense rule that a pointer should not have a longer lifetime than any other pointers it points at. In other words, if I create a pointer to a BorrowedCursor<'b, T>, my pointer cannot have a lifetime that exceeds the lifetime 'b of the buffer.

If we assume this rule holds, then when you have a function that takes an argument of type &'c BorrowedCursor<'b, T> (such as the self argument to get()), the compiler can deduce that 'c must be a smaller lifetime than 'b, because otherwise the caller would have encountered a type error. This means that the get() method for BorrowedCursor<'b, T> is permitted to return a pointer with lifetime 'c—the largest possible lifetime is still 'b, but 'c is a sound approximation (it can only be shorter than 'b, after all).

I wonder if this problem arises in other similar type systems. I remember Safe Java had a rule that the “main owner” of an object must own all the other owners, or something like that, but I think this was a soundness concern having to do with downcasting. I’ll have to go and re-read the various papers. Anyway, I’ve been debating whether to allow types like &'a &'b uint where 'a outlives 'b for a while. They would seem to have no practical use but I couldn’t think of a reason to add extra rules to prohibit them… until now, anyway.

Parallel execution mode

To support ParallelJS, we introduce a second mode of compilation called parallel execution mode. JavaScript compiled in this mode produces executable code that is suitable to be run in parallel. To accommodate this new mode, each JSScript* potentially contains pointers to two IonScript* data structures, one for standard sequential mode and one for parallel mode.

Execution normally stays confined within one mode. So if you are running a function f in sequential mode and it invokes another function g, then we will run the sequential mode version of g. But if you are running f in parallel mode, it will call the parallel version of g. The only place where we move between modes is in the ForkJoin intrinsic, which invokes the parallel mode script for the first time.

You may wonder why we permit each script to be compiled in both modes simultaneously. The reason is that it is possible to have helper functions and code that runs in both sequential and parallel mode. Imagine, for example, that you have a helper function for searching an array to find the object with a given name:

function findObject(list, name) {
    for (var i = 0; i < list.length; i++) {
        if (list[i].name === name)
            return list[i];
    }
    throw Error("No object with name " + name + " found!");
}

It is perfectly reasonable to want to invoke this helper function both from sequential and from parallel code. If we only permitted a function to be compiled in one mode or the other, we would always be recompiling findObject each time we started or finished a parallel operation.

Differences between parallel and sequential execution mode

The biggest difference between parallel and sequential mode is that code executing in parallel mode is guaranteed to be pure. That is, it can never write to any shared state that might be visible from other threads. This purity requirement generally includes not only user-visible JavaScript state but also internal engine details. For example, in sequential mode code, after we have done several property lookups on an object that has a large number of properties, we will “hashify” the property chain, meaning that we convert it from an array into a dictionary to make later lookups faster. This hashification operation is not visible to the JavaScript user (except insofar as subsequent property lookups are faster), but it is still disallowed in parallel execution mode because it would cause data races.

There are some exceptions to the purity requirement. The first and most obvious is the UnsafeSetElement intrinsic I discussed in part one, which is used to track the progress of parallel work. The second exception is that it is ok to modify internal engine details so long as those modifications are threadsafe. For example, in bug 846111, Shu has implemented threadsafe inline-caching, which is of course a mutation to shared state.

Generally speaking, though, when you call a parallel mode function you can be sure that it will either complete successfully or bailout. In either case, you know that it has no lasting effects that are visible to end-user JavaScript code, except those that might have occurred via the UnsafeSetElement intrinsic (which of course is only usable from self-hosted code and which must be carefully audited).

Changes to the Ion compilation process

There are two major changes when compiling in ParallelExecutionMode: The first change is the so-called “parallel array analysis”, which analyzes the actions that the scripts take and modifies them as needed to ensure that each action is either threadsafe (and pure) or else that the script bails out. The second change is that do not compile a single script in isolation but rather attempt to compile the transitive closure of a starting script and all scripts that it may call.

Parallel array analysis

The parallel array analysis can be found in js/src/ion/ParallelArrayAnalysis.cpp, in the function ParallelCompileContext::analyzeAndGrowWorklist(). It runs after the normal suite of optimizations have taken place. Its primary goal is to ensure that the parallel code will be pure and threadsafe.

To that end, it performs a walk of the control-flow graph and examines each MIR instruction using a visitor. The MIR instructions are categorized into one of several categories, as follows:

• Safe operations are operations that can be safely executed in parallel without changes, such as Constant or Box.
• Write-guarded operations are operations that are safe as long as the value being modified is not shared. To verify this, we insert a write guard before the operation in question. The write guard will cause a bailout should the object be shared (more on the details of this check to come in a later post). N.B.—write guards are not to be confused with write barriers, which have to do with incremental and generational garbage collection.
• Specialized operations are numeric operations that are safe so long as they are operating over scalar data, such as Add, Mul, etc.
• Unsafe operations are operations that are just plain disallowed in parallel execution, generally because we have not made an equivalent threadsafe path. An example is RegExp.
• Custom operations are, well, everything else. Generally speaking these are operations that are not safe by default in parallel mode, but where there exists an alternative version that is safe, such as NewArray or NewObject.

The categorization of instructions is done using macros. The visitor expects one method per MIR instruction type. There are various macros for each of the above categories, and the macro expands into a pre-canned method definition (in the case of custom operations, the macro expands to an out-of-line method, and the method body appears later in the file).

I’ll talk a little bit more about the safe and unsafe operations now, and I’ll cover the other cases (write guards, memory allocation, etc) in later posts.

Safe operations are simply left unchanged, and they execute just as they would in sequential mode (though in some cases there are checks in the CodeGenerator so that the MIR behaves somewhat differently).

When an unsafe operation is encountered, the basic block in which it resides is removed from the graph along with its dominated subtree. In its place, we add a bailout block that will cause parallel execution to bailout should it ever execute. This ensures that unsafe operations that never execute do not prohibit safe code from running.

Transitive compilation

In normal sequential mode, if we encounter a call to a script that is not compiled, we just invoke the interpreter. In parallel mode this option is not available. So what we do instead is to take advantage of the information that TI makes available and, when compiling a script x, collect all scripts that x might call. Then, once we have compiled x, we go on and compile those scripts. The process is transitive, meaning that we will then continue on to compile the scripts that x’s callees might call and so forth until we reach a fixed point.

Note that we do not monitor for hot paths, as we do in sequential mode. That is, we don’t care if the script has been called 10 times or 100 times before. This is for two reasons: one, we assume that parallel paths are going to be hot, since they are going to be called over all the entries in a large array. Two, seeing as we will have to bailout if we encounter a call to an uncompiled script, it’s worth erring on the side of more compilation rather than less. We do however check that the use count of the script is at least one, so as to avoid compiling things that never run.

At runtime, when we see a call to a JavaScript function, we check whether it has been compiled for parallel execution. If so, we can simply call it as normal and carry on. This is the expected case, of course.

If we encounter a call to an uncompiled script, which can happen either because our transitive compilation was incomplete or because the callee was invalidated or garbage-collected in the mean-time, we bailout with an “uncompiled script” error. At this point, control returns to the ForkJoin function I described in my previous post. Presuming that we haven’t encountered too many bailouts yet, ForkJoin will cycle around and try to compile the uncompiled script.

When compiling an uncompiled script, we also set a flag on all the currently executing scripts in the stack trace. This flag is a warning that execution of that script is likely to encounter an uncompiled script. The purpose for this flag is to notify later callers that while the script itself is valid, it likely has callees that have not been compiled, so before running the script in parallel we should re-walk the transitive closure of things it might call and check for anything that is missing.

These proposals have the same descriptive power as what I described before, but they are backwards compatible. This is nice.

Object-oriented style name resolution

In the object-oriented, C++-like version of associated items that I introduced before, the names of associated items and methods were resolved relative to a type. To see what I mean by this, consider a (slightly expanded) variant the graph example I introduced before:

trait Graph {
    type Node;      // associated type
    static K: uint; // associated constant
}

mod graph {
    fn depth_first_search<G: Graph>(
        graph: &G) -> ~[G::Node]
    {
        let k = G::K;
        ...
    }
}

Consider a path like graph::depth_first_search, which names an item within a module. This kind of path is based solely on the module hierarchy and can be resolved without knowing anything types whatsoever.

Now consider the paths G::Node and G::K that appear in depth_first_search. These paths look similar to in form to graph::depth_first_search, but they are resolved quite differently. Because G is a type and not a module, if we want to figure out what names Node and K refer to, we don’t examine the module hierarchy but rather the properties of the type G. In this case, G is a type parameter that implements the Graph trait, and the Graph trait defines an associated type Node and an associated constant K.

Note that the name lookup process here is exactly analogous to what happens on a method call. With an expression like a.b(), the meaning of the name b is resolved by first examining the type of the expression a and then checking to see what methods that type offers. The module hierarchy is not consulted.

The object-oriented style of naming specification is not fully explicit. In particular, a path like G::Node does not specify the trait in which Node was defined, the compiler must figure it out. It is also possible that the type G implements multiple traits that have an associated item Node, so the syntax could be ambiguous. To make things fully explicit, I proposed in my previous post that the full syntax would be Type::(Trait::Item). So the fully explicit form of G::Node would be G::(Graph::Node), since the type Node is defined in the trait Graph.

Functional-style name resolution (take 1)

In Haskell, all name resolution is done based on lexical scoping and the module hierarchy. This is no accident. It means that the Haskell compiler can figure out what each name in a program refers to without knowing anything about the types involved, which is helpful when performing aggressive type inference.

What this means is that we can’t use a path like G::Node to mean “the type Node relative to the type G”, because interpreting this path would require examining the definition of the type G (as we saw before). Instead, if we were to use a syntax analogous to what Haskell uses, one would write something like Graph::Node<G>. Note that all the names here (Graph::Node, G) can be resolved using only the module hierarchy.

So the example I gave before would look as follows:

trait Graph {
    type Node;      // associated type
    static K: uint; // associated constant
}

mod graph {
    fn depth_first_search<G: Graph>(
        graph: &G) -> ~[Graph::Node<G>]
    {
        let k = Graph::K::<G>;
        ...
    }
}

Note that where before we wrote G::K to refer to the constant K associated with the type G, we would now write Graph::K::<G>. As is typical in Rust, the extra :: that appears before the type parameter <G> is necessary to avoid parsing ambiguities when the path appears as part of an expression.

Let’s look a bit more closely at what’s going on here. Effectively what is happening is that, for each associated item within a trait, we are adding a synthetic type parameter. For any reference to an associated item, this type parameter tells the compiler which type is implementing the trait. The path Graph::Node by itself is not complete; Graph::Node<G> means “the type Node defined for the type G”.

Let’s dig into it this Haskell-style convention a bit to see some of the implications.

Return type inference

One benefit of the Haskell style convention is that the values for the type parameters can often be deduced by inference. For example, let’s return to the trait FromStr from my previous post. The trait FromStr is used to parse a string and produce a value of some other type:

trait FromStr {
    fn parse(input: &str) -> Self;
}

We might implement FromStr for unsigned integers as follows:

impl FromStr for uint {
    fn parse(input: &str) -> uint {
        uint::parse(input, 10) // 10 is the radix
    }
}

Now we could write a function that invokes parse() like so:

fn parse_strings(v: &[&str]) -> ~[uint] {
    v.map(|s| FromStr::parse(*s))
}

Note that when we called FromStr::parse(*s), we did not say what type it should parse to. The compiler was able to infer that we wanted to parse a string into a uint based on the return type of parse_strings() as a whole. A fully explicit version of parse_strings would look like:

fn parse_strings(v: &[&str]) -> ~[uint] {
    v.map(|s| FromStr::parse::<uint>(*s))
    //                        ^~~~~~ specify return type
}

Generic traits

Imagine that we have a generic trait, like this Add trait:

trait Add<Rhs> {
    type Sum;

    fn add(&self, r: &Rhs) -> Sum<Self, Rhs>;
}

This trait is very similar to the trait used in Rust to implement operator overloading, except it has been adapted to use an associated type for the Sum (which is probably how the Rust type should be defined as well, since the type of the sum ought to be determined by the types of the things being added).

Previously, with the associated type Node, we said that any reference to node had to include a single type parameter to indicate the type that was implementing Graph. But with a generic trait like Add a simple type parameter is not enough. To fully specify all the types involved, we need to include both the Self type and any type parameters. This is why the return type of the method add() is Sum<Self, Rhs>—a mere reference to Sum or Sum<Self> would be incomplete.

Comparison to object-oriented form. Interestingly, this case is something that the object-oriented style of naming cannot handle very well. This is because the object-oriented convention is strongly oriented towards specifying the Self type but does not easily expand to accommodate generic traits. Using the fully explicit syntax that I suggested in my previous post, I think the result would look like Self::(Add<Rhs>::Sum). (It’s plausible that, within the trait definition itself, one could simply write Sum, but from outside the trait definition I think it would be necessary to specify the full type parameters).

Generic items

It is also possible to have an associated item which itself has type parameters. For example, we might want to have a graph where kind of node can carry its own userdata:

trait Graph {
    type Node<B>;

    fn get_node_userdata<B>(n: &Node<Self, B>) -> B;
}

Here we see that when we refer to the Node type in get_node_userdata, we specify both the Self type parameter and the type parameters defined on Node itself. I think this is a bit surprising.

Comparison to object-oriented form. The object-oriented naming scheme handles this case very naturally. For example, get_node_userdata() would be declared as follows:

fn get_node_userdata<B>(n: &Self::Node<B>) -> B;

Functional-style name resolution (take 2)

In the previous section we added implicit type parameters to each associated item. Particularly in the cases of generic traits or generic items, this can be a bit confusing. You wind up mixing type parameters that were declared on the trait together with type parameters declared on the item.

An alternative that would be a bit more explicit is to (1) designate the implicit parameter for the trait’s Self type using a special keyword, such as for or self (I prefer for since it echoes the impl Trait for Type form) and (2) push the trait type parameters into the path itself. So instead of writing Graph::Node<G> you would write Graph::Node<for G>, and instead of Add::Sum<Lhs, Rhs> you would write Add<Rhs>::Sum<for Lhs>. You’ll see more examples of how this looks in the next section.

Conclusion: Comparing the conventions

I think none of these conventions is perfect. Each has cases where it is a bit counterintuitive or ugly. To try and make the comparison easier, I’m going to create a table summarizing the object-oriented, functional 1, and functional 2 styles, and show how each syntax looks for each of the use cases I identified in this post. For each use case, I’ll provide both the shortest possible form and the fully explicit variant.

Reference to an associated type
G::Node	Node<G>	Node<for G>
G::(Graph::Node)	Graph::Node<G>	Graph::Node<for G>
Reference to an associated constant
G::K	K::<G>	K::<for G>
G::(Graph::K)	Graph::K::<G>	Graph::K::<for G>
Call of an associated function
uint::parse()	parse()	parse()
uint::(Graph::parse())	FromStr::parse::<uint>()	FromStr::parse()::<for uint>
Generic trait
Self::(Add<Rhs>::Sum)	Sum<Self,Rhs>	Add<Rhs>::Sum<for Self>
Self::(Add<Rhs>::Sum)	Add::Sum<Self,Rhs>	Add<Rhs>::Sum<for Self>
Generic associated item
Self::Node<B>	Node<Self,B>	Node<B for Self>
Self::(Graph::Node<B>)	Graph::Node<Self,B>	Graph::Node<B for Self>

Based on this table, my feeling is that the object-oriented style handles the simple cases the best (G::Node, G::K), but it handles the “generic trait” case very badly.

There are also some side considerations:

1. Functional 1 is (mostly) backwards compatible with the current code.
2. Functional 1 provides return-type inference, which many people find appealing.
3. The object-oriented style means that a.b(...) is always sugar for T::b(a, ...) where T is the type of a, which is elegant.
4. The functional styles mean that :: is always module-based name resolution and . is always type-based resolution, which has an elegance of its own.

It’s a tough call, but right now I think on balance I lean towards one of the two functional notations, probably functional 2 because, despite being wordier, it seems a bit clearer what’s going on. Just appending the type parameters from the trait and the method together is confusing.

Appendix A. Functional notation (take 3)

There is one other where you might handle the placement of type parameters in the functional style. You might take the “self” type and place it on the trait: i.e., instead of Graph::Node<for G> you’d write Graph<for G>::Node. This is arguably more correct if you think about traits in terms of Haskell type classes, since the self type is really the same as any other type parameter on the generic trait. But when I experimented with it I found that it was so wordy and ugly it was a non-starter.

Appendix B. Haskell and functional dependencies

In addition to associated types, Haskell also offers a feature called functional dependencies, which is basically another, independently developed, means of solving this same problem. The idea of a functional dependency is that you can define when some type parameters of a trait are determined by others. So, if we were to adapt functional dependencies in their full generality to Rust syntax, we might write out the graph example as something like this:

// Associated types:
trait Graph {
    type Node;
    type Edge;
}

// Functional dependencies:
trait Graph<Node, Edge> {
    Self -> Node;
    Self -> Edge;
    ...
}

The line Self -> Node states that, given the type of Self, you can determine the type Node (and likewise for Edge). You can see that associated types can be translated to functional dependencies in a quite straightforward fashion.

When functional dependencies have been declared, it implies that there is no need to specify the values of all the type parameters. For example, it would be legal to to write our depth_first_search routine without specifying the type parameter E on Graph:

fn depth_first_search<N, G: Graph<N>>(
    graph: &mut Graph,
    start_node: &N) -> ~[N]
{
    /* same as before */
}

The reason that we do not have to specify E is because (1) we do not use it and (2) it is fully determined by the type G anyhow, so there is no ambiguity here. In other words, there can’t be multiple implementations of Graph that have the same self type but different edge types.

Functional dependencies are more general than associated types. They allow you to say a number of other things that you could never write with an associated type, for example:

trait Graph<Node, Edge> {
    Node -> Edge;
    ...
}

This trait declaration says that, if you know the type of the nodes Node, then you know the type of the edges Edge. However, knowing the type Self isn’t enough to tell you either of them. I don’t know of any examples where expressiveness like this is useful, however.

Appendix C. “where” clauses.

There is one not-entirely-obvious interaction between associated types and other parts of the syntax. Suppose that I wanted to write a function that worked over any graph whose nodes were represented as integers (it is very common to represent graph nodes as integers when working with large graphs). If we defined the graph trait using a simple type parameter, like so:

trait Graph1<N> { ... }

then I could write a depth-first-search routine that expects a graph with uint nodes as follows:

fn depth_first_search_over_uints<G: Graph1<uint>>(graph: &G) { ... }

But we saw in the previous post that this definition of Graph has a number of downsides. In fact, it was the motivating example for associated types. So we’d rather write the trait like so:

trait Graph {
    type N;
    ...
}

But now it seems that I cannot write a depth_first_search_over_uints routine anymore! After all, where would I write it?

fn depth_first_search_over_uints<G: Graph>(graph: &G) { ... }

Many languages answer this problem by adding a separate clause that can be used to specify additional constraints. In Rust we might write it like so (hearkening back to the typestate constraint syntax):

fn depth_first_search_over_uints<G: Graph>(graph: &G)
    : G::Node == uint
{ ... }

This is not the end of the world, but it’s also unfortunate, since this kind of clause leaks into closure types and all throughout the language. But while discussing associated types with [Felix][pnkfelix] at some point I realized that there is a workaround for this situation. If you have a trait like Graph that uses an associated type, but you would like to write a routine like depth_first_search_over_uints, you can write an adapter:

trait Graph1<N> { ... } // as before
impl<G: Graph> Graph1<G::Node> for G { ... }

Now I can write depth_first_search_over_uints and have it work for any type that implements Graph.

This adapter trait is not the most elegant solution but it works. I would not expect this situation to arise that frequently, but it will come up from time-to-time. The Add and Iterable traits come to mind.

As the inaugural post in this series, I want to take a look at associated items (e.g., associated types, constants, functions, etc). Associated items are requested often, though under various names. When I first started this post, it was actually part of a larger post, but I quickly found that the topic of associated items was too large to be a footnote of another post. In fact, I’m finding it’s too large to fit into one post at all. So I’ll be breaking this post up until multiple pieces. This first post will cover what an associated item is and what you might want to use it for, and it will do so from a C++ perspective.

I will also propose some changes to how we handle so-called “static” fns (which I will be calling “associated” functions, because the name “static” gives all the wrong connotations). These changes are not backwards compatible. I do not take such an idea lightly; we are trying very hard to stabilize Rust so such changes must pass a high bar (I personally think the change would be worth it, but opinions will vary). In the next post, I will present the Haskell approach to associated items, which is closer to what we have today and which can be adapted in a mostly backwards-compatible fashion.

Associated items sound like some kind of crazy language extension, but they’re actually pretty straight-forward and natural. They are used very frequently both in C++ and Haskell, as well as other languages. To get an idea what you might want one for, imagine you were going to design a generic graph library, and you want to implement some algorithms that operate over any sort of graph.

You might begin with defining a generic graph trait that defines the interface your algorithms will expect to manipulate the graph:

trait Graph<Node> {
    fn get_visited(&self, n: &Node) -> bool;
    fn set_visited(&mut self, n: &Node);
    fn get_successors(&self, n: &Node) -> ~[Node];
    ...
}

The details are not too important but you get the idea. Now, we might implement a function like depth_first_search, which executes a depth first search and returns the nodes we visited in order:

fn depth_first_search<N, G: Graph<N>>(
    graph: &mut Graph,
    start_node: &N) -> ~[N]
{
    let mut nodes = ~[];
    let mut stack = ~[start_node];
    while !stack.is_empty() {
        let node = stack.pop();
        if graph.get_visited(node) {
            loop; // already visited
        }
        graph.set_visited(node);
        nodes.push(node);
        stack.push_all(graph.get_successors(node));
    }
    return nodes;
}

Notice that depth_first_search takes two type parameters, N and G, where N is the type of the nodes used by the graph G. If you think about it, this is a bit odd, because these two type parameters are not really independent. Typically, when one implements a graph, you implement it for a specific kind of node, and only that kind of node. Now, so long as the Graph trait is only parameterized by the type of the nodes, this is not so bad, but in practice a real graph library will grow a number of similar type parameters. For example, we might want the type of the edges, which would give us two type parameters:

trait Graph<Node, Edge> {
    ...
}

fn depth_first_search<N, E, G: Graph<N, E>>(
    graph: &mut Graph,
    start_node: &N) -> ~[N]
{
    ...
}

Already you can see that our signatures are getting complicated. There is another problem as well: even though depth_first_search does not need to consider edges, after all we saw the implementation before and it only needed the type N, we must include the edge type E in the signature.

Now imagine that we want to make an efficient graph type. It is likely that we can use a specialized type to represent a set of edges or nodes; a bitset, for example. In that case, we would want a third and maybe even a fourth type parameter (NodeSet or EdgeSet). The list just keeps growing. And for each such type parameter, we will have to extend the signature of depth_first_search along with every generic function that is implemented over our graph. This is not only unwieldy, it’s a refactoring hazard that will limit the ability of people to write generic libraries.

Enter associated types

C++ had a similar problem in the design of the STL. Because C++ traits are basically just macros, however, clever C++ programmers were able to come up with a useful pattern that avoids all these hazards (I probably have my history wrong here, no doubt C++ programmers adapted a solution first used in other languages, perhaps without even knowing it, but it reads better this way, doesn’t it?). Instead of defining the trait Graph as being parameterized over the node type Node, define the node type Node as an “associated type”:

trait Graph {
    type Node; // associated type

    fn get_visited(&self, n: &N) -> bool;
    fn set_visited(&mut self, n: &N);
    fn get_successors(&self, n: &N) -> ~[N];
    ...
}

Notice that the definition of Node has moved inside the trait. The meaning of this is that any given Graph implementation will define a type Node that represents nodes. That is, rather than Node being a “input” to the trait, it is an “output”, just like the functions get_visited() etc are “outputs”.

Now we can adapt our depth_first_search routine as follows:

fn depth_first_search<G: Graph>(
    graph: &mut Graph,
    start_node: &G::Node) -> ~[G::N]
{
    /* same as before */
}

Note that depth_first_search only takes one type parameter, the graph type G. The type of the node is then relative to G (so G::Node would be “the node type used by the graph type G”).

Interestingly, I can now add as many associated types to Graph as I like without affecting the signature of depth_first_search in the slightest.

Associated constants

It is not hard to imagine extending this idea to other kinds of associated members. For example, we might write up a trait like Vector that has an associated constant specifying the number of dimensions in vectors of this type:

trait Vector {
    static dims: uint;
    fn get(&self, dim: uint) -> uint;
}

Now I can write up an implementation, say for a two-dimensional point type:

struct Point2D { x: uint, y: uint }

impl Vector for Point2D {
    static dims: uint = 2;
    fn get(&self, dim: uint) -> uint {
        assert!(dim < 2);
        if dim == 0 {self.x} else {self.y}
    }
}

And then I can use this with generic code:

fn sum<V: Vector>(v: &V) -> uint {
    let sum = 0;
    for uint::range(0, V::dims) |i| {
        sum += v.get_dim(i);
    }
    return sum;
}

Associated functions

Associated functions are useful in a couple of different contexts. One common example is where you would like to define a trait that includes some sort of constructor, such as FromStr:

trait FromStr {
    fn parse(input: &str) -> Self;
}

Here the trait defines an associated function parse() that will parse a string and return an instance of the Self type. I could for example implement FromStr for integers:

impl FromStr for uint {
    fn parse(input: &str) -> uint {
        uint::parse(input, 10) // 10 is the radix
    }
}

Using FromStr, I can write a generic routine that, for example, parses a comma-separate list of values:

fn parse_comma_separated<T: FromStr>(input: &str) -> ~[T] {
    let substrings = input.split(",");
    substrings.map(|substring| T::parse(substring))
}

Experienced Rust users might note that the syntax in that example is actually not what one would write today. This is the “non-backwards-compatible change” I alluded to earlier. In Rust today, when one invokes an associated function, it is not named via the self type as I did above, but rather it is named via the trait to which the function belongs:

substrings.map(|substring| FromStr::parse(substring))

The compiler uses inference to decide that the return type here is T and therefore the self type for this call to parse must be T. This approach is elegant in many ways, as I’ll cover in the next post in more detail, but it also has some downsides. Perhaps the most serious is that, if the associated function does not return an instance of Self, then the compiler cannot disambiguate what version of the function you are trying to call!

To see where you might have an associated function that does not return Self, consider a trait like the following:

trait TemperatureUnit {
    fn to_kelvin(f: float) -> float;
}

Using the C++-approach I have been describing thus far, I could write a generic function like:

fn do_some_chemistry<TU: TemperatureUnit>(f: float) -> float {
    let kelvin = TU::to_kelvin(f);
    ...
}

Of course, this example is somewhat artificial, because one would be better off integrate the temperature units as types in your type system rather than using floats. But real examples like this do come up. The associated constant V::dims is an example.

So is there a proposal here?

Yes and no. Partially I just wanted to explain what an associated item is and what you might use it for. But I’ve also kind of baked in an alternate proposal for how we should address associated items, which is to switch from a Haskell-like approach to a C++-like approach. In the next post, I’ll explain how the Haskell solution works, and what it would look like in Rust. Frankly the difference is not so great so it’s a matter of taste.

Anyway, if you wanted to implement the scheme I’ve described in this post, it would work as follows. When resolving a path, if you find that some prefix of the path evaluates to a type, then later elements in the path are resolved using the same algorithm that we use today for method lookup. So, to look at our examples, if I wrote G::Node, the path G here is a type, which means that the type Node would be determined by examining the traits that are in scope to see whether any of them both (1) define a type member Node and (2) are implemented by G.

This is exactly analogous to how method lookup operates. When you see a call a.b(), we determine the type T of the expression a and then look to see whether any of the traits which are in scope (1) offer a method b() and (2) are implemented by T.

In fact, it’s a bit more complex, because we also consider the inherent members of a type that are defined without any trait at all. We can do the same thing when resolving associated items.

Interestingly, unifying the algorithm used to specify associated items and method calls also allows us to say that a call like a.b(...) is just sugar for T::b(a, ...) where T is the type of a.

Corner cases

There are a few corner cases to consider in this proposal.

Ambiguous references

It is possible to have two traits A and B that define the same associated item I. If both those traits are imported, and both those traits are implemented by the same type T, then a reference like T::I could refer to the item defined by A or the item defined by B. If we wish to provide an explicit syntax to disambiguate the reference, it could be something like T::(A::I). That is, we refer to the item I as defined in the trait A implemented for the type T.

Another possible ambiguity can arise when you have a generic trait. Consider something like the following:

trait Getter<T> {
    static default: T;
    fn get(&self) -> T;
}

Now imagine that I have some type with two implementations of Getter:

struct Circle {
    center: Point, radius: float
}

impl Getter<Point> for Circle {
    static default: Point = Point {x: 0, y: 0};
    fn get(&self) -> Point { self.point }
}

impl Getter<float> for Circle {
    static default: float = 0;
    fn get(&self) -> Point { self.radius }
}

If I then write a generic routine such as:

fn is_default<G: Getter<float> Getter<Point>>(g: &G) -> bool {
    let x = G::default;
    let y = g.get();
    x == y
}

Then what value for default is G::default are we going to obtain? The Point or the float?

Using the syntax that I proposed, one could write this unambiguously, if verbosely:

fn is_default<G: Getter<float> Getter<Point>>(g: &G) -> bool {
    let x = G::(Getter::<float>::default);
    let y = g.(Getter::<float>::get)();
    x == y
}

Not all types are paths

Another problem is that if you wanted to get an associated member of a type like ~[int], you couldn’t write ~[int]::foo. But this is easily circumvented by creating a type alias

type T<U> = U;

and writing T::<~[int]>::foo, or else by permitting the syntax <~[int]>::foo.

Now, I definitely think it is a good idea to clearly define the subset of JavaScript that our engine will be able to execute in parallel. As I wrote in my preivous post, I want to do this both via documentation and via developer tools that provide live feedback. In some cases, I think, the rules will probably depend on type inference or other dynamic analysis techniques that are subtle and hard to explain, but live feedback should be helpful in detecting and resolving those cases.

Nonetheless, I do not think that the formal specification of ParallelJS should include these sorts of details. In my view, this would be similar to having the ECMAScript committee define what patterns in JavaScript will be efficiently JITted and which will not. This is ultimately going to vary depending on the implementation.

In particular, the JavaScript subset that will be acceptable is going to vary substantially depending on what techniques are used to implement the parallelization. On the extreme end, if we had an implementation based on transactional memory, you could imagine that the full JavaScript language might be accepted. If you think that’s science fiction, consider that newer Intel chips will have hardware support for a limited form of transactional memory. On the other extreme, engines that utilize the GPU will only support a very limited subset, one that most likely excludes memory allocation.

I find the precedent of asm.js to be a more promising approach. The formal specification should only state what the the parallel methods are. Preferably, this specification should be loose enough to accommodate as many different parallel execution techniques as possible, but strict enough to prevent wide divergence between engines. I have argued in the past for “equivalent to some sequential execution”, and I still think that’s the right standard, but there’s room for discussion on this point.

Meanwhile, there are can be several independent specifications that provide guidance as to what subset of JavaScript should be supported to parallelize in different ways. Writing such a specification now is probably immature, I think it would be better to have multiple JavaScript engines involved so that the specification is not tailored to SpiderMonkey.

It will be challenging, I think, to come up with a specification that offers the very strong guarantees that “asm.js” can offer (no recompilation, no bailouts, etc). This is because “asm.js” is a very narrow slice of JS intended to be output by compilers, not by humans. It excludes, for example, all normal JavaScript objects. Now, a specification like this might be useful in a parallel context as well; it could serve as the backend for other languages. But I would hope that we have some broader specifications that define code that humans can write.

It is also important to point out that “asm.js” builds upon a lot of precedent. Smart folk working on projects like Emscripten and Mandreel have already done a lot of the leg work to define the idioms that “asm.js” codifies. I hope that as ParallelJS evolves we’ll also evolve a common set of idioms and “ways of doing things” that we can then formalize.

I will start my discussion at the level of the intrinsic ForkJoin() function. As an intrinsic function, ForkJoin() is not an API intended for use by end-users. Rather, it is available only to self-hosted code and is intended to serve as a building block for other APIs (ParallelArray among them).

The ForkJoin function

The idealized view of how ForkJoin works is that you give it a callback fjFunc and it will invoke fjFunc once on each worker thread. Each call to fjFunc should therefore do one slice of the total work.

In reality, though, the workings of ForkJoin are rather more complex. The problem is that we can only execute Ion-compiled code in parallel. Moreover, we can only handle ion-compiled code that avoids the interpreter, since most of the pathways in the interpreter are not thread-safe. This means that we require accurate type information. However, we can only get accurate type information by running the code (and, moreover, we can’t run the code in parallel because the type monitoring infrastructure is not thread-safe).

What we wind up doing therefore is using sequential executions whenever we can’t run in parallel. This might be because there isn’t enough type information, or it might be because the code contains operations that require parts of the interpreter that are not thread-safe.

In the general case, a single call to ForkJoin can move back and forth between parallel and sequential execution many times, gathering more information and potentially recompiling at each step. After a certain number of bailouts, however, we will just give up and execute the remainder of the operations sequentially.

The ForkJoin function is designed such that the caller does not have to care whether the execution was done in parallel or sequential. Either way, presuming that the callback fjFunc is properly written, the same results will have been computed in the end.

The arguments to ForkJoin

ForkJoin expects one or two arguments:

ForkJoin(fjFunc)               // Either like this...
ForkJoin(fjFunc, feedbackFunc) // ...or like this.

Both arguments are functions. fjFunc defines the operation that will execute in parallel and feedbackFunc is used for reporting on whether bailouts occurred and why. feedbackFunc is optional and may be undefined or null. Not passing in feedback will result in slightly faster execution as less data is gathered. The ForkJoin function does not return anything and neither fjFunc nor feedbackFunc are expected to return any values; instead, fjFunc and feedbackFunc are expected to mutate values in place to produce their output.

fjFunc: The parallel operation

The signature of fjFunc is as follows:

fjFunc(sliceId, numSlices, warmup)

Here sliceId and numSlices are basically the thread id and the thread count respectively (though we purposefully distinguish between the slice, a unit of work, and the worker thread—today there is always one slice per worker thread, but someday we may improve the scheduler to support work-stealing or other more intelligent strategies for dividing work and then this would not necessarily be true).

The warmup flag indicates whether the function is being called in warmup mode. As will be explained in the next section, we expect fjFunc to generally track how much work it has done so far. When warmup is true, the function should do “some” of the remaining work, but not too much. When warmup is false, it should attempt to do all the remaining work. Thus, if fjFunc successfully returns when warmup is false, then ForkJoin can assume that all of the work for that slice has been completed.

Warmups and bailouts

On the very first call to ForkJoin, it is very likely that the callback fjFunc has never been executed and therefore no type information is available. In that case, ForkJoin will begin by invoking the callback fjFunc sequentially (i.e., with the normal interpreter) and with the warmup argument set to true. We currently invoke fjFunc once for each slice. As we just said, because warmup is true, each call to fjFunc should do some of the work in its slice but not all (fjFunc is responsible for tracking how much work it has done; I’ll explain that in a second). Once the calls to fjFunc return, and presuming no exceptions are thrown, ForkJoin will attempt compilation for parallel execution.

Presuming compilation succeeds, ForkJoin will attempt parallel execution. This means that we will spin up worker threads and invoke fjFunc in each one. This time, the warmup argument will be set to false, so fjFunc should try and do all the rest of the work that remains in each slice. If all of these invocations are successful, then, the ForkJoin procedure is done.

However, it is possible that one or more of those calls to fjFunc may bailout, meaning that it will attempt some action that is not permitted in parallel mode. There are many possible reasons for bailouts but they generally fall into one of three categories:

• The type information could be incomplete, leading to a failed type guard;
• The script might have attempted some action that is not (yet?) supported in parallel mode even though it seems like it might be theoretically safe, such as access to a JS proxy, built-in C++ function, or DOM object;
• The script might have attempted to mutate shared state.

What we do in response to a bailout is to fallback to another sequential, warmup phase. As part of this fallback, we typically invalidate the parallel version of fjFunc that bailed out, meaning that we’ll recompile it later. Next, just as we did in the initial warmup, we invoke fjFunc using the normal interpterer once for each slice with the warmup argument set to true.

Once this “recovery” warmup phase has completed, we will re-attempt parallel execution. The idea is that we now have more accurate type and profiling information, so we should be able to compile successfully this time.

This process of alternating parallel execution and sequential recovery runs continues until either (1) a parallel run completes without error (in which case we’re done) or (2) we have bailoud out three times (which is a random number, obviously, that we probably want to tune). Once we’ve had three bailouts, we’ll give up and just invoke fjFunc sequentially with warmup set to false.

An example: ParallelArray.map

To make this more concrete, I want to look in more detail about how ParallelArray.map is implemented in the self-hosted code. All the other ParallelArray functions work in a similar fashion so I will just focus on this one.

The semantics of a parallel map are simple: when the user writes pa.map(kernelFunc), a new ParallelArray is returned with the result of invoking kernelFunc on each element in pa. In effect, this is just like Array.map, except that the order of each iteration is undefined.

Our implementation works by dividing the array to be mapped into chunks, which are groups of 32 elements. These chunks are then divided evenly amongst the N worker threads. The implementation relies on shared mutable state to track how many chunks each thread has been able to process thus far. There is a private array called info that stores, for each chunk, a start index, end index, and a current index. The start and end indices simply reflect the range of items assigned to the worker. The current index, which is initially the same as the start index, indicates the next chunk that the worker should attempt to process. This array is shared across all threads and is unsafely mutated using special intrinsics (thus bypassing the normal restrictions against mutating shared state).

The ParallelArray map function is built on ForkJoin. A simplified verison looks something like this:

function map(kernelFunc) {
    // Compute the bounds each slice will have to operate on:
    var length = this.length;
    var numSlices = ForkJoinSlices();
    var info = prepareInfoArray(length, numSlices);

    // Create the result buffer:
    var buffer = NewDenseArray(length);

    // Perform the computation itself, writing into `buffer`:
    ForkJoin(mapSlice);

    // Package up the buffer in a parallel array and return it:
    return NewParallelArray(buffer);

    function mapSlice(sliceId, numSlices, warmup) {
        // ... see below ...
    }
}

Here is the source to the map callback function mapSlice:

function mapSlice(sliceId, numSlices, warmup) {
  var chunkPos = info[SLICE_POS(sliceId)];
  var chunkEnd = info[SLICE_END(sliceId)];

  if (warmup && chunkEnd > chunkPos)
    chunkEnd = chunkPos + 1;

  while (chunkPos < chunkEnd) {
    var indexStart = chunkPos << CHUNK_SHIFT;
    var indexEnd = std_Math_min(indexStart + CHUNK_SIZE, length);

    // Process current chunk:
    for (var i = indexStart; i < indexEnd; i++)
      UnsafeSetElement(buffer, i, kernelFunc(self.get(i), i, self));

    UnsafeSetElement(info, SLICE_POS(sliceId), ++chunkPos);
  }
}

This same code is used both for parallel execution and the sequential fallback. Each time mapSlice is invoked, it will use the sliceId it is given to lookup the current chunk (info[SLICE_POS(sliceId)]; SLICE_POS is a macro that computes the correct index). It will then process that chunk and update the shared array with the index of the next chunk (results are unsafely written into the result array buffer—note that this result is not yet exposed to non-self-hosted code). If we are in warmup mode, it will stop and return once it has processed a single chunk. Otherwise, it keeps going and processes the remaining chunks, updating the shared array at each point.

The purpose of updating the shared array is to record our progress in the case of a bailout. If a bailout occurs, it means that after processing some portion of the current chunk, the function will simply exit. As a result, the “current chunk” will not be incremented, and the next time that mapSlice is invoked with that same sliceId, it will pick up and start re-processing the same chunk. This does mean that if a bailout occurs we will process some portion of the chunk twice, once in parallel mode and then again in sequential mode after the bailout. This is unobservable to the end user, though, because parallel executions are guaranteed to be pure and thus the user could not have modified shared state or made any observable changes.

The various worker threads will unsafely mutate this shared array to track their progress. Unsafe mutations make use of the intrinsic UnsafeSetElement(array, index, value), which is more-or-less equivalent to array[index] = value except that (1) it assumes the index is in bounds; (2) it assumes that array is a dense array or a typed array; (3) it does not do any data race detection. In other words, you have to know what you’re doing. The same intrinsic is also used to store the intermediate results.

feedback: Reporting on bailouts

The precise API for feedback is to some extent still being hammered out. Right now this function is used primarily for unit testing so that we can be sure that parallelization works when we think it should. The eventual goal is to display information in the profiler or other dev tools that indicate what happened. This post is already long enough, so I’ll defer a discussion of the precise process by which we gather bailout information. Suffice to say that in the event of a bailout, each thread records the cause of the bailout (e.g., “write to illegal object” or “type guard failure”) along with a stack trace showing the script and its position.

Note

This note describes the code as it is found on our branch. This differs slightly from what is currently landed on trunk. In particular, there have been some recent refactorings that renamed the ParallelDo function to ForkJoin. This is because we recently refactored the source so that ParallelDo.cpp and ForkJoin.cpp, originally two distinct but tightly interwoven layers, are now fused into one abstraction.

What is available now

Once Nightly builds are available, users will be able to run what is essentially a “first draft” of Parallel JS. The code that will be landing first is not really ready for general use yet. It supports a limited set of JavaScript and there is no good feedback mechanism to tell you whether you got parallel execution and, if not, why not. Moreover, it is not heavily optimized, and the performance can be uneven. Sometimes we see linear speedups and zero overhead, but in other cases the overhead can be substantial, meaning that it takes several cores to gain from parallelism. Nonetheless, it is pretty exciting to see multithreaded execution landing in a JavaScript engine. As far as I know, this is the first time that something like this has been available (WebWorkers, with their Share Nothing, Copy Everything architecture, do not count).

UPDATE: It has been pointed out to me that WebWorkers were recently extended to support moving typed arrays from place to place, though there is still no way for multiple workers to share a read-only view on a typed array.

We have already written several patches that we hope to land in the near future. These patches expand the set of JavaScript functions that can run in parallel. They also help to reduce compilation overheads and generally improve performance, as well as making the code less vulnerable to disruptions from garbage collection.

Where we are going in the medium term

Looking at the medium term, the main focus is on ensuring that there is a large, usable subset of JavaScript that can be reliably parallelized. Moreover, there should be a good feedback mechanism to tell you when you are not getting parallel execution and why not.

I think that we can achieve a state where if you write a pure function (meaning one that does not mutate shared state) in “plain vanilla” JS, it will basically work. “Plain vanilla” is of course a highly technical industry term meaning “no weird stuff”. Intuitively, I mean code that uses only JS objects (i.e., no DOM objects) and avoids some of the more advanced JS features like proxies. A more rigorous definition of what I mean by “plain vanilla” is a big piece of this medium-term work.

Supporting “plain vanilla” JS is mostly a matter of going through individual code paths in SpiderMonkey and refactoring them so that they can cleanly support parallel execution. It is difficult to do this and keep the code relatively DRY. We are currently exploring the best techniques for this. I think there is no magic bullet here, though; the code was written to assume single-threaded execution and is riddled with various bits of cleverness that unfortunately make it hard to parallelize.

The other part of the story is providing feedback that informs users when parallelization has failed and why. Once we support a large enough portion of JS, I think good feedback is probably even more important than expanding the subset we support.

Finally, there will always be ongoing work on lowering overhead and improving performance. Some of that can come from more advanced optimization techniques (like vectorized compilation or GPU support), but to some extent this also arises just from looking over the relevant code paths and tuning them repeatedly.

Where we are going in the long term

I am basically obsessed with the idea of making parallelism easy and omnipresent wherever I can. The code we are landing now is a very significant step in that direction, though there is a long road ahead.

I want to see a day where there are a variety of parallel APIs for a variety of situations. I want to see a day where you can write arbitrary JS and know that it will parallelize and run efficiently across all browsers.

I expect the final APIs with which we expose parallel execution will evolve over time. There will be debate, some of which is already visible on this blog. For example, should we just offer a ParallelArray type or instead attach methods to Array? How should we specify the semantics of parallel execution, precisely? I expect that once we have good prototypes available, this dialog will grow, paricularly as the JS community and ECMAScript committee gets involved (neither group is exactly known for a shortage of opinions, and rightly so).

I also want to add better support for task parallelism via something like the PJs API I have talked about before. Part of the goal with the current work has been to lay the foundations to make it possible to iterate on APIs and introduce new ones.

Note: To give credit where credit is due, I should note that a lot of the ideas in this post originate with other members of the Parallel JS team (Shu-yu Guo, Dave Herman, Felix Klock). But I don’t want to speak for them, since we seem to each have our own opinions on the best arrangement, so I’m writing the post from the first person singular (“I”) and not a team perspective (“we”). This does not imply “ownership” of the ideas within.

The basic idea

The basic idea is to add “unordered” or parallel variants of the standard higher-order methods to JavaScript arrays as well as to typed arrays (and binary data arrays when those become available). For example, in addition to map() and reduce(), we’d offer unorderedMap() and unorderedReduce() (in the case of typed arrays, I think we’d have to add map() as well).

The semantics of the unordered variants are the same as their ordered cousins, except that the ordering in which they perform their iterations is not defined. However, if you used the unordered variants, we will attempt parallel execution where possible.

Why call the methods “unordered”?

I chose the (admittedly somewhat clunky) prefix unordered because I want to emphasize the fundamental contract our parallel execution engine offers, which is that parallel execution is equivalent to some sequential ordering, but it doesn’t say which one. This is a somewhat controversial design, but I still feel it’s the right one. In any case, it’s basically orthgonal to this post.

Note that there is no reason we can’t someday try parallel execution for the ordered map() as well. However, we’d have to be very careful to avoid introducing overhead in the case that parallelization fails or would change the semantics of the program. The use of the unordered variant effectively serves as a hint that parallelization is likely to pay off.

What about immutability?

Some readers will remember that ParallelArray objects are immutable while normal JS arrays are not. This is true but it’s not a big obstacle. During any parallel operation, mutations to pre-existing objects are forbidden and must be detected; in the case of a call like array.unorderedMap(func), the array array that is being mapped is itself a pre-existing object and thus would be at least temporarily immutable.

There are of course some good reasons to have immutable data, particularly if we wind up doing GPU operations, in which case memory will have to be transferred back and forth, and we may have to worry about invalidation. If this ever becomes an issue, we can accommodate these more advanced use cases either by the existing freezing interfaces that JS provides or through the multi-dimensional API described below.

What are the benefits of this API?

The biggest benefit of this approach, I think, is that it’s about the simplest way to offer parallelism. You can work with the JS array types we all know and love (or hate, as you prefer). Moreover, integration with existing codebases becomes easier. If you have some loops that are performing pure transformations, such as filtering out records on some criteria, you can change them to execute in parallel just by changing the name of the method you use. On other or older browsers, it’s trivial to polyfill unorderedMap as equivalent to map.

What does this mean for ParallelArray?

Right now, the ParallelArray API serves two masters. It tries to be a very lightweight one-dimensional array but it also tries to be a fairly powerful multi-dimensional matrix. If we offer parallel transformations on normal arrays, that frees up ParallelArray so that it can be targeted at more advanced use cases. In particular, it can be (1) always multi-dimensional and (2) type-annotated to permit efficient storage when you have a matrix of scalar values like bytes or ints. I am right now working on another post regarding some ideas relating to how we can handle the multi-dimensional case; it was originally part of this post but this post was rapidly becoming too long.

Extern Function Types

One thing we need is a type for a simple function pointer. Rust’s function types to date have always been closure types, meaning that they referred to the combination of a function pointer and some environment. So we have added an “extern fn” type, which is written as follows:

extern "ABI" fn(T) -> U

Here the "ABI" string must be some ABI that is supported by the Rust compiler. The most common values will be either C or Rust, I imagine, but stdcall (or pascal) may be used occasionally as well, and who knows what we’ll support in the future.

I imagine that the default for “ABI” should be “C”, as it will be the most common thing people really want to use. Calls to any extern function with non-Rust ABI is an unsafe action.

Extern Blocks and Function Declarations

We are moving towards a model where function declarations are placed within extern blocks. This looks something like:

extern "C" {
    fn foo();
    fn bar();
}

In this case, the type of foo and bar would be extern "C" fn().

The reason that we declare extern functions in extern blocks, as opposed to individually, is that on some platforms it is necessary to load blocks of functions that are defined by a common library together.

“crust” functions

In addition to being able to call C functions from within Rust, it is useful to be able to call Rust functions from within C. To this end the compiler will permit Rust fns to be declared with a specific ABI like so:

extern "C" fn crust(t: T) -> U {
}

If you declare a function as having a non-Rust ABI, then this implies a few things:

• A reference to crust() will have type extern "C" fn(T) -> U.
• We cannot catch and process failure for you, since the propagation of failure results is ABI specific. Thus is the Rust code within an external function fails, it will cause the process to abort. We may later add some way to catch failure so that you can propagate it yourself (perhaps by returning false, etc).

Stack Switching

Now we come to the interesting (and tricky) part. Internally, Rust makes use of a split stack approach where stack segments are allocated dynamically as the stack grows. This allows us to have a very large number of threads without exhausting our address space (particularly on 32-bit systems). This also allows your programs to recurse as long as there is memory available, which is sometimes useful. It is not, however, what C expects. C functions just expect to have a big chunk of stack available. Hopefully infinite.

Therefore, whenever we recurse into C code, we must make sure that a lot of stack is available. The way we do this today is somewhat magical: functions declared as extern are not in fact the raw C function, but rather a wrapper around the C function that will switch over from the Rust stack (which may be small) to a very big stack. This was more-or-less an ok solution back before we had the idea of getting a raw pointer to a C function and so forth but it’s not very appealing now. Also it can be a performance bottleneck.

The new proposal is to say that when you call an extern "C" fn, nothing magical happens. The stack stays just as it was. To perform the stack switching, we offer a function in the runtime (perhaps a number of functions) called prepare_extern_call(), which can be used like so:

let my_c_function: extern "C" fn() = ...;
do prepare_extern_call {
    my_c_function()
}

Of course, it would be easy to forget to use this function, which would be a recipe for stackfaults. Therefore, we will also offer a lint-mode check that defaults to error. This check will trigger if we see a call to a function of non-Rust ABI that is not lexically enclosing within a call to prepare_extern_call.

There will be variants of prepare_extern_call that allow you to specify the amount of stack size to guarantee more precisely if you prefer, along with other options as those arise.

Auto-generating wrappers

It is our expectation that most people will not directly call C functions. Instead, you will wrap them in a Rust-friendly wrapper that performs some sanity checking, converts from Rust types, etc. This wrapper will also perform the stack switching shown above.

In some cases, though, writing such wrappers can be tedious, so we can supply some annotations in the compiler that will autogenerate these wrappers. This is basically a macro. I am envisioning something like this:

#[auto_wrap] // autogenerate wrappers for enclosing functions
extern "C" {
    #[no_wrap] // ...not this one, I'll do it by hand
    fn my_func1(x: *char) -> bool;

    fn my_func2();
}

fn my_func1(x: ~str) -> bool {
    do x.as_c_string |p| {
        do prepare_extern_call {
        }
    }
}

which would then expand into:

extern "C" {
    fn my_func1(x: *char) -> bool;
    fn my_func2();
}

fn my_func2() -> bool {
   do prepare_extern_call {
       my_func2()
   }
}

One issue that is obvious here is the name collisions. I’m not sure how to resolve that. It seems like the older way of native functions within their own module (extern "C" mod foo) would solve it. Well, we’ll do something. And the precise details of this auto-generation remain to be resolved. But you get the idea.

Dangers today

The root of our problems lies in the fact that if you have a struct type S that has a destructor, it is legal to place an instance of S into a managed box (@S). This is problematic because it implies that the destructor will run when the managed box is collected, which can occur at any arbitrary time (in fact, if the garbage collector were to run on a different thread, it could even occur in parallel with the owning thread!). I will use the term finalizer to mean a destructor associated with an object that is owned by a managed box. In other words, a destructor that can run asynchronously with respect to the main program.

Note: Many of the thoughts in this post were inspired by Hans Boehm. For those seeking a deeper undestanding, I recommend his paper “Destructors, Finalizers, and Synchronization”.

Problem number one: finalizers and borrowed pointers

In our current system, there is nothing to prevent a borrowed pointer from being stored in a managed box. Although it is sometimes surprising to people that it is legal, this scenario is generally harmless. Although the managed box may outlive the data that the borrowed pointer references, the type system will guarantee that the managed box will never be dereferenced once the loan expires. In other words, you can put a pointer into your stack frame into a managed box, but you could never return that managed box to your caller or store it into any data structure that outlives your stack frame. So we know for certain that if we were to run the garbage collector, that box would be collected. Finalizers change this equation. A finalizer provides a backdoor that would allow borrowed pointers in managed boxes to be dereferenced. See issue 3167 for examples of dangerous programs and more details.

The only way I can see to address this unsoundness is to create a new intrinsic trait that indicates when data can safely be placed into a managed box. I have some thoughts on this at the end.

Problem number two: finalizers and managed data

There is another dangerous situation that can arise which has nothing to do with borrowed pointers. Imagine we have a cycle of managed data and two objects on that cycle have a finalizer. Which finalizer do you run first? Normally, you want to finalize an object X before you finalize any object Y that X references, but because there is a cycle that is impossible to guarantee. Different systems have solved this problem in different ways, none of which are wholly satisfactory.

Problem number three: finalizers and mutable state

Another more subtle problem which can occur with finalizers is that the finalizer may have access to mutable state which is not yet dead. Imagine, for example, a struct whose job is to increment and decrement a counter automatically:

struct SomeDataStructure { value: uint, ... }

struct Counter { s: @mut SomeDataStructure }
impl Counter: Drop {
    fn new(s: @mut SomeDataStructure) -> Counter {
        s.value += 1;
        Counter { s: s }
    }
    fn drop(self) { self.s.value -= 1; }
}

As long as this counter is stored on the stack frame, everything should be fine. But if you were to place this counter into a managed box, suddenly you have a ticking time bomb: now the field s.value will be decremented at some random time, whenever the garbage collector elects to collect this managed box. Even if the garbage collector does not run in parallel with the mutator thread, this can essentially cause s.value to be decremented in between virtually any statement, leading to race conditions that are very similar to those problems you face with threads and mutable state. Note that due to compiler optimizations and so forth it is entirely possible for value to be decremented earlier than you might expect as well as later.

Of all the problems, I am perhaps most worried about this one, because it is relatively easy to overlook. It’s not a soundness issue per se but it can lead to very surprising bugs, particularly in light of aggressive compiler optimization. Hans Boehm goes so far as to say that finalizes require a multithreaded, shared memory context to make any sense, precisely because of Problem #3. Basically, the asynchrony inherent in finalizers is more natural in a parallel language and you have tools like locks to defend against it. If finalizers run in the mutator thread, locks lead to deadlocks and not having locks leads to bugs.

You might think that moving data into a managed box can only cause the destructor to be delayed from when it would otherwise run, but this is not the case. In fact the destructor can also run much earlier than you might expect. Consider this Java program from Boehm’s paper:

class X {
    Y mine;
    public foo() { Mine m = mine; ...; m.bar(); }
    public void finalize() { mine.baz(); }
}

Here, in the foo() method, the this pointer may actually be dead right after the first statement, and so this can be collected before m.bar() is called. Boehm’s point with this example is that, in Java, this could result in m.bar() and mine.baz() executing in parallel, but in general the behavior is very surprising. I recall that similar problems were prevalent with Apple’s failed attempt at an Objective-C garbage collector.

Restricting to owned data

All of these problems are solved by limiting destructors to types which contain only owned data. Borrowed pointers and managed pointers are disallowed, so problems one and two cannot arise. Problem three cannot arise because there is no way for the destructor to directly access shared, mutable state.

Limiting to owned data still permits many interesting use cases for destructors. You can embed a file descriptor and guarantee it gets closed. You can ensure that random C resources, such as database descriptors or blocks of memory obtained from malloc(), are cleaned up, since these are typically described by unsafe pointers anyhow. You can also embed a channel and use it to send messages from the destructor.

There is one very useful scenario that is ruled out, however, which is basically the “auto counter” (or any “auto adjustment”) type from problem number three. That is, it is often very useful to have some adjustment that will automatically occur when a stack frame exits, and destructors are one common way to achieve that. Of course this is dangerous if abused, as we have seen, but what about the good guys, who don’t put an auto-object into managed data?

The good news is that even with the limitation I propose there are still two valid ways to achieve the auto-pattern, depending on your precise needs. First, if you don’t care whether the auto code executes on failure—and you probably don’t, remember that Rust failures are unrecoverable—you can just use a function with a closure argument:

fn auto_adjust<R>(s: @mut SomeDataStructure, f: &fn() -> R) -> R{
    s.value += 1;
    let v = f();
    s.value -= 1;
    return v;
}

Now in your code you can write:

do auto_adjust(s) { ... }

But what if you really do care about failure and you need access to the current stack frame when unwinding? We should be able to provide a function in the standard library to handle this case. That function would look something like:

do defer(|| {
    /* This code will execute once the block below exits, even on failure */
}) {
    /* This code executes immediately */
}

Naturally we can play around with the precise signature of this function a bit, but you get the idea: you supply two closures to a library function, it executes them as appropriate. Internally, the function would use unsafe pointers and a destructor, but it would never expose the object that carries the destructor to the outside, and thus could ensure that this object is never placed into a managed box.

Caveat: Not quite future proof

In some sense, I am advocating the conservative approach: we begin with a narrow set of types that can have a destructor, and we can then expand later if that proves to be insufficient. However, there is a catch. If we ever wanted to permit borrowed pointers to be referenced by destructors, the only way that this can be made sound is to limit the set of types that can be placed into a managed box. Since, at the moment, any type can be placed into a managed box, this is a backwards incompatible change. To see what I mean, consider a function like box():

fn box<A>(a: A) -> @A { @a }

This function is legal today, but it would become illegal. This is because there is no guarantee that A can be placed into a managed box. So you’d need to write something like:

fn box<A:Manageable>(a: A) -> @A { @a }

where Manageable is the hypothetical intrinsic trait that characterizes types that can safely be placed into managed boxes. Of course we could change the defaults, so that <A> no longer means “any type at all” but rather “the usual set of types you want to do the usual set of operations” (in which case, perhaps A: would mean “any type at all”, I don’t know). But that too is backwards incompatible.

Current for protocol

The current “for protocol” is best explained by giving an example of how to implement it for slices:

fn each<E>(v: &[E], f: &fn(&E) -> bool) {
    let mut i = 0;
    let n = v.len();
    while i < n {
        if !f(&v[i]) {
            return;
        }
        i += 1
    }
}

As you can see, the idea is that the last parameter to the each() method is a function of type &fn(&E) -> bool, which means that it is given a pointer to an element in the collection and it returns true or false. The return value indicates whether we should continue iterating.

A little known fact is that the for statement returns whatever the each() method returns. This means that each() methods typically have unit return type so that the Rust compiler doesn’t require a semicolon, which would be used to disregard the result of the for expression.

Problems

The biggest problem with this protocol is that it is not easily composable. In particular, imagine that I have a simple tree like this:

struct Tree<E> {
    elem: E,
    children: ~[Tree<E>]
}

Now let’s try to implement the pre-order traversal method for such a tree. You might think you could do it like this:

fn each<E>(t: &Tree<E>, f: &fn(&E) -> bool) {
    if !f(&t.elem) {
        return;
    }

    for t.children.each |child| { each(child, f); }
}

While this will compile, it will not work as expected. For example, this program:

fn main() {
    let t = Tree {
        elem: 0,
        children: ~[
            Tree { elem: 1, children: ~[
                Tree { elem: 2, children: ~[] }
            ] },
            Tree { elem: 3, children: ~[] }
        ]
    };

    for each(&t) |e| {
        io::println(fmt!("%d", *e));
        if *e == 1 { break; }
    }
}

should print “0” and “1”, but it prints “0”, “1”, and “3”. The reason is that while each() does indeed return early when the iteration function returns false, it doesn’t abort the entire iteration, only the current subtree.

One way to fix this is to wrap the each() function with an inner each function that returns a bool to indicate whether execution should stop:

fn each1<E>(t: &Tree<E>, f: &fn(&E) -> bool) {
    each_inner(t, f);

    fn each_inner<E>(t: &Tree<E>, f: &fn(&E) -> bool) -> bool {
        if !f(&t.elem) {
            return false;
        }

        for t.children.each |child| {
            if !each_inner(child, f) {
                return false;
            }
        }

        return true;
    }
}

Making each() composable

I think that we should change the standard each signature to:

fn each<E>(c: &Coll<E>, f: &fn(&E) -> bool) -> bool

Here the return value of each is always a boolean, and it will be false if the last call to f() returned false, and true otherwise. This makes it easier to write composed each() methods. We would also adjust for statements so that they always return unit and do not return the result of each().

Under this definition, we could write the tree iterator as follows:

fn each2<E>(t: &Tree<E>, f: &fn(&E) -> bool) -> bool {
    f(&t.elem) && t.children.each(|c| each2(c, f))
}

This is clearly an improvement over each1()!

My newest proposal is that we use <> to designate lifetime parameters on types and we lean on semantic analysis (the resolve pass, more precisely) to handle the ambiguity between a lifetime name and a type name. Before I always wanted to have the distinction between lifetimes and types be made in the parser itself, but I think this is untenable. This proposal has the advantage that the most common cases are still written as they are today.

Here is the example from the previous post in my proposed notation:

struct StringReader<&self> {
                 // ^~~~~ Lifetime parameter designated with &
    value: &self/str,
        // ^~~~~~~~~ Same as today.
    count: uint
}

impl StringReader {
    fn new(value: &self/str) -> StringReader<&self> {
                             //              ^~~~~
                             // Interpreted as a lifetime reference due to
                             // the declaration of StringReader, which states
                             // that first parameter is a lifetime.
        StringReader { value: value, count: 0 }
    }
}

fn value(s: &v/StringReader<&v>) -> &v/str {
         // ^~~~~~~~~~~~~~~~~~~     ^~~~~~
         // As today, lifetime names that appear in a function declaration
         // do not have to be declared anywhere and are implicitly scoped
         // to the containing function declaration.
    return s.value;
}

fn remaining(s: &StringReader<&> -> uint {
             // ^~~~~~~~~~~~~~~~
             // A bare & in a fn decl means "use a fresh name",
             // so this is equivalent to &x/StringReader<&y>.
             // This may be the right thing, see Option 2 below.
    return s.value.len() - s.count;
}

What follows are miscellaneous notes and thoughts. There are a few options that could be tweaked, which I have noted.

Considerations

The only way I have found to distinguish lifetime names purely in the parser that is also visually appealing is to use braces to designate lifetimes (options 7 and 8 in my previous post). As a reminder, the impl of StringReader would look like:

impl StringReader {
    fn new(value: &{self} str) -> StringReader{self} {
        StringReader { value: value, count: 0 }
    }
}

The major problem here is that, as bstrie pointed out on IRC, it’s ambiguous: the {self} which appears in the return type could be interpreted as the function body. His proposed fix was to use whitespace sensitivity, so that StringReader{self} and StringReader {self} are parsed differently, but whitespace sensitivity is something we have always tried to avoid.

I personally find it appealing to use <> both for lifetime and type parameters, because I think it gives the right intution. A lifetime-parameterized declaration is just like a type-parameted declaration with regard to how it works in the type system.

OPTION 1: I opted to include & in the lifetime parameters to a type for consistency (this way, a lifetime name is always preceded by &). However, they are not strictly necessary and they are visually heavy. We could remove them, which would mean you have &v/StringReader<v> and StringReader<self> and not &v/StringReader<&v> and StringReader<&self>. However, the default would still have to be written &, so you’d still have StringReader<&>.

The default lifetime &

In this proposal, the “default lifetime” & would only be usable inside a function declaration or function body. In a function declaration, it means “use a fresh lifetime. In the function body it means “use inference”.

OPTION 2: It would be possible to make & a little smarter, as it is today. Today it means “use a fresh name unless & appears on a nested type, then use the lifetime you are nested within”. If we took that interpretation, then &StringReader<&> would be equivalent to &x/StringReader<&x> and not &x/StringReader<&y>. This is more likely to be what the user wanted, though I don’t think it makes much difference in practice. I’d probably just want to experiment a bit here: start with the simpler version, as I proposed here, and then see how many type errors we get

OPTION 3: We could also allow users to leave off the <> if the only parameter is a lifetime parameter, in which case it would be equivalent to <&>. This means that you could write &StringReader instead of &StringReader<&>.

OPTION 4: The one place that I opted to eschew explicit declarations is on functions. If we wanted, we could always require that all named lifetimes be declared, which would mean that the function value() above would be written:

fn value<&v>(s: &v/StringReader<v>) -> &v/str {
    return s.value;
}

I can’t decide about this option. It strikes me as a reasonably simple story, which appeals to me, but it’s also fairly heavyweight.

How complex can it get?

UPDATE: Per bstrie’s request, here is an example of a type that uses both lifetime and type parameters with trait bounds:

struct Foo<&self, T: Reader+Eq> {
    value: &self/T,
    count: uint
}

fn operate<R: Reader+Eq>(f: Foo<&, R>)
{
    ...
}

Why determinism?

All things being equal, it’s clear that deterministic execution semantics are preferable. They’re easier to debug and they avoid the question of browser incompatibilities.

One interesting observation (most recently pointed out by Roc in this comment) is that while the intention of a nondeterministic ordering is to free up the implementor, what sometimes happens is that all people wind up relying on the behavior of one implementation, and then the others follow suit.

That said, there seem to be numerous examples of nondeterministic portions of JavaScript: the iteration order for properties, for example, or the order of callbacks to the comparator in Array.sort. But then there are plenty of examples of implementations being constrained by arbitrary behavior inherited from legacy interpreters. In any case, I am not an expert in these kind of nitty gritty cross-browser compatibility details.

If we did opt for nondeterministic semantics, it might be plausible to use a cheap PRNG like Xorshift in the sequential fallback so as to make it more likely that unwanted dependencies on execution ordering would be seen during testing (though, of course, this adds overhead too!).

What about performance?

It is true that nondeterministic semantics give maximum efficiency, but the magnitude of these performance gains is not entirely clear. In my previous post, for example, I stressed the behavior around bailouts. It is true that guaranteeing deterministic semantics will result in wasted work and in general make bailouts less efficient. However, it is also true that bailouts are the exceptional case: if things are working properly, they should only occur at the beginning of execution. Once sufficient type information has been gathered, parallel execution without bailouts should be the norm—unless of course it turns out the code is not parallelizable, either because it is impure or because it uses some unsupported language features, in which case we will simply use the sequential fallback from the start and not even attempt parallelism.

So, at least in the case of functions like map(), efficiency in the steady state should not be negatively impacted by deterministic semantics, presuming that the kernel function is pure.

But what about reduce, scan, and scatter?

As I wrote in the previous post, the current semantics of ParallelArray are inconsistent. They give deterministic results for map() but not for reduce(), scan(), or scatter(). The core problem here is that the standard sequential ordering for reduce() (i.e., left-to-right) is inherently sequential—and the most efficient ordering will depend on the precise implementation strategy (how many worker threads are involved, etc). But, at the cost of some efficiency, we can choose a deterministic ordering that still permits parallel execution.

For reduce, a good ordering might be to evaluate in a tree-like fashion. So we would first reduce indices 0 and 1, then 2 and 3, 4 and 5, and so on, resulting in an array with length N/2. We can then repeat the reduction until the result has length 1. If at any step we get an array with an odd length, we can reduce the final element in with the final pair. A similar ordering can be used for scan, though the need to preserve intermediate results creates complications.

The scatter operation, at least when a conflict function is provided, is much more difficult to parallelize. I think that the only way it is possible is to make each thread walk the entire list of targets but only process writes to a specific subset of the array. If no conflict function is provided, or if the conflict function is one that is known to be associative and commutative (such as integer addition—though not floating point, sadly), then parallelization of scatter is also relatively straightforward.

So what’s the right thing to do?

At this point, the answer is probably “measure” or perhaps “wait and see”. I am somewhat concerned though about basing too many decisions on the performance of our current implementation, both because it has not been heavily optimized and because it is only one model of execution (parallel worker threads). But we’ve got to go on something.

If the API were designed for me personally to use, I would want it to have nondeterministic semantics. This gives maximum flexibility to the implementation without opening the door to data races. However, I am certainly appreciative of the concerns regarding debugability—performance is not everything!

1. Deterministic results where possible: The function behaves “as if” it executed sequentially, executing the kernel from 0 to n, just like Array.map.
2. Error: An exception is thrown.
3. Non-determinstic results: The function behaves “as if” it executed sequentially, but the items were mapped in an unspecified order.

The branch currently implements option 3: I believe it is the most consistent and will yield the best performance. However, reasonable people can differ on this point, so I want to make my case.

Why not ensure deterministic results?

At first glance, at least, it seems like having deterministic results would be the most convenient option. After all, ParallelArray.map() could then be used as a drop-in equivalent to Array.map(). However, there are two reasons that I am concerned about this option:

1. Deterministic results are not possible for all operations.
2. Deterministic ordering can hamper efficient parallel execution.

Deterministic results are not possible for all operations.

If you examine the specification for ParallelArray.reduce(), you will see that it permits reduce() to reduce the items in the array in any order:

Reduce is free to group calls to the elemental function and reorder the calls. For an elemental function that is associative and commutative the final result will be the same as reducing from left to right…reduce is only required to return a result consistent with some call ordering and is not required to chose the same call ordering on subsequent calls.

This means that the result of reduce() is only deterministic if the kernel function is associative and commutative. Without this requirement, implementations would be severely limited in how they could perform parallel reduction, and in some cases parallel execution would be too expensive to be worthwhile (interestingly, the sequential fallback would be more expensive too, because the best sequential ordering for reduction is not parallelizable at all).

For this reason, I am wary of telling users that ParallelArray methods have equivalent semantics to Array methods. This is only partially true and it can never be fully true. In general, I think users will want the best parallel performance they can get, and they will accept whatever restrictions are required to get it.

Deterministic ordering can hamper efficient parallel execution.

We just saw that, in the case of reduce(), any deterministic ordering either prevents efficient parallel execution or it prevents efficient sequential execution. To a lesser degree, the same is true of map(). Unlike reduce(), though, the problems with map() are somewhat subtle.

You may recall that when we perform parallel execution, there is always the possibility of bailout. A bailout can occur because we detect that a write would cause a visible side-effect, but it can also occur for arbitrary, internal reasons. Bailouts often occur because there is some portion of the code that has not yet been executed in the warmup runs, and hence the type information that we gathered was inaccurate.

The question is, when a bailout occurs, what do we do with the results that were successfully computed by the previous parallel iterations? It would be very nice if we were able to make use of those results. If we do guarantee deterministic results, however, this is somewhat tricky.

Imagine for a moment there are just two threads processing a 1000-element array. In our current system, each worker will be responsible for mapping half of the array. So imagine that the first worker has processed indices 0 to 22 when it encounters a bailout due to insufficient type information. To gather type information, we need to execute iteration 23 sequentially in the interpreter. In fact, while we’re in the interpreter, it’s probably best if we do a chunk of iterations—say, 23 to 32 or so, just to gather up more data. Meanwhile, let’s say that the second worker has processed entries 500 to 600 before it notices that the first worker bailed out and follows suit.

Unfortunately, if we are guaranteeting deterministic results, it is very difficult for us to make use of the results for entries 500 to 600 that were already computed. It’s possible, after all, that when we re-run iterations 23 to 32 in the interpreter, they will modify shared state, and that could affect the computations that already occurred. In effect, all parallel iterations are inherently speculative.

As annoying as it is to have to throw away indices 500 to 600, it is equally annoying if the bailout should occur in the second worker (say, while processing index 601). In that case, we have a problem: we’d like to run index 601 in the interpreter, but it’s possible that this too will cause side-effects. That means that we must ensure that all indices prior to 601 are fully processed (which, at the time of bailout, they are not). Given enough bookkeeping, we can manage this too: we could perhaps re-spawn parallel workers to process from 23 to 600, for example, and then re-spawn to process from 600 to 1000.

What if we don’t guarantee deterministic ordering?

If, however, we choose not to guarantee deterministic results, these problems become much simpler. When a bailout occurs, we can simply have each worker execute sequentially from wherever it left off. So to continue with our example, worker 0 would run from indices 23 to 30 or so, and perhaps worker 1 would run from 601 to 632. This way they can both gather a bit more data. Then we resume parallel execution.

This is in fact more-or-less exactly what our current code does. Each worker tracks its current position. Whenever the kernel function is invoked, it processes the next data it has to proccess, and it updates its current position as it goes. If a bailout occurs, then the current position is not updated, so when the kernel function is invoked next (which will be from the interpreter), it will pick up where it left off and process for a while. I can dive into the intimate details of this in a separate post.

What about reporting an error?

Prohibiting impure operations altogether does enable the same optimizations as nondeterministic ordering, but (as I argued in a previous post) it does so by imposing a significant and unnecessary implementation and performance burden on the sequential fallback code. So I am not a fan of this approach.

The take-away

My feeling is this. Deterministic ordering sounds like it makes things easier for the end-user, but the story is actually more complex, as only some operations are deterministic. Moreover it imposes performance burdens on the implementation, so even the “good guys” who stick to pure operations will pay the price, at least until optimizations become more highly tuned to cover all kinds of corner cases.

In general, I think people will turn to ParallelArray when they have pure operations to perform: for pure operations, deterministic ordering simply imposes a performance burden. If you have an impure operation for which ordering is significant, it seems to me you would be better off just using normal arrays or coding the operation in a sequential fashion.

Clearly this is an area where reasonable people can differ, though, and I would not be surprised if we ultimately decide to guarantee determinism where possible (i.e., map() and filter()).