Goal: richer, place-based permissions

Dada aims to exceed Rust’s capabilities by using place-based permissions. Dada lets you write functions and types that capture both a value and things borrowed from that value.

As a fun example, imagine you are writing some Rust code to process a comma-separated list, just looking for entries of length 5 or more:

let list: String = format!("...something big, with commas...");
let items: Vec<&str> = list
    .split(",")
    .map(|s| s.trim()) // strip whitespace
    .filter(|s| s.len() > 5)
    .collect();

One of the cool things about Rust is how this code looks a lot like some high-level language like Python or JavaScript, but in those languages the split call is going to be doing a lot of work, since it will have to allocate tons of small strings, copying out the data. But in Rust the &str values are just pointers into the original string and so split is very cheap. I love this.

On the other hand, suppose you want to package up some of those values, along with the backing string, and send them to another thread to be processed. You might think you can just make a struct like so…

struct Message {
    list: String,
    items: Vec<&str>,
    //         ----
    // goal is to hold a reference
    // to strings from list
}

…and then create the list and items and store them into it:

let list: String = format!("...something big, with commas...");
let items: Vec<&str> = /* as before */;
let message = Message { list, items };
//                      ----
//                        |
// This *moves* `list` into the struct.
// That in turn invalidates `items`, which 
// is borrowed from `list`, so there is no
// way to construct `Message`.

But as experienced Rustaceans know, this will not work. When you have borrowed data like an &str, that data cannot be moved. If you want to handle a case like this, you need to convert from &str into sending indices, owned strings, or some other solution. Argh!

Dada’s permissions use places, not lifetimes

Dada does things a bit differently. The first thing is that, when you create a reference, the resulting type names the place that the data was borrowed from, not the lifetime of the reference. So the type annotation for items would say ref[list] String¹ (at least, if you wanted to write out the full details rather than leaving it to the type inferencer):

let list: given String = "...something big, with commas..."
let items: given Vec[ref[list] String] = list
    .split(",")
    .map(_.trim()) // strip whitespace
    .filter(_.len() > 5)
    //      ------- I *think* this is the syntax I want for closures?
    //              I forget what I had in mind, it's not implemented.
    .collect()

I’ve blogged before about how I would like to redefine lifetimes in Rust to be places as I feel that a type like ref[list] String is much easier to teach and explain: instead of having to explain that a lifetime references some part of the code, or what have you, you can say that “this is a String that references the variable list”.

But what’s also cool is that named places open the door to more flexible borrows. In Dada, if you wanted to package up the list and the items, you could build a Message type like so:

class Message(
    list: String
    items: Vec[ref[self.list] String]
    //             ---------
    //   Borrowed from another field!
)

// As before:
let list: String = "...something big, with commas..."
let items: Vec[ref[list] String] = list
    .split(",")
    .map(_.strip()) // strip whitespace
    .filter(_.len() > 5)
    .collect()

// Create the message, this is the fun part!
let message = Message(list.give, items.give)

Note that last line – Message(list.give, items.give). We can create a new class and move list into it along with items, which borrows from list. Neat, right?

OK, so let’s back up and talk about how this all works.

References in Dada are the default

Let’s start with syntax. Before we tackle the Message example, I want to go back to the Character example from previous posts, because it’s a bit easier for explanatory purposes. Here is some Rust code that declares a struct Character, creates an owned copy of it, and then gets a few references into it.

struct Character {
    name: String,
    class: String,
    hp: u32,
}

let ch: Character = Character {
    name: format!("Ferris"),
    class: format!("Rustacean"),
    hp: 22
};

let p: &Character = &ch;
let q: &String = &p.name;

The Dada equivalent to this code is as follows:

class Character(
    name: String,
    klass: String,
    hp: u32,
)

let ch: Character = Character("Tzara", "Dadaist", 22)
let p: ref[ch] Character = ch
let q: ref[p] String = p.name

The first thing to note is that, in Dada, the default when you name a variable or a place is to create a reference. So let p = ch doesn’t move ch, as it would in Rust, it creates a reference to the Character stored in ch. You could also explicitly write let p = ch.ref, but that is not preferred. Similarly, let q = p.name creates a reference to the value in the field name. (If you wanted to move the character, you would write let ch2 = ch.give, not let ch2 = ch as in Rust.)

Notice that I said let p = ch “creates a reference to the Character stored in ch”. In particular, I did not say “creates a reference to ch”. That’s a subtle choice of wording, but it has big implications.

References in Dada are not pointers

The reason I wrote that let p = ch “creates a reference to the Character stored in ch” and not “creates a reference to ch” is because, in Dada, references are not pointers. Rather, they are shallow copies of the value, very much like how we saw in the previous post that a shared Character acts like an Arc<Character> but is represented as a shallow copy.

So where in Rust the following code…

let ch = Character { ... };
let p = &ch;
let q = &ch.name;

…looks like this in memory…

        # Rust memory representation

            Stack                       Heap
            ─────                       ────

┌───► ch: Character {
│ ┌───► name: String {
│ │         buffer: ───────────► "Ferris"
│ │         length: 6
│ │         capacity: 12
│ │     },
│ │     ...
│ │   }
│ │   
└──── p
  │
  └── q

in Dada, code like this

let ch = Character(...)
let p = ch
let q = ch.name

would look like so

# Dada memory representation

Stack                       Heap
─────                       ────

ch: Character {
    name: String {
            buffer: ───────┬───► "Ferris"
            length: 6      │
            capacity: 12   │
    },                     │
    ..                     │
}                          │
                           │
p: Character {             │
    name: String {         │
            buffer: ───────┤
            length: 6      │
            capacity: 12   │
    ...                    │
}                          │
    }                      │
                           │
q: String {                │
    buffer: ───────────────┘
    length: 6
    capacity: 12
}

Clearly, the Dada representation takes up more memory on the stack. But note that it doesn’t duplicate the memory in the heap, which tends to be where the vast majority of the data is found.

Dada talks about values not references

This gets at something important. Rust, like C, makes pointers first-class. So given x: &String, x refers to the pointer and *x refers to its referent, the String.

Dada, like Java, goes another way. x: ref String is a String value – including in memory representation! The difference between a given String, shared String, and ref String is not in their memory layout, all of them are the same, but they differ in whether they own their contents.²

So in Dada, there is no *x operation to go from “pointer” to “referent”. That doesn’t make sense. Your variable always contains a string, but the permissions you have to use that string will change.

In fact, the goal is that people don’t have to learn the memory representation as they learn Dada, you are supposed to be able to think of Dada variables as if they were all objects on the heap, just like in Java or Python, even though in fact they are stored on the stack.³

Rust does not permit moves of borrowed data

In Rust, you cannot move values while they are borrowed. So if you have code like this that moves ch into ch1…

let ch = Character { ... };
let name = &ch.name; // create reference
let ch1 = ch;        // moves `ch`

…then this code only compiles if name is not used again:

let ch = Character { ... };
let name = &ch.name; // create reference
let ch1 = ch;        // ERROR: cannot move while borrowed
let name1 = name;    // use reference again

…but Dada can

There are two reasons that Rust forbids moves of borrowed data:

• References are pointers, so those pointers may become invalidated. In the example above, name points to the stack slot for ch, so if ch were to be moved into ch1, that makes the reference invalid.
• The type system would lose track of things. Internally, the Rust borrow checker has a kind of “indirection”. It knows that ch is borrowed for some span of the code (a “lifetime”), and it knows that the lifetime in the type of name is related to that lifetime, but it doesn’t really know that name is borrowed from ch in particular.⁴

Neither of these apply to Dada:

• Because references are not pointers into the stack, but rather shallow copies, moving the borrowed value doesn’t invalidate their contents. They remain valid.
• Because Dada’s types reference actual variable names, we can modify them to reflect moves.

Dada tracks moves in its types

OK, let’s revisit that Rust example that was giving us an error. When we convert it to Dada, we find that it type checks just fine:

class Character(...) // as before
let ch: given Character = Character(...)
let name: ref[ch.name] String = ch.name
//            -- originally it was borrowed from `ch`
let ch1 = ch.give
//        ------- but `ch` was moved to `ch1`
let name1: ref[ch1.name] = name
//             --- now it is borrowed from `ch1`

Woah, neat! We can see that when we move from ch into ch1, the compiler updates the types of the variables around it. So actually the type of name changes to ref[ch1.name] String. And then when we move from name to name1, that’s totally valid.

In PL land, updating the type of a variable from one thing to another is called a “strong update”. Obviously things can get a bit complicated when control-flow is involved, e.g., in a situation like this:

let ch = Character(...)
let ch1 = Character(...)
let name = ch.name
if some_condition_is_true() {
    // On this path, the type of `name` changes
    // to `ref[ch1.name] String`, and so `ch`
    // is no longer considered borrowed.
    ch1 = ch.give
    ch = Character(...) // not borrowed, we can mutate
} else {
    // On this path, the type of `name`
    // remains unchanged, and `ch` is borrowed.
}
// Here, the types are merged, so the
// type of `name` is `ref[ch.name, ch1.name] String`.
// Therefore, `ch` is considered borrowed here.

Renaming lets us call functions with borrowed values

OK, let’s take the next step. Let’s define a Dada function that takes an owned value and another value borrowed from it, like the name, and then call it:

fn character_and_name(
    ch1: given Character,
    name1: ref[ch1] String,
) {
    // ... does something ...
}

We could call this function like so, as you might expect:

let ch = Character(...)
let name = ch.name
character_and_name(ch.give, name)

So…how does this work? Internally, the type checker type-checks a function call by creating a simpler snippet of code, essentially, and then type-checking that. It’s like desugaring but only at type-check time. In this simpler snippet, there are a series of let statements to create temporary variables for each argument. These temporaries always have an explicit type taken from the method signature, and they are initialized with the values of each argument:

// type checker "desugars" `character_and_name(ch.give, name)`
// into more primitive operations:
let tmp1: given Character = ch.give
    //    ---------------   -------
    //            |         taken from the call
    //    taken from fn sig
let tmp2: ref[tmp1.name] String = name
    //    ---------------------   ----
    //            |         taken from the call
    //    taken from fn sig,
    //    but rewritten to use the new
    //    temporaries

If this type checks, then the type checker knows you have supplied values of the required types, and so this is a valid call. Of course there are a few more steps, but that’s the basic idea.

Notice what happens if you supply data borrowed from the wrong place:

let ch = Character(...)
let ch1 = Character(...)
character_and_name(ch, ch1.name)
//                     --- wrong place!

This will fail to type check because you get:

let tmp1: given Character = ch.give
let tmp2: ref[tmp1.name] String = ch1.name
    //                            --------
    //       has type `ref[ch1.name] String`,
    //       not `ref[tmp1.name] String`

Class constructors are “just” special functions

So now, if we go all the way back to our original example, we can see how the Message example worked:

class Message(
    list: String
    items: Vec[ref[self.list] String]
)

Basically, when you construct a Message(list, items), that’s “just another function call” from the type system’s perspective, except that self in the signature is handled carefully.

This is modeled, not implemented

I should be clear, this system is modeled in the dada-model repository, which implements a kind of “mini Dada” that captures what I believe to be the most interesting bits. I’m working on fleshing out that model a bit more, but it’s got most of what I showed you here.⁵ For example, here is a test that you get an error when you give a reference to the wrong value.

The “real implementation” is lagging quite a bit, and doesn’t really handle the interesting bits yet. Scaling it up from model to real implementation involves solving type inference and some other thorny challenges, and I haven’t gotten there yet – though I have some pretty interesting experiments going on there too, in terms of the compiler architecture.⁶

This could apply to Rust

I believe we could apply most of this system to Rust. Obviously we’d have to rework the borrow checker to be based on places, but that’s the straight-forward part. The harder bit is the fact that &T is a pointer in Rust, and that we cannot readily change. However, for many use cases of self-references, this isn’t as important as it sounds. Often, the data you wish to reference is living in the heap, and so the pointer isn’t actually invalidated when the original value is moved.

Consider our opening example. You might imagine Rust allowing something like this in Rust:

struct Message {
    list: String,
    items: Vec<&{self.list} str>,
}

In this case, the str data is heap-allocated, so moving the string doesn’t actually invalidate the &str value (it would invalidate an &String value, interestingly).

In Rust today, the compiler doesn’t know all the details of what’s going on. String has a Deref impl and so it’s quite opaque whether str is heap-allocated or not. But we are working on various changes to this system in the Beyond the & goal, most notably the Field Projections work. There is likely some opportunity to address this in that context, though to be honest I’m behind in catching up on the details.

There are many chasms out there

For some time now I’ve been debating with myself, has Rust “crossed the chasm”? If you’re not familiar with that term, it comes from a book that gives a kind of “pop-sci” introduction to the Technology Adoption Life Cycle.

The answer, of course, is it depends on who you ask. Within Amazon, where I have the closest view, the answer is that we are “most of the way across”: Rust is squarely established as the right way to build at-scale data planes or resource-aware agents and it is increasingly seen as the right choice for low-level code in devices and robotics as well – but there remains a lingering perception that Rust is useful for “those fancy pants developers at S3” (or wherever) but a bit overkill for more average development³.

On the other hand, within the realm of Safety Critical Software, as Pete LeVasseur wrote in a recent rust-lang blog post, Rust is still scrabbling for a foothold. There are a number of successful products but most of the industry is in a “wait and see” mode, letting the early adopters pave the path.

“Crossing the chasm” means finding “reference customers”

The big idea that I at least took away from reading Crossing the Chasm and other references on the technology adoption life cycle is the need for “reference customers”. When you first start out with something new, you are looking for pioneers and early adopters that are drawn to new things:

What an early adopter is buying [..] is some kind of change agent. By being the first to implement this change in the industry, the early adopters expect to get a jump on the competition. – from Crossing the Chasm

But as your technology matures, you have to convince people with a lower and lower tolerance for risk:

The early majority want to buy a productivity improvement for existing operations. They are looking to minimize discontinuity with the old ways. They want evolution, not revolution. – from Crossing the Chasm

So what is most convincing to people to try something new? The answer is seeing that others like them have succeeded.

You can see this at play in both the Amazon example and the Safety Critical Software example. Clearly seeing Rust used for network services doesn’t mean it’s ready to be used in your car’s steering column⁴. And even within network services, seeing a group like S3 succeed with Rust may convince other groups building at-scale services to try Rust, but doesn’t necessarily persuade a team to use Rust for their next CRUD service. And frankly, it shouldn’t! They are likely to hit obstacles.

Ubuntu is helping Rust “cross the (user-land linux) chasm”

All of this was on my mind as I watched the keynote by Jon Seager, the VP of Engineering at Canonical, which is the company behind Ubuntu. Similar to Lars Bergstrom’s epic keynote from year’s past on Rust adoption within Google, Jon laid out a pitch for why Canonical is adopting Rust that was at once visionary and yet deeply practical.

“Visionary and yet deeply practical” is pretty much the textbook description of what we need to cross from early adopters to early majority. We need folks who care first and foremost about delivering the right results, but are open to new ideas that might help them do that better; folks who can stand on both sides of the chasm at once.

Jon described how Canonical focuses their own development on a small set of languages: Python, C/C++, and Go, and how they had recently brought in Rust and were using it as the language of choice for new foundational efforts, replacing C, C++, and (some uses of) Python.

Ubuntu is building the bridge across the chasm

Jon talked about how he sees it as part of Ubuntu’s job to “pay it forward” by supporting the construction of memory-safe foundational utilities. Jon meant support both in terms of finances – Canonical is sponsoring the Trifecta Tech Foundation’s to develop sudo-rs and ntpd-rs and sponsoring the uutils org’s work on coreutils – and in terms of reputation. Ubuntu can take on the risk of doing something new, prove that it works, and then let others benefit.

Remember how the Crossing the Chasm book described early majority people? They are “looking to minimize discontinuity with the old ways”. And what better way to do that than to have drop-in utilities that fit within their existing workflows.

The challenge for Rust: listening to these new adopters

With new adoption comes new perspectives. On Thursday night I was at dinner⁵ organized by Ernest Kissiedu⁶. Jon Seager was there along with some other Rust adopters from various industries, as were a few others from the Rust Foundation and the open-source project.

Ernest asked them to give us their unvarnished takes on Rust. Jon made the provocative comment that we needed to revisit our policy around having a small standard library. He’s not the first to say something like that, it’s something we’ve been hearing for years and years – and I think he’s right! Though I don’t think the answer is just to ship a big standard library. In fact, it’s kind of a perfect lead-in to (what I hope will be) my next blog post, which is about a project I call “battery packs”⁷.

To grow, you have to change

The broader point though is that shifting from targeting “pioneers” and “early adopters” to targeting “early majority” sometimes involves some uncomfortable changes:

Transition between any two adoption segments is normally excruciatingly awkward because you must adopt new strategies just at the time you have become most comfortable with the old ones. [..] The situation can be further complicated if the high-tech company, fresh from its marketing success with visionaries, neglects to change its sales pitch. [..] The company may be saying “state-of-the-art” when the pragmatist wants to hear “industry standard”. – Crossing the Chasm (emphasis mine)

Not everybody will remember it, but in 2016 there was a proposal called the Rust Platform. The idea was to bring in some crates and bless them as a kind of “extended standard library”. People hated it. After all, they said, why not just add dependencies to your Cargo.toml? It’s easy enough. And to be honest, they were right – at least at the time.

I think the Rust Platform is a good example of something that was a poor fit for early adopters, who want the newest thing and don’t mind finding the best crates, but which could be a great fit for the Early Majority.⁸

Anyway, I’m not here to argue for one thing or another in this post, but more for the concept that we have to be open to adapting our learned wisdom to new circumstances. In the past, we were trying to bootstrap Rust into the industry’s consciousness – and we have succeeded.

The task before us now is different: we need to make Rust the best option not just in terms of “what it could be” but in terms of “what it actually is” – and sometimes those are in tension.

Another challenge for Rust: turning adoption into investment

Later in the dinner, the talk turned, as it often does, to money. Growing Rust adoption also comes with growing needs placed on the Rust project and its ecosystem. How can we connect the dots? This has been a big item on my mind, and I realize in writing this paragraph how many blog posts I have yet to write on the topic, but let me lay out a few interesting points that came up over this dinner and at other recent points.

Investment can mean contribution, particularly for open-source orgs

First, there are more ways to offer support than $$. For Canonical specifically, as they are an open-source organization through-and-through, what I would most want is to build stronger relationships between our organizations. With the Rust for Linux developers, early on Rust maintainers were prioritizing and fixing bugs on behalf of RfL devs, but more and more, RfL devs are fixing things themselves, with Rust maintainers serving as mentors. This is awesome!

Money often comes before a company has adopted Rust, not after

Second, there’s an interesting trend about $$ that I’ve seen crop up in a few places. We often think of companies investing in the open-source dependencies that they rely upon. But there’s an entirely different source of funding, and one that might be even easier to tap, which is to look at companies that are considering Rust but haven’t adopted it yet.

For those “would be” adopters, there are often individuals in the org who are trying to make the case for Rust adoption – these individuals are early adopters, people with a vision for how things could be, but they are trying to sell to their early majority company. And to do that, they often have a list of “table stakes” features that need to be supported; what’s more, they often have access to some budget to make these things happen.

This came up when I was talking to Alexandru Radovici, the Foundation’s Silver Member Directory, who said that many safety critical companies have money they’d like to spend to close various gaps in Rust, but they don’t know how to spend it. Jon’s investments in Trifecta Tech and the uutils org have the same character: he is looking to close the gaps that block Ubuntu from using Rust more.

Conclusions…?

Well, first of all, you should watch Jon’s talk. “Brilliant”, as the Brits have it.

But my other big thought is that this is a crucial time for Rust. We are clearly transitioning in a number of areas from visionaries and early adopters towards that pragmatic majority, and we need to be mindful that doing so may require us to change some of the way that we’ve always done things. I liked this paragraph from Crossing the Chasm:

To market successfully to pragmatists, one does not have to be one – just understand their values and work to serve them. To look more closely into these values, if the goal of visionaries is to take a quantum leap forward, the goal of pragmatists is to make a percentage improvement–incremental, measurable, predictable progress. [..] To market to pragmatists, you must be patient. You need to be conversant with the issues that dominate their particular business. You need to show up at the industry-specific conferences and trade shows they attend.

Re-reading Crossing the Chasm as part of writing this blog post has really helped me square where Rust is – for the most part, I think we are still crossing the chasm, but we are well on our way. I think what we see is a consistent trend now where we have Rust champions who fit the “visionary” profile of early adopters successfully advocating for Rust within companies that fit the pragmatist, early majority profile.

Open source can be a great enabler to cross the chasm…

It strikes me that open-source is just an amazing platform for doing this kind of marketing. Unlike a company, we don’t have to do everything ourselves. We have to leverage the fact that open source helps those who help themselves – find those visionary folks in industries that could really benefit from Rust, bring them into the Rust orbit, and then (most important!) support and empower them to adapt Rust to their needs.

…but only if we don’t get too “middle school” about it

This last part may sound obvious, but it’s harder than it sounds. When you’re embedded in open source, it seems like a friendly place where everyone is welcome. But the reality is that it can be a place full of cliques and “oral traditions” that “everybody knows”⁹. People coming with an idea can get shutdown for using the wrong word. They can readily mistake the, um, “impassioned” comments from a random contributor (or perhaps just a troll…) for the official word from project leadership. It only takes one rude response to turn somebody away.

What Rust needs most is empathy

So what will ultimately help Rust the most to succeed? Empathy in Open Source. Let’s get out there, find out where Rust can help people, and make it happen. Exciting times!

We are shooting for a GC-like experience without GC

Let’s start with the goal: earlier, I said that Dada was like “Rust where you never have to type as_ref”. But what I really meant is that I want a GC-like experience–without the GC.

We are shooting for a “composable” experience

I also often use the word “composable” to describe the Dada experience I am shooting for. Composable means that you can take different things and put them together to achieve something new.

Obviously Rust has many composable patterns – the Iterator APIs, for example. But what I have found is that Rust code is often very brittle: there are many choices when it comes to how you declare your data structures and the choices you make will inform how those data structures can be consumed.

Running example: Character

Defining the Character type

Let’s create a type that we can use as a running example throughout the post: Character. In Rust, we might define a Character like so:

#[derive(Default)]
struct Character {
    name: String,
    class: String,
    hp: u32,
}

Creating and Arc’ing the Character

Now, suppose that, for whatever reason, we are going to build up a character programmatically:

let mut ch = Character::default();
ch.name.push_str("Ferris");
ch.class.push_str("Rustacean");
ch.hp = 44;

So far, so good. Now suppose I want to share that same Character struct so it can be referenced from a lot of places without deep copying. To do that, I am going to put it in an Arc:

let mut ch = Character::default();
ch.name.push_str("Ferris");
// ...
let ch1 = Arc::new(ch);
let ch2 = ch1.clone();

OK, cool! Now I have a Character that is readily sharable. That’s great.

Rust is composable here, which is cool, we like that

Side note but this is an example of where Rust is composable: we defined Character once in a fully-owned way and we were able to use it mutably (to build it up imperatively over time) and then able to “freeze” it and get a read-only, shared copy of Character. This gives us the advantages of an imperative programming language (easy data construction and manipulation) and the advantages of a functional language (immutability prevents bugs when things are referenced from many disjoint places). Nice!

Creating and Arc’ing the Character

Now, suppose that I have some other code, written independently, that just needs to store the character’s name. That code winds up copying the name into a lot of different places. So, just like we used Arc to let us cheaply reference a single character from multiple places, it uses Arc so it can cheaply reference the character’s name from multiple places:

struct CharacterSheetWidget {
    // Use `Arc<String>` and not `String` because
    // we wind up copying this into name different
    // places and we don't want to deep clone
    // the string each time.
    name: Arc<String>,

    // ... assume more fields here ...
}

OK. Now comes the rub. I want to create a character-sheet widget from our shared character:

fn create_character_sheet_widget(ch: Arc<Character>) -> CharacterSheetWidget {
    CharacterSheetWidget {
        // FIXME: Huh, how do I bridge this gap?
        // I guess I have to do this.
        name: Arc::new(ch.name.clone()),

        // ... assume more fields here ...
    }
}

Shoot, that’s frustrating! What I would like to do is to write name: ch.name.clone() or something similar (actually I’d probably like to just write ch.name, but anyhow) and get back an Arc<String>. But I can’t do that. Instead, I have to deeply clone the string and allocate a new Arc. Of course any subsequent clones will be cheap. But it’s not great.

Rust often gives rise to these kind of “impedance mismatches”

I often find patterns like this arise in Rust: there’s a bit of an “impedance mismatch” between one piece of code and another. The solution varies, but it’s generally something like

• clone some data – it’s not so big anyway, screw it (that’s what happened here).
• refactor one piece of code – e.g., modify the Character class to store an Arc<String>. Of course, that has ripple effects, e.g., we can no longer write ch.name.push_str(...) anymore, but have to use Arc::get_mut or something.
• invoke some annoying helper – e.g., write opt.as_ref() to convert from an &Option<String> to a Option<&String> or write a &**r to convert from a &Arc<String> to a &str.

The goal with Dada is that we don’t have that kind of thing.

Sharing is how Dada copies

So let’s walk through how that same Character example would play out in Dada. We’ll start by defining the Character class:

class Character(
    name: String,
    klass: String,  # Oh dang, the perils of a class keyword!
    hp: u32,
)

Just as in Rust, we can create the character and then modify it afterwards:

class Character(name: String, klass: String, hp: u32)

let ch: given Character = Character("", "", 22)
      # ----- remember, the "given" permission
      #       means that `ch` is fully owned
ch.name!.push("Tzara")
ch.klass!.push("Dadaist")
   #    - and the `!` signals mutation

The .share operator creates a shared object

Cool. Now, I want to share the character so it can be referenced from many places. In Rust, we created an Arc, but in Dada, sharing is “built-in”. We use the .share operator, which will convert the given Character (i.e., fully owned character) into a shared Character:

class Character(name: String, klass: String, hp: u32)

let ch = Character("", "", 22)
ch!.push("Tzara")
ch!.push("Dadaist")

let ch1: shared Character = ch.share
      #  ------                -----
      # The `share` operator consumes `ch`
      # and returns the same object, but now
      # with *shared* permissions.

shared objects can be copied freely

Now that we have a shared character, we can copy it around:

class Character(name: String, klass: String, hp: u32)

# Create a shared character to start
let ch1 = Character("Tzara", "Dadaist", 22).share
    #                                       -----

# Create another shared character
let ch2 = ch1

Sharing propagates from owner to field

When you have a shared object and you access its field, what you get back is a shared (shallow) copy of the field:

class Character(...)

# Create a `shared Character`
let ch: shared Character = Character("Tristan Tzara", "Dadaist", 22).share
      # ------                                                       -----

# Extracting the `name` field gives a `shared String`
let name: shared String = ch1.name
        # ------

Propagation using a Vec

To drill home how cool and convenient this is, imagine that I have a Vec[String] that I share with .share:

let v: shared Vec[String] = ["Hello", "Dada"].share

and then I share it with v.share. What I get back is a shared Vec[String]. And when I access the elements of that, I get back a shared String:

let v = ["Hello", "Dada"].share
let s: shared String = v[0]

This is as if one could take a Arc<Vec<String>> in Rust and get out a Arc<String>.

How sharing is implemented

So how is sharing implemented? The answer lies in a not-entirely-obvious memory layout. To see how it works, let’s walk how a Character would be laid out in memory:

# Character type we saw earlier.
class Character(name: String, klass: String, hp: u32)

# String type would be something like this.
class String {
    buffer: Pointer[char]
    initialized: usize
    length: usize
}

Here Pointer is a built-in type that is the basis for Dada’s unsafe code system.¹

Layout of a given Character in memory

Now imagine we have a Character like this:

let ch = Character("Duchamp", "Dadaist", 22)

The character ch would be laid out in memory something like this (focusing just on the name field):

[Stack frame]              [Heap]         
ch: Character {                           
    _flag: 1                              
    name: String {                        
        _flag: 1         { _ref_count: 1  
        buffer: ──────────►'D'            
        initialized: 7     ...            
        capacity: 8        'p' }          
    }                                     
    klass: ...                            
    hp: 22                                
}                                         

Let’s talk this through. First, every object is laid out flat in memory, just like you would see in Rust. So the fields of ch are stored on the stack, and the name field is laid out flat within that.

Each object that owns other objects begins with a hidden field, _flag. This field indicates whether the object is shared or not (in the future we’ll add more values to account for other permissions). If the field is 1, the object is not shared. If it is 2, then it is shared.

Heap-allocated objects (i.e., using Pointer[]) begin with a ref-count before the actual data (actually this is at the offset of -4). In this case we have a Pointer[char] so the actual data that follows are just simple characters.

Layout of a shared Character in memory

If I were to instead create a shared character:

let ch1 = Character("Duchamp", "Dadaist", 22).share
          #                                   -----

The memory layout would be the same, but the flag field on the character is now 2:

[Stack frame]              [Heap]         
ch: Character {                           
    _flag: 2 👈 (This is 2 now!)                             
    name: String {                        
        _flag: 1         { _ref_count: 1  
        buffer: ──────────►'D'            
        initialized: 7     ...            
        capacity: 8        'p' }          
    }                                     
    klass: ...                            
    hp: 22                                
}                                         

Copying a shared Character

Now imagine that we created two copies of the same shared character:

let ch1 = Character("Duchamp", "Dadaist", 22).share
let ch2 = ch1

What happens is that we will copy all the fields of _ch1 and then, because _flag is 2, we will increment the ref-counts for the heap-allocated data within:

[Stack frame]              [Heap]            
ch1: Character {                             
    _flag: 2                                 
    name: String {                           
        _flag: 1         { _ref_count: 2     
        buffer: ────────┬─►'D'        👆     
        initialized: 7  │  ...      (This is 
        capacity: 8     │  'p' }     2 now!) 
    }                   │                    
    class: ...          │                    
    hp: 22              │                    
}                       │                    
                        │                    
ch2: Character {        │                    
    _flag: 2            │                    
    name: String {      │                    
        _flag: 1        │                    
        buffer: ────────┘                    
        initialized: 7                       
        capacity: 8                          
    }                                        
    class: ...                               
    hp: 22                                   
}                                            

Copying out the name field

Now imagine we were to copy out the name field, instead of the entire character:

let ch1 = Character("Duchamp", "Dadaist", 22).share
let name = ch1.name

…what happens is that:

1. traversing ch1, we observe that the _flag field is 2 and therefore ch1 is shared
2. we copy out the String fields from name. Because the character is shared:
   • we modify the _flag field on the new string to 2
   • we increment the ref-count for any heap values

The result is that you get:

[Stack frame]              [Heap]       
ch1: Character {                        
    _flag: 2                            
    name: String {                      
        _flag: 1         { _ref_count: 2
        buffer: ────────┬─►'D'          
        initialized: 7  │  ...          
        capacity: 8     │  'p' }        
    }                   │               
    class: ...          │               
    hp: 22              │               
}                       │               
                        │               
name: String {          │               
    _flag: 2            │               
    buffer: ────────────┘               
    initialized: 7                      
    capacity: 8                         
}                                       

“Sharing propagation” is one example of permission propagation

This post showed how shared values in Dada work and showed how the shared permission propagates when you access a field. Permissions are how Dada manages object lifetimes. We’ve seen two so far

• the given permission indicates a uniquely owned value (T, in Rust-speak);
• the shared permission indicates a copyable value (Arc<T> is the closest Rust equivalent).

In future posts we’ll see the ref and mut permissions, which roughly correspond to & and &mut, and talk out how the whole thing fits together.

Dada is more than a pretty face

This is the first post where we started to see a bit more of Dada’s character. Reading over the previous few posts, you could be forgiven for thinking Dada was just a cute syntax atop familiar Rust semantics. But as you can see from how shared works, Dada is quite a bit more than that.

I like to think of Dada as “opinionated Rust” in some sense. Unlike Rust, it imposes some standards on how things are done. For example, every object (at least every object with a heap-allocated field) has a _flag field. And every heap allocation has a ref-count.

These conventions come at some modest runtime cost. My rule is that basic operations are allowed to do “shallow” operations, e.g., toggling the _flag or adjusting the ref-counts on every field. But they cannot do “deep” operations that require traversing heap structures.

In exchange for adopting conventions and paying that cost, you get “composability”, by which I mean that permissions in Dada (like shared) flow much more naturally, and types that are semantically equivalent (i.e., you can do the same things with them) generally have the same layout in memory.

Class struggle

Classes in Dada are one of the basic ways that we declare new types (there are also enums, we’ll get to that later).

The most convenient way to declare a class is to put the fields in parentheses. This implicitly declares a constructor at the same time:

class Point(x: u32, y: u32) {}

This is in fact sugar for a more Rust like form:

class Point {
    x: u32
    y: u32
    fn new() -> Point {
        Point { x, y }
    }
}

And you can create an instance of a class by calling the constructor:

let p = Point(22, 44) // sugar for Point.new(22, 44)

Mutating fields

I can mutate the fields of p as you would expect:

p.x += 1
p.x = p.y

Read by default

In Dada, the default when you declare a parameter is that you are getting read-only access:

fn print_point(p: Point) {
    print("The point is {p.x}, {p.y}")
}

let p = Point(22, 44)
print_point(p)

If you attempt to mutate the fields of a parameter, that would get you an error:

fn print_point(p: Point) {
    p.x += 1 # <-- ERROR!
}

Use ! to mutate

If you declare a parameter with !, then it becomes a mutable reference to a class instance from your caller:

fn translate_point(point!: Point, x: u32, y: u32) {
    point.x += x
    point.y += y
}

In Rust, this would be like point: &mut Point. When you call translate_point, you also put a ! to indicate that you are passing a mutable reference:

let p = Point(22, 44)     # Create point
print_point(p)            # Prints 22, 44
translate_point(p!, 2, 2) # Mutate point
print_point(p)            # Prints 24, 46 

As you can see, when translate_point modifies p.x, that changes p in place.

Moves are explicit

If you’re familiar with Rust, that last example may be a bit surprising. In Rust, a call like print_point(p) would move p, giving ownership away. Trying to use it later would give an error. That’s because the default in Dada is to give a read-only reference, like &x in Rust (this gives the right intuition but is also misleading; we’ll see in a future post that references in Dada are different from Rust in one very important way).

If you have a function that needs ownership of its parameter, you declare that with given:

fn take_point(p: given Point) {
    // ...
}

And on the caller’s side, you call such a function with .give:

let p = Point(22, 44)
take_point(p.give)
take_point(p.give) # <-- Error! Can't give twice.

Comparing with Rust

It’s interesting to compare some Rust and Dada code side-by-side:

Design rationale and objectives

Convenient is the default

The most convenient things are the shortest and most common. So we make reads the default.

Everything is explicit but unobtrusive

The . operator in Rust can do a wide variety of things depending on the method being called. It might mutate, move, create a temporary, etc. In Dada, these things are all visible at the callsite– but they are unobtrusive.

This actually dates from Dada’s “gradual programming” days – after all, if you don’t have type annotations on the method, then you can’t decide foo.bar() should take a shared or mutable borrow of foo. So we needed a notation where everything is visible at the call-site and explicit.

Postfix operators play more nicely with others

Dada tries hard to avoid prefix operators like &mut, since they don’t compose well with . notation.

Following on my Fun with Dada post, this post is going to start teaching Dada. I’m going to keep each post short – basically just what I can write while having my morning coffee.¹

You have the right to write code

Here is a very first Dada program

println("Hello, Dada!")

I think all of you will be able to guess what it does. Still, there is something worth noting even in this simple program:

“You have the right to write code. If you don’t write a main function explicitly, one will be provided for you.” Early on I made the change to let users omit the main function and I was surprised by what a difference it made in how light the language felt. Easy change, easy win.

Convenient is the default

Here is another Dada program

let name = "Dada"
println("Hello, {name}!")

Unsurprisingly, this program does the same thing as the last one.

“Convenient is the default.” Strings support interpolation (i.e., {name}) by default. In fact, that’s not all they support, you can also break them across lines very conveniently. This program does the same thing as the others we’ve seen:

let name = "Dada"
println("
    Hello, {name}!
")

When you have a " immediately followed by a newline, the leading and trailing newline are stripped, along with the “whitespace prefix” from the subsequent lines. Internal newlines are kept, so something like this:

let name = "Dada"
println("
    Hello, {name}!
    
    How are you doing?
")

would print

Hello, Dada!

How are you doing?

Just one familiar String

Of course you could also annotate the type of the name variable explicitly:

let name: String = "Dada"
println("Hello, {name}!")

You will find that it is String. This in and of itself is not notable, unless you are accustomed to Rust, where the type would be &'static str. This is of course a perennial stumbling block for new Rust users, but more than that, I find it to be a big annoyance – I hate that I have to write "Foo".to_string() or format!("Foo") everywhere that I mix constant strings with strings that are constructed.

Similar to most modern languages, strings in Dada are immutable. So you can create them and copy them around:

let name: String = "Dada"
let greeting: String = "Hello, {name}"
let name2: String = name

Next up: mutation, permissions

OK, we really just scratched the surface here! This is just the “friendly veneer” of Dada, which looks and feels like a million other languages. Next time I’ll start getting into the permission system and mutation, where things get a bit more interesting.

Waaaaaay back in 2021, I started experimenting with a new programming language I call “Dada”. I’ve been tinkering with it ever since and I just realized that (oh my gosh!) I’ve never written even a single blog post about it! I figured I should fix that. This post will introduce some of the basic concepts of Dada as it is now.

Before you get any ideas, Dada isn’t fit for use. In fact the compiler doesn’t even really work because I keep changing the language before I get it all the way working. Honestly, Dada is more of a “stress relief” valve for me than anything else¹ – it’s fun to tinker with a programming language where I don’t have to worry about backwards compatibility, or RFCs, or anything else.

That said, Dada has been a very fertile source of ideas that I think could be applicable to Rust. And not just for language design: playing with the compiler is also what led to the new salsa design², which is now used by both rust-analyzer and Astral’s ty. So I really want to get those ideas out there!

I took a break, but I’m back baby!

I stopped hacking on Dada about a year ago³, but over the last few days I’ve started working on it again. And I realized, hey, this is a perfect time to start blogging! After all, I have to rediscover what I was doing anyway, and writing about things is always the best way to work out the details.

Dada started as a gradual programming experiment, but no longer

Dada has gone through many phases. Early on, the goal was to build a gradually typed programming language that I thought would be easier for people to learn.

The idea was that you could start writing without any types at all and just execute the program. There was an interactive playground that would let you step through and visualize the “borrow checker” state (what Dada calls permissions) as you go. My hope was that people would find that easier to learn than working with type checker checker.

I got this working and it was actually pretty cool. I gave a talk about it at the Programming Language Mentoring Workshop in 2022, though skimming that video it doesn’t seem like I really demo’d the permission modeling. Too bad.

At the same time, I found myself unconvinced that the gradually typed approach made sense. What I wanted was that when you executed the program without type annotations, you would get errors at the point where you violated a borrow. And that meant that the program had to track a lot of extra data, kind of like miri does, and it was really only practical as a teaching tool. I still would like to explore that, but it also felt like it was adding a lot of complexity to the language design for something that would only be of interest very early in a developer’s journey⁴.

Therefore, I decided to start over, this time, to just focus on the static type checking part of Dada.

Dada is like a streamlined Rust

Dada today is like Rust but streamlined. The goal is that Dada has the same basic “ownership-oriented” feel of Rust, but with a lot fewer choices and nitty-gritty details you have to deal with.⁵

Rust often has types that are semantically equivalent, but different in representation. Consider &Option<String> vs Option<&String>: both of them are equivalent in terms of what you can do with them, but of course Rust makes you carefully distinguish between them. In Dada, they are the same type. Dada also makes &Vec<String>, &Vec<&String>, &[String], &[&str], and many other variations all the same type too. And before you ask, it does it without heap allocating everything or using a garbage collector.

To put it pithily, Dada aims to be “Rust where you never have to call as_ref()”.

Dada has a fancier borrow checker

Dada also has a fancier borrow checker, one which already demonstrates much of the borrow checker within, although it doesn’t have view types. Dada’s borrow checker supports internal borrows (e.g., you can make a struct that has fields that borrow from other fields) and it supports borrow checking without lifetimes. Much of this stuff can be brought to Rust, although I did tweak a few things in Dada that made some aspects easier.

Dada targets WebAssembly natively

Somewhere along the line in refocusing Dada, I decided to focus exclusively on building WebAssembly components. Initially I felt like targeting WebAssembly would be really convenient:

• WebAssembly is like a really simple and clean assembly language, so writing the compiler backend is easy.
• WebAssembly components are explicitly designed to bridge between languages, so they solve the FFI problem for you.
• With WASI, you even get a full featured standard library that includes high-level things like “fetch a web page”. So you can build useful things right off the bat.

WebAssembly and on-demand compilation = compile-time reflection almost for free

But I came to realize that targeting WebAssembly has another advantage: it makes compile-time reflection almost trivial. The Dada compiler is structured in a purely on-demand fashion. This means we can compile one function all the way to WebAssembly bytecode and leave the rest of the crate untouched.

And once we have the WebAssembly bytecode, we can run that from inside the compiler! With wasmtime, we have a high quality JIT that runs very fast. The code is even sandboxed!

So we can have a function that we compile and run during execution and use to produce other code that will be used by other parts of the compilation step. In other words, we get something like miri or Zig’s comptime for free, essentially. Woah.

Wish you could try it? Me too!

Man, writing this blog post made ME excited to play with Dada. Too bad it doesn’t actually work. Ha! But I plan to keep plugging away on the compiler and get it to the point of a live demo as soon as I can. Hard to say exactly how long that will take.

In the meantime, to help me rediscover how things work, I’m going to try to write up a series of blog posts about the type system, borrow checker, and the compiler architecture, all of which I think are pretty interesting.

TL;DR

The idea itself is simple, within a closure (or future), we add the option to write move($expr). This is a value expression (“rvalue”) that desugars into a temporary value that is moved into the closure. So

|| something(&move($expr))

is roughly equivalent to something like:

{ 
    let tmp = $expr;
    || something(&{tmp})
}

How it would look in practice

Let’s go back to one of our running examples, the “Cloudflare example”, which originated in this excellent blog post by the Dioxus folks. As a reminder, this is how the code looks today – note the let _some_value = ... lines for dealing with captures:

// task:  listen for dns connections
let _some_a = self.some_a.clone();
let _some_b = self.some_b.clone();
let _some_c = self.some_c.clone();
tokio::task::spawn(async move {
  	do_something_else_with(_some_a, _some_b, _some_c)
});

Under this proposal it would look something like this:

tokio::task::spawn(async {
    do_something_else_with(
        move(self.some_a.clone()),
        move(self.some_b.clone()),
        move(self.some_c.clone()),
    )
});

There are times when you would want multiple clones. For example, if you want to move something into a FnMut closure that will then give away a copy on each call, it might look like

data_source_iter
    .inspect(|item| {
        inspect_item(item, move(tx.clone()).clone())
        //                      ----------  -------
        //                           |         |
        //                   move a clone      |
        //                   into the closure  |
        //                                     |
        //                             clone the clone
        //                             on each iteration
    })
    .collect();

// some code that uses `tx` later...

Credit for this idea

This idea is not mine. It’s been floated a number of times. The first time I remember hearing it was at the RustConf Unconf, but I feel like it’s come up before that. Most recently it was proposed by Zachary Harrold on Zulip, who has also created a prototype called soupa. Zachary’s proposal, like earlier proposals I’ve heard, used the super keyword. Later on @simulacrum proposed using move, which to me is a major improvement, and that’s the version I ran with here.

This proposal makes closures more “continuous”

The reason that I love the move variant of this proposal is that it makes closures more “continuous” and exposes their underlying model a bit more clearly. With this design, I would start by explaining closures with move expressions and just teach move closures at the end, as a convenient default:

A Rust closure captures the places you use in the “minimal way that it can” – so || vec.len() will capture a shared reference to the vec, || vec.push(22) will capture a mutable reference, and || drop(vec) will take ownership of the vector.

You can use move expressions to control exactly what is captured: so || move(vec).push(22) will move the vector into the closure. A common pattern when you want to be fully explicit is to list all captures at the top of the closure, like so:

|| {
    let vec = move(input.vec); // take full ownership of vec
    let data = move(&cx.data); // take a reference to data
    let output_tx = move(output_tx); // take ownership of the output channel

    process(&vec, &mut output_tx, data)
}

As a shorthand, you can write move || at the top of the closure, which will change the default so that closures > take ownership of every captured variable. You can still mix-and-match with move expressions to get more control. > So the previous closure might be written more concisely like so:

move || {
    process(&input.vec, &mut output_tx, move(&cx.data))
    //       ---------       ---------       --------      
    //           |               |               |         
    //           |               |       closure still  
    //           |               |       captures a ref
    //           |               |       `&cx.data`        
    //           |               |                         
    //       because of the `move` keyword on the clsoure,
    //       these two are captured "by move"
    //       
}

This proposal makes move “fit in” for me

It’s a bit ironic that I like this, because it’s doubling down on part of Rust’s design that I was recently complaining about. In my earlier post on Explicit Capture Clauses I wrote that:

To be honest, I don’t like the choice of move because it’s so operational. I think if I could go back, I would try to refashion our closures around two concepts

• Attached closures (what we now call ||) would always be tied to the enclosing stack frame. They’d always have a lifetime even if they don’t capture anything.
• Detached closures (what we now call move ||) would capture by-value, like move today.

I think this would help to build up the intuition of “use detach || if you are going to return the closure from the current stack frame and use || otherwise”.

move expressions are, I think, moving in the opposite direction. Rather than talking about attached and detached, they bring us to a more unified notion of closures, one where you don’t have “ref closures” and “move closures” – you just have closures that sometimes capture moves, and a “move” closure is just a shorthand for using move expressions everywhere. This is in fact how closures work in the compiler under the hood, and I think it’s quite elegant.

Why not suffix?

One question is whether a move expression should be a prefix or a postfix operator. So e.g.

|| something(&$expr.move)

instead of &move($expr).

My feeling is that it’s not a good fit for a postfix operator because it doesn’t just take the final value of the expression and so something with it, it actually impacts when the entire expression is evaluated. Consider this example:

|| process(foo(bar()).move)

When does bar() get called? If you think about it, it has to be closure creation time, but it’s not very “obvious”.

We reached a similar conclusion when we were considering .unsafe operators. I think there is a rule of thumb that things which delineate a “scope” of code ought to be prefix – though I suspect unsafe(expr) might actually be nice, and not just unsafe { expr }.

Edit: I added this section after-the-fact in response to questions.

Conclusion

I’m going to wrap up this post here. To be honest, what this design really has going for it, above anything else, is its simplicity and the way it generalizes Rust’s existing design. I love that. To me, it joins the set of “yep, we should clearly do that” pieces in this puzzle:

• Add a Share trait (I’ve gone back to preferring the name share 😁)
• Add move expressions

These both seem like solid steps forward. I am not yet persuaded that they get us all the way to the goal that I articulated in an earlier post:

“low-level enough for a Kernel, usable enough for a GUI”

but they are moving in the right direction.

Continuing my series on ergonomic ref-counting, I want to explore another idea, one that I’m calling “just call clone (or alias)”. This proposal specializes the clone and alias methods so that, in a new edition, the compiler will (1) remove redundant or unnecessary calls (with a lint); and (2) automatically capture clones or aliases in move closures where needed.

The goal of this proposal is to simplify the user’s mental model: whenever you see an error like “use of moved value”, the fix is always the same: just call clone (or alias, if applicable). This model is aiming for the balance of “low-level enough for a Kernel, usable enough for a GUI” that I described earlier. It’s also making a statement, which is that the key property we want to preserve is that you can always find where new aliases might be created – but that it’s ok if the fine-grained details around exactly when the alias is created is a bit subtle.

The proposal in a nutshell

Part 1: Closure desugaring that is aware of clones and aliases

Consider this move future:

fn spawn_services(cx: &Context) {
    tokio::task::spawn(async move {
        //                   ---- move future
        manage_io(cx.io_system.alias(), cx.request_name.clone());
        //        --------------------  -----------------------
    });
    ...
}

Because this is a move future, this takes ownership of cx.io_system and cx_request_name. Because cx is a borrowed reference, this will be an error unless those values are Copy (which they presumably are not). Under this proposal, capturing aliases or clones in a move closure/future would result in capturing an alias or clone of the place. So this future would be desugared like so (using explicit capture clause strawman notation):

fn spawn_services(cx: &Context) {
    tokio::task::spawn(
        async move(cx.io_system.alias(), cx.request_name.clone()) {
            //     --------------------  -----------------------
            //     capture alias/clone respectively

            manage_io(cx.io_system.alias(), cx.request_name.clone());
        }
    );
    ...
}

Part 2: Last-use transformation

Now, this result is inefficient – there are now two aliases/clones. So the next part of the proposal is that the compiler would, in newer Rust editions, apply a new transformat called the last-use transformation. This transformation would identify calls to alias or clone that are not needed to satisfy the borrow checker and remove them. This code would therefore become:

fn spawn_services(cx: &Context) {
    tokio::task::spawn(
        async move(cx.io_system.alias(), cx.request_name.clone()) {
            manage_io(cx.io_system, cx.request_name);
            //        ------------  ---------------
            //        converted to moves
        }
    );
    ...
}

The last-use transformation would apply beyond closures. Given an example like this one, which clones id even though id is never used later:

fn send_process_identifier_request(id: String) {
    let request = Request::ProcessIdentifier(id.clone());
    //                                       ----------
    //                                       unnecessary
    send_request(request)
}

the user would get a warning like so¹:

warning: unnecessary `clone` call will be converted to a move
 --> src/main.rs:7:40
  |
8 |     let request = Request::ProcessIdentifier(id.clone());
  |                                              ^^^^^^^^^^ unnecessary call to `clone`
  |
  = help: the compiler automatically removes calls to `clone` and `alias` when not
    required to satisfy the borrow checker
help: change `id.clone()` to `id` for greater clarity
  |
8 -     let request = Request::ProcessIdentifier(id.clone());
8 +     let request = Request::ProcessIdentifier(id);
  |

and the code would be transformed so that it simply does a move:

fn send_process_identifier_request(id: String) {
    let request = Request::ProcessIdentifier(id);
    //                                       --
    //                                   transformed
    send_request(request)
}

Mental model: just call “clone” (or “alias”)

The goal of this proposal is that, when you get an error about a use of moved value, or moving borrowed content, the fix is always the same: you just call clone (or alias). It doesn’t matter whether that error occurs in the regular function body or in a closure or in a future, the compiler will insert the clones/aliases needed to ensure future users of that same place have access to it (and no more than that).

I believe this will be helpful for new users. Early in their Rust journey new users are often sprinkling calls to clone as well as sigils like & in more-or-less at random as they try to develop a firm mental model – this is where the “keep calm and call clone” joke comes from. This approach breaks down around closures and futures today. Under this proposal, it will work, but users will also benefit from warnings indicating unnecessary clones, which I think will help them to understand where clone is really needed.

Experienced users can trust the compiler to get it right

But the real question is how this works for experienced users. I’ve been thinking about this a lot! I think this approach fits pretty squarely in the classic Bjarne Stroustrup definition of a zero-cost abstraction:

“What you don’t use, you don’t pay for. And further: What you do use, you couldn’t hand code any better.”

The first half is clearly satisfied. If you don’t call clone or alias, this proposal has no impact on your life.

The key point is the second half: earlier versions of this proposal were more simplistic, and would sometimes result in redundant or unnecessary clones and aliases. Upon reflection, I decided that this was a non-starter. The only way this proposal works is if experienced users know there is no performance advantage to using the more explicit form.This is precisely what we have with, say, iterators, and I think it works out very well. I believe this proposal hits that mark, but I’d like to hear if there are things I’m overlooking.

The last-use transformation codifies a widespread intuition, that clone is never necessary

I think most users would expect that changing message.clone() to just message is fine, as long as the code keeps compiling. But in fact nothing requires that to be the case. Under this proposal, APIs that make clone significant in unusual ways would be more annoying to use in the new Rust edition and I expect ultimately wind up getting changed so that “significant clones” have another name. I think this is a good thing.

Frequently asked questions

I think I’ve covered the key points. Let me dive into some of the details here with a FAQ.

Can you summarize all of these posts you’ve been writing? It’s a lot to digest!

I get it, I’ve been throwing a lot of things out there. Let me begin by recapping the motivation as I see it:

• I believe our goal should be to focus first on a design that is “low-level enough for a Kernel, usable enough for a GUI”.
  • The key part here is the word enough. We need to make sure that low-level details are exposed, but only those that truly matter. And we need to make sure that it’s ergonomic to use, but it doesn’t have to be as nice as TypeScript (though that would be great).
• Rust’s current approach to Clone fails both groups of users;
  • calls to clone are not explicit enough for kernels and low-level software: when you see something.clone(), you don’t know that is creating a new alias or an entirely distinct value, and you don’t have any clue what it will cost at runtime. There’s a reason much of the community recommends writing Arc::clone(&something) instead.
  • calls to clone, particularly in closures, are a major ergonomic pain point, this has been a clear consensus since we first started talking about this issue.

I then proposed a set of three changes to address these issues, authored in individual blog posts:

• First, we introduce the Alias trait (originally called Handle). The Alias trait introduces a new method alias that is equivalent to clone but indicates that this will be creating a second alias of the same underlying value.
• Second, we introduce explicit capture clauses, which lighten the syntactic load of capturing a clone or alias, make it possible to declare up-front the full set of values captured by a closure/future, and will support other kinds of handy transformations (e.g., capturing the result of as_ref or to_string).
• Finally, we introduce the just call clone proposal described in this post. This modifies closure desugaring to recognize clones/aliases and also applies the last-use transformation to replace calls to clone/alias with moves where possible.

What would it feel like if we did all those things?

Let’s look at the impact of each set of changes by walking through the “Cloudflare example”, which originated in this excellent blog post by the Dioxus folks:

let some_value = Arc::new(something);

// task 1
let _some_value = some_value.clone();
tokio::task::spawn(async move {
    do_something_with(_some_value);
});

// task 2:  listen for dns connections
let _some_a = self.some_a.clone();
let _some_b = self.some_b.clone();
let _some_c = self.some_c.clone();
tokio::task::spawn(async move {
  	do_something_else_with(_some_a, _some_b, _some_c)
});

As the original blog post put it:

Working on this codebase was demoralizing. We could think of no better way to architect things - we needed listeners for basically everything that filtered their updates based on the state of the app. You could say “lol get gud,” but the engineers on this team were the sharpest people I’ve ever worked with. Cloudflare is all-in on Rust. They’re willing to throw money at codebases like this. Nuclear fusion won’t be solved with Rust if this is how sharing state works.

Applying the Alias trait and explicit capture clauses makes for a modest improvement. You can now clearly see that the calls to clone are alias calls, and you don’t have the awkward _some_value and _some_a variables. However, the code is still pretty verbose:

let some_value = Arc::new(something);

// task 1
tokio::task::spawn(async move(some_value.alias()) {
    do_something_with(some_value);
});

// task 2:  listen for dns connections
tokio::task::spawn(async move(
    self.some_a.alias(),
    self.some_b.alias(),
    self.some_c.alias(),
) {
  	do_something_else_with(self.some_a, self.some_b, self.some_c)
});

Applying the Just Call Clone proposal removes a lot of boilerplate and, I think, captures the intent of the code very well. It also retains quite a bit of explicitness, in that searching for calls to alias reveals all the places that aliases will be created. However, it does introduce a bit of subtlety, since (e.g.) the call to self.some_a.alias() will actually occur when the future is created and not when it is awaited:

let some_value = Arc::new(something);

// task 1
tokio::task::spawn(async move {
    do_something_with(some_value.alias());
});

// task 2:  listen for dns connections
tokio::task::spawn(async move {
  	do_something_else_with(
        self.some_a.alias(),
        self.some_b.alias(),
        self.some_c.alias(),
    )
});

I’m worried that the execution order of calls to alias will be too subtle. How is thie “explicit enough for low-level code”?

There is no question that Just Call Clone makes closure/future desugaring more subtle. Looking at task 1:

tokio::task::spawn(async move {
    do_something_with(some_value.alias());
});

this gets desugared to a call to alias when the future is created (not when it is awaited). Using the explicit form:

tokio::task::spawn(async move(some_value.alias()) {
    do_something_with(some_value)
});

I can definitely imagine people getting confused at first – “but that call to alias looks like its inside the future (or closure), how come it’s occuring earlier?”

Yet, the code really seems to preserve what is most important: when I search the codebase for calls to alias, I will find that an alias is creating for this task. And for the vast majority of real-world examples, the distinction of whether an alias is creating when the task is spawned versus when it executes doesn’t matter. Look at this code: the important thing is that do_something_with is called with an alias of some_value, so some_value will stay alive as long as do_something_else is executing. It doesn’t really matter how the “plumbing” worked.

What about futures that conditionally alias a value?

Yeah, good point, those kind of examples have more room for confusion. Like look at this:

tokio::task::spawn(async move {
    if false {
        do_something_with(some_value.alias());
    }
});

In this example, there is code that uses some_value with an alias, but only under if false. So what happens? I would assume that indeed the future will capture an alias of some_value, in just the same way that this future will move some_value, even though the relevant code is dead:

tokio::task::spawn(async move {
    if false {
        do_something_with(some_value);
    }
});

Can you give more details about the closure desugaring you imagine?

Yep! I am thinking of something like this:

• If there is an explicit capture clause, use that.
• Else:
  • For non-move closures/futures, no changes, so
    • Categorize usage of each place and pick the “weakest option” that is available:
      • by ref
      • by mut ref
      • moves
  • For move closures/futures, we would change
    • Categorize usage of each place P and decide whether to capture that place…
      • by clone, there is at least one call P.clone() or P.alias() and all other usage of P requires only a shared ref (reads)
      • by move, if there are no calls to P.clone() or P.alias() or if there are usages of P that require ownership or a mutable reference
    • Capture by clone/alias when a place a.b.c is only used via shared references, and at least one of those is a clone or alias.
      • For the purposes of this, accessing a “prefix place” a or a “suffix place” a.b.c.d is also considered an access to a.b.c.

Examples that show some edge cased:

if consume {
    x.foo().
}

Why not do something similar for non-move closures?

In the relevant cases, non-move closures will already just capture by shared reference. This means that later attempts to use that variable will generally succeed:

let f = async {
    //  ----- NOT async move
    self.some_a.alias()
};

do_something_else(self.some_a.alias());
//                ----------- later use succeeds

f.await;

This future does not need to take ownership of self.some_a to create an alias, so it will just capture a reference to self.some_a. That means that later uses of self.some_a can still compile, no problem. If this had been a move closure, however, that code above would currently not compile.

There is an edge case where you might get an error, which is when you are moving:

let f = async {
    self.some_a.alias()
};

do_something_else(self.some_a);
//                ----------- move!

f.await;

In that case, you can make this an async move closure and/or use an explicit capture clause:

Can you give more details about the last-use transformation you imagine?

Yep! We would during codegen identify candidate calls to Clone::clone or Alias::alias. After borrow check has executed, we would examine each of the callsites and check the borrow check information to decide:

• Will this place be accessed later?
• Will some reference potentially referencing this place be accessed later?

If the answer to both questions is no, then we will replace the call with a move of the original place.

Here are some examples:

fn borrow(message: Message) -> String {
    let method = message.method.to_string();

    send_message(message.clone());
    //           ---------------
    //           would be transformed to
    //           just `message`

    method
}

fn borrow(message: Message) -> String {
    send_message(message.clone());
    //           ---------------
    //           cannot be transformed
    //           since `message.method` is
    //           referenced later

    message.method.to_string()
}

fn borrow(message: Message) -> String {
    let r = &message;

    send_message(message.clone());
    //           ---------------
    //           cannot be transformed
    //           since `r` may reference
    //           `message` and is used later.

    r.method.to_string()
}

Why are you calling it the last-use transformation and not optimization?

In the past, I’ve talked about the last-use transformation as an optimization – but I’m changing terminology here. This is because, typically, an optimization is supposed to be unobservable to users except through measurements of execution time (or though UB), and that is clearly not the case here. The transformation would be a mechanical transformation performed by the compiler in a deterministic fashion.

Would the transformation “see through” references?

I think yes, but in a limited way. In other words I would expect

Clone::clone(&foo)

and

let p = &foo;
Clone::clone(p)

to be transformed in the same way (replaced with foo), and the same would apply to more levels of intermediate usage. This would kind of “fall out” from the MIR-based optimization technique I imagine. It doesn’t have to be this way, we could be more particular about the syntax that people wrote, but I think that would be surprising.

On the other hand, you could still fool it e.g. like so

fn identity<T>(x: &T) -> &T { x }

identity(&foo).clone()

Would the transformation apply across function boundaries?

The way I imagine it, no. The transformation would be local to a function body. This means that one could write a force_clone method like so that “hides” the clone in a way that it will never be transformed away (this is an important capability for edition transformations!):

fn pipe<Msg: Clone>(message: Msg) -> Msg {
    log(message.clone()); // <-- keep this one
    force_clone(&message)
}

fn force_clone<Msg: Clone>(message: &Msg) -> Msg {
    // Here, the input is `&Msg`, so the clone is necessary
    // to produce a `Msg`.
    message.clone()
}

Won’t the last-use transformation change behavior by making destructors run earlier?

Potentially, yes! Consider this example, written using explicit capture clause notation and written assuming we add an Alias trait:

async fn process_and_stuff(tx: mpsc::Sender<Message>) {
    tokio::spawn({
        async move(tx.alias()) {
            //     ---------- alias here
            process(tx).await
        }
    });

    do_something_unrelated().await;
}

The precise timing when Sender values are dropped can be important – when all senders have dropped, the Receiver will start returning None when you call recv. Before that, it will block waiting for more messages, since those tx handles could still be used.

So, in process_and_stuff, when will the sender aliases be fully dropped? The answer depends on whether we do the last-use transformation or not:

• Without the transformation, there are two aliases: the original tx and the one being held by the future. So the receiver will only start returning None when do_something_unrelated has finished and the task has completed.
• With the transformation, the call to tx.alias() is removed, and so there is only one alias – tx, which is moved into the future, and dropped once the spawned task completes. This could well be earlier than in the previous code, which had to wait until both process_and_stuff and the new task completed.

Most of the time, running destructors earlier is a good thing. That means lower peak memory usage, faster responsiveness. But in extreme cases it could lead to bugs – a typical example is a Mutex<()> where the guard is being used to protect some external resource.

How can we change when code runs? Doesn’t that break stability?

This is what editions are for! We have in fact done a very similar transformation before, in Rust 2021. RFC 2229 changed destructor timing around closures and it was, by and large, a non-event.

The desire for edition compatibility is in fact one of the reasons I want to make this a last-use transformation and not some kind of optimization. There is no UB in any of these examples, it’s just that to understand what Rust code does around clones/aliases is a bit more complex than it used to be, because the compiler will do automatic transformation to those calls. The fact that this transformation is local to a function means we can decide on a call-by-call basis whether it should follow the older edition rules (where it will always occur) or the newer rules (where it may be transformed into a move).

Does that mean that the last-use transformation would change with Polonius or other borrow checker improvements?

In theory, yes, improvements to borrow-checker precision like Polonius could mean that we identify more opportunities to apply the last-use transformation. This is something we can phase in over an edition. It’s a bit of a pain, but I think we can live with it – and I’m unconvinced it will be important in practice. For example, when thinking about the improvements I expect under Polonius, I was not able to come up with a realistic example that would be impacted.

Isn’t it weird to do this after borrow check?

This last-use transformation is guaranteed not to produce code that would fail the borrow check. However, it can affect the correctness of unsafe code:

let p: *const T = &*some_place;

let q: T = some_place.clone();
//         ---------- assuming `some_place` is
//         not used later, becomes a move

unsafe {
    do_something(p);
    //           -
    // This now refers to a stack slot
    // whose value is uninitialized.
}

Note though that, in this case, there would be a lint identifying that the call to some_place.clone() will be transformed to just some_place. We could also detect simple examples like this one and report a stronger deny-by-default lint, as we often do when we see guaranteed UB.

Shouldn’t we use a keyword for this?

When I originally had this idea, I called it “use-use-everywhere” and, instead of writing x.clone() or x.alias(), I imagined writing x.use. This made sense to me because a keyword seemed like a stronger signal that this was impacting closure desugaring. However, I’ve changed my mind for a few reasons.

First, Santiago Pastorino gave strong pushback that x.use was going to be a stumbling block for new learners. They now have to see this keyword and try to understand what it means – in contrast, if they see method calls, they will likely not even notice something strange is going on.

The second reason though was TC who argued, in the lang-team meeting, that all the arguments for why it should be ergonomic to clone a ref-counted value in a closure applied equally well to clone, depending on the needs of your application. I completely agree. As I mentioned earlier, this also [addresses the concern I’ve heard with the Alias trait], which is that there are things you want to ergonomically clone but which don’t correspond to “aliases”. True.

In general I think that clone (and alias) are fundamental enough to how Rust is used that it’s ok to special case them. Perhaps we’ll identify other similar methods in the future, or generalize this mechanism, but for now I think we can focus on these two cases.

What about “deferred ref-counting”?

One point that I’ve raised from time-to-time is that I would like a solution that gives the compiler more room to optimize ref-counting to avoid incrementing ref-counts in cases where it is obvious that those ref-counts are not needed. An example might be a function like this:

fn use_data(rc: Rc<Data>) {
    for datum in rc.iter() {
        println!("{datum:?}");
    }
}

This function requires ownership of an alias to a ref-counted value but it doesn’t actually do anything but read from it. A caller like this one…

use_data(source.alias())

…doesn’t really need to increment the reference count, since the caller will be holding a reference the entire time. I often write code like this using a &:

fn use_data(rc: &Rc<Data>) {
    for datum in rc.iter() {
        println!("{datum:?}");
    }
}

so that the caller can do use_data(&source) – this then allows the callee to write rc.alias() in the case that it wants to take ownership.

I’ve basically decided to punt on adressing this problem. I think folks that are very performance sensitive can use &Arc and the rest of us can sometimes have an extra ref-count increment, but either way, the semantics for users are clear enough and (frankly) good enough.

Handle doesn’t cover everything

The first concern with the Handle trait is that, while it gives a clear semantic basis for when to implement the trait, it does not cover all the cases where calling clone is annoying. In other words, if we opt to use Handle, and then we make creating new handles very ergonomic, but calling clone remains painful, there will be a temptation to use the Handle when it is not appropriate.

In one of our lang team design meetings, TC raised the point that, for many applications, even an “expensive” clone isn’t really a big deal. For example, when writing CLI tools and things, I regularly clone strings and vectors of strings and hashmaps and whatever else; I could put them in an Rc or Arc but I know it just doens’t matter.

My solution here is simple: let’s make solutions that apply to both Clone and Handle. Given that I think we need a proposal that allows for handles that are both ergonomic and explicit, it’s not hard to say that we should extend that solution to include the option for clone.

The explicit capture clause post already fits this design. I explicitly chose a design that allowed for users to write move(a.b.c.clone()) or move(a.b.c.handle()), and hence works equally well (or equally not well…) with both traits

The name Handle doesn’t fit the Rust conventions

A number of people have pointed out Handle doesn’t fit the Rust naming conventions for traits like this, which aim for short verbs. You can interpret handle as a verb, but it doesn’t mean what we want. Fair enough. I like the name Handle because it gives a noun we can use to talk about, well, handles, but I agree that the trait name doesn’t seem right. There was a lot of bikeshedding on possible options but I think I’ve come back to preferring Jack Huey’s original proposal, Share (with a method share). I think Alias and alias is my second favorite. Both of them are short, relatively common verbs.

I originally felt that Share was a bit too generic and overly associated with sharing across threads – but then I at least always call &T a shared reference¹, and an &T would implement Share, so it all seems to work well. Hat tip to Ariel Ben-Yehuda for pushing me on this particular name.

Coming up next

The flurry of posts in this series have been an attempt to survey all the discussions that have taken place in this area. I’m not yet aiming to write a final proposal – I think what will come out of this is a series of multiple RFCs.

My current feeling is that we should add the Hand^H^H^H^H, uh, Share trait. I also think we should add explicit capture clauses. However, while explicit capture clauses are clearly “low-level enough for a kernel”, I don’t really think they are “usable enough for a GUI” . The next post will explore another idea that I think might bring us closer to that ultimate ergonomic and explicit goal.

This post focuses on explicit capture clauses, which would permit closures to be annotated with an explicit set of captured places. My take is that explicit capture clauses are a no brainer, for reasons that I’ll cover below, and we should definitely do them; but they may not be enough to be considered ergonomic, so I’ll explore more proposals afterwards.

Motivation

Rust closures today work quite well but I see a few problems:

• Teaching and understanding closure desugaring is difficult because it lacks an explicit form. Users have to learn to desugar in their heads to understand what’s going on.
• Capturing the “clone” of a value (or possibly other transformations) has no concise syntax.
• For long closure bodies, it is hard to determine precisely which values are captured and how; you have to search the closure body for references to external variables, account for shadowing, etc.
• It is hard to develop an intuition for when move is required. I find myself adding it when the compiler tells me to, but that’s annoying.

Let’s look at a strawperson proposal

Some time ago, I wrote a proposal for explicit capture clauses. I actually see a lot of flaws with this proposal, but I’m still going to explain it: right now it’s the only solid proposal I know of, and it’s good enough to explain how an explicit capture clause could be seen as a solution to the “explicit and ergonomic” goal. I’ll then cover some of the things I like about the proposal and what I don’t.

Begin with move

The proposal begins by extending the move keyword with a list of places to capture:

let closure = move(a.b.c, x.y) || {
    do_something(a.b.c.d, x.y)
};

The closure will then take ownership of those two places; references to those places in the closure body will be replaced by accesses to these captured fields. So that example would desugar to something like

let closure = {
    struct MyClosure {
        a_b_c: Foo,
        x_y: Bar,
    }

    impl FnOnce<()> for MyClosure {
        fn call_once(self) -> Baz {
            do_something(self.a_b_c.d, self.x_y)
            //           ----------    --------
            //   The place `a.b.c` is      |
            //   rewritten to the field    |
            //   `self.a_b_c`              |
            //                  Same here but for `x.y`
        }
    }

    MyClosure {
        a_b_c: self.a.b.c,
        x_y: self.x.y,
    }
};

When using a simple list like this, attempts to reference other places that were not captured result in an error:

let closure = move(a.b.c, x.y) || {
    do_something(a.b.c.d, x.z)
    //           -------  ---
    //           OK       Error: `x.z` not captured
};

Capturing with rewrites

It is also possible to capture a custom expression by using an = sign. So for example, you could rewrite the above closure as follows:

let closure = move(
    a.b.c = a.b.c.clone(),
    x.y,
) || {
    do_something(a.b.c.d, x.z)
};

and it would desugar to:

let closure = {
    struct MyClosure { /* as before */ }
    impl FnOnce<()> for MyClosure { /* as before */ }

    MyClosure {
        a_b_c: self.a.b.c.clone(),
        //     ------------------
        x_y: self.x.y,
    }
};

When using this form, the expression assigned to a.b.c must have the same type as a.b.c in the surrounding scope. So this would be an error:

let closure = move(
    a.b.c = 22, // Error: `i32` is not `Foo`
    x.y,
) || {
    /* ... */
};

Shorthands and capturing by reference

You can understand move(a.b) as sugar for move(a.b = a.b). We support other convenient shorthands too, such as

move(a.b.clone()) || {...}
// == anything that ends in a method call becomes ==>
move(a.b = a.b.clone()) || {...}

and two kinda special shorthands:

move(&a.b) || { ... }
move(&mut a.b) || { ... }

These are special because the captured value is indeed &a.b and &mut a.b – but that by itself wouldn’t work, because the type doesn’t match. So we rewrite each access to a.b to desugar to a dereference of the a_b field, like *self.a_b:

move(&a.b) || { foo(a.b) }

// desugars to

struct MyStruct<'l> {
    a_b: &'l Foo
}

impl FnOnce for MyStruct<'_> {
    fn call_once(self) {
        foo(*self.a_b)
        //  ---------
        //  we insert the `*` too
    }
}

MyStruct {
    a_b: &a.b,
}

move(&a.b) || { foo(*a.b) }

There’s a lot of precedence for this sort of transform: it’s precisely what we do for the Deref trait and for existing closure captures.

Fresh variables

We should also allow you to define fresh variables. These can have arbitrary types. The values are evaluated at closure creation time and stored in the closure metadata:

move(
    data = load_data(),
    y,
) || {
    take(&data, y)
}

Open-ended captures

All of our examples so far fully enumerated the captured variables. But Rust closures today infer the set of captures (and the style of capture) based on the paths that are used. We should permit that as well. I’d permit that with a .. sugar, so these two closures are equivalent:

let c2 = move || /* closure */;
//       ---- capture anything that is used,
//            taking ownership

let c1 = move(..) || /* closure */;
//           ---- capture anything else that is used,
//                taking ownership

Of course you can combine:

let c = move(x.y.clone(), ..) || {

};

And you could write ref to get the equivalent of || closures:

let c2 = || /* closure */;
//       -- capture anything that is used,
//          using references if possible
let c1 = move(ref) || /* closure */;
//            --- capture anything else that is used,
//                using references if possible

This lets you

let c = move(
    a.b.clone(), 
    c,
    ref
) || {
    combine(&a.b, &c, &z)
    //       ---   -   -
    //        |    |   |
    //        |    | This will be captured by reference
    //        |    | since it is used by reference
    //        |    | and is not explicitly named.
    //        |    |
    //        |   This will be captured by value
    //        |   since it is explicitly named.
    //        |
    // We will capture a clone of this because
    // the user wrote `a.b.clone()`
}

Frequently asked questions

How does this help with our motivation?

Let’s look at the motivations I named:

Teaching and understanding closure desugaring is difficult

There’s a lot of syntax there, but it also gives you an explicit form that you can use to do explanations. To see what I mean, consider the difference between these two closures (playground).

The first closure uses ||:

fn main() {
    let mut i = 3;
    let mut c_attached = || {
        let j = i + 1;
        std::mem::replace(&mut i, j)
    };
    ...
}

While the second closure uses move:

fn main() {
    let mut i = 3;
    let mut c_detached = move || {
        let j = i + 1;
        std::mem::replace(&mut i, j)
    };

These are in fact pretty different, as you can see in this playground. But why? Well, the first closure desugars to capture a reference:

let mut i = 3;
let mut c_attached = move(&i) || {...};

and the second captures by value:

let mut i = 3;
let mut c_attached = move(i) || {...};

Before, to explain that, I had to resort to desugaring to structs.

Capturing a clone is painful

If you have a closure that wants to capture the clone of something today, you have to introduce a fresh variable. So something like this:

let closure = move || {
    begin_actor(data, self.tx.clone())
};

becomes

let closure = {
    let self_tx = self.tx.clone();
    move || {
        begin_actor(data, self_tx.clone())
    }
};

This is awkward. Under this proposal, it’s possible to point-wise replace specific items:

let closure = move(self.tx.clone(), ..) || {
    begin_actor(data, self.tx.clone())
};

For long closure bodies, it is hard to determine precisely which values are captured and how

Quick! What variables does this closure use from the environment?

.flat_map(move |(severity, lints)| {
    parse_tt_as_comma_sep_paths(lints, edition)
    .into_iter()
    .flat_map(move |lints| {
        // Rejoin the idents with `::`, so we have no spaces in between.
        lints.into_iter().map(move |lint| {
            (
                lint.segments().filter_map(
                    |segment| segment.name_ref()
                ).join("::").into(),
                severity,
            )
        })
    })
})

No idea? Me either. What about this one?

.flat_map(move(edition) |(severity, lints)| {
    /* same as above */
})

Ah, pretty clear! I find that once a closure moves beyond a couple of lines, it can make a function kind of hard to read, because it’s hard to tell what variables it may be accessing. I’ve had functions where it’s important to correctness for one reason or another that a particular closure only accesses a subset of the values around it, but I have no way to indicate that right now. Sometimes I make separate functions, but it’d be nicer if I could annotate the closure’s captures explicitly.

It is hard to develop an intuition for when move is required

Hmm, actually, I don’t think this notation helps with that at all! More about this below.

Let me cover some of the questions you may have about this design.

Why allow the “capture clause” to specify an entire place, like a.b.c?

Today you can write closures that capture places, like self.context below:

let closure = move || {
    send_data(self.context, self.other_field)
};

My goal was to be able to take such a closure and to add annotations that change how particular places are captured, without having to do deep rewrites in the body:

let closure = move(self.context.clone(), ..) || {
    //            --------------------------
    //            the only change
    send_data(self.context, self.other_field)
};

This definitely adds some complexity, because it means we have to be able to “remap” a place like a.b.c that has multiple parts. But it makes the explicit capture syntax far more powerful and convenient.

Why do you keep the type the same for places like a.b.c?

I want to ensure that the type of a.b.c is the same wherever it is type-checked, it’ll simplify the compiler somewhat and just generally makes it easier to move code into and out of a closure.

Why the move keyword?

Because it’s there? To be honest, I don’t like the choice of move because it’s so operational. I think if I could go back, I would try to refashion our closures around two concepts

• Attached closures (what we now call ||) would always be tied to the enclosing stack frame. They’d always have a lifetime even if they don’t capture anything.
• Detached closures (what we now call move ||) would capture by-value, like move today.

I think this would help to build up the intuition of “use detach || if you are going to return the closure from the current stack frame and use || otherwise”.

What would a max-min explicit capture proposal look like?

A maximally minimal explicit capture close proposal would probably just let you name specific variables and not “subplaces”:

move(
    a_b_c = a.b.c,
    x_y = &x.y
) || {
    *x_y + a_b_c
}

I think you can see though that this makes introducing an explicit form a lot less pleasant to use and hence isn’t really going to do anything to support ergonomic RC.

Conclusion: Explicit closure clauses make things better, but not great

I think doing explicit capture clauses is a good idea – I generally think we should have explicit syntax for everything in Rust, for teaching and explanatory purposes if nothing else; I didn’t always think this way, but it’s something I’ve come to appreciate over time.

I’m not sold on this specific proposal – but I think working through it is useful, because it (a) gives you an idea of what the benefits would be and (b) gives you an idea of how much hidden complexity there is.

I think the proposal shows that adding explicit capture clauses goes some way towards making things explicit and ergonomic. Writing move(a.b.c.clone()) is definitely better than having to create a new binding.

But for me, it’s not really nice enough. It’s still quite a mental distraction to have to find the start of the closure, insert the a.b.c.clone() call, and it makes the closure header very long and unwieldy. Particularly for short closures the overhead is very high.

This is why I’d like to look into other options. Nonetheless, it’s useful to have discussed a proposal for an explicit form: if nothing else, it’ll be useful to explain the precise semantics of other proposals later on.

Nothing this good comes for free. The big catch of the proposal is that it introduces more “core splits” into Rust’s types. I believe these splits are well motivated and reasonable – they reflect inherent complexity, in other words, but they are something we’ll want to think carefully about nonetheless.

Summary

The TL;DR of the proposal is that we should:

• Introduce a new “default trait bound” Forget and an associated trait hierarchy:
  • trait Forget: Drop, representing values that can be forgotten
  • trait Destruct: Move, representing values with a destructor
  • trait Move: Pointee, representing values that can be moved
  • trait Pointee, the base trait that represents any value
• Use the “opt-in to weaker defaults” scheme proposed for sizedness by RFC #3729 (Hierarchy of Sized Traits)
  • So fn foo<T>(t: T) defaults to “a T that can be forgotten/destructed/moved”
  • And fn foo<T: Destruct>(t: T) means “a T that can be destructed, but not necessarily forgotten”
  • And fn foo<T: Move>(t: T) means “a T that can be moved, but not necessarily forgotten”
  • …and so forth.
• Integrate and enforce the new traits:
  • The bound on std::mem::forget will already require Forget, so that’s good.
  • Borrow check can enforce that any dropped value must implement Destruct; in fact, we already do this to enforce const Destruct bounds in const fn.
  • Borrow check can be extended to require a Move bound on any moved value.
• Adjust the trait bound on closures (luckily this works out fairly nicely)

Motivation

In a talk I gave some years back at Rust LATAM in Uruguay¹, I said this:

• It’s easy to expose a high-performance API.
• But it’s hard to help users control it – and this is what Rust’s type system does.

Rust currently does a pretty good job with preventing parts of your program from interfering with one another, but we don’t do as good a job when it comes to guaranteeing that cleaup happens². We have destructors, of course, but they have two critical limitations:

• All destructors must meet the same signature, fn drop(&mut self), which isn’t always adequate.
• There is no way to guarantee a destructor once you give up ownership of a value.

Making it concrete.

That motivation was fairly abstract, so let me give some concrete examples of things that tie back to this limitation:

• The ability to have async or const drop, both of which require a distinct drop signature.
• The ability to have a “drop” operation that takes arguments, such as e.g. a message that must be sent, or a result code that must be provided before the program terminates.
• The ability to have async scopes that can access the stack, which requires a way to guarantee that a parallel thread will be joined even in an async context.
• The ability to integrate at maximum efficiency with WebAssembly async tasks, which require guaranteed cleanup.³

The goal of this post is to outline an approach that could solve all of the above problems and which is backwards compatible with Rust today.

The “capabilities” of value disposal

The core problem is that Rust today assumes that every Sized value can be moved, dropped, and forgotten:

// Without knowing anything about `T` apart
// from the fact that it's `Sized`, we can...
fn demonstration<T>(a: T, b: T, c: T) {
    // ...drop `a`, running its destructor immediately.
    std::mem::drop(a);

    // ...forget `b`, skipping its destructor
    std::mem::forget(b);

    // ...move `c` into `x`
    let x = c;
} // ...and then have `x` get dropped automatically,
// as exit the block.

Destructors are like “opt-out methods”

The way I see, most methods are “opt-in” – they don’t execute unless you call them. But destructors are different. They are effectively a method that runs by default – unless you opt-out, e.g., by calling forget. But the ability to opt-out means that they don’t fundamentally add any power over regular methods, they just make for a more ergonomic API.

The implication is that the only way in Rust today to guarantee that a destructor will run is to retain ownership of the value. This can be important to unsafe code – APIs that permit scoped threads, for example, need to guarantee that those parallel threads will be joined before the function returns. The only way they have to do that is to use a closure which gives &-borrowed access to a scope:

scope(|s| ...)
//     -  --- ...which ensures that this
//     |      fn body cannot "forget" it.
//     |  
// This value has type `&Scope`... 

Because the API nevers gives up ownership of the scope, it can ensure that it is never “forgotten” and thus that its destructor runs.

The scoped thread approach works for sync code, but it doesn’t work for async code. The problem is that async functions return a future, which is a value. Users can therefore decide to “forget” this value, just like any other value, and thus the destructor may never run.

Guaranteed cleanup is common in systems programming

When you start poking around, you find that guaranteed destructors turn up quite a bit in systems programming. Scoped APIs in futures are one example, but DMA (direct memory access) is another. Many embedded devices have a mode where you begin a DMA transfer that causes memory to be written into memory asynchronously. But you need to ensure that this DMA is terminated before that memory is freed. If that memory is on your stack, that means you need a destructor that will either cancel or block until the DMA finishes.⁴

So what can we do about it?

This situation is very analogous to the challenge of revisiting the default Sized bound, and I think the same basic approach that I outlined in [this blog post][sized] will work.

The core of the idea is simple: have a “special” set of traits arranged in a hierarchy:

trait Forget: Destruct {} // Can be "forgotten"
trait Destruct: Move {}   // Can be "destructed" (dropped)
trait Move: Pointee {}    // Can be "moved"
trait Pointee {}          // Can be referenced by pointer

By default, generic parameters get a Forget bound, so fn foo<T>() is equivalent to fn foo<T: Forget>(). But if the parameter opts in to a weaker bound, then the default is suppressed, so fn bar<T: Destruct>() means that T is assumed by “destructible” but not forgettable. And fn baz<T: Move>() indicates that T can only be moved.

Impact of these bounds

Let me explain briefly how these bounds would work.

The default can forget, drop, move etc

Given a default type T, or one that writes Forget explicitly, the function can do anything that is possible today:

fn just_forget<T: Forget>(a: T, b: T, c: T) {
    //         --------- this bound is the default
    std::mem::drop(a);   // OK
    std::mem::forget(b); // OK
    let x = c;           // OK
}

The forget function requires T: Forget

The std::mem::forget function would require T: Forget as well:

pub fn forget<T: Forget>(value: T) { /* magic intrinsic */ }

This means that if you have only Destruct, the function can only drop or move, it can’t “forget”:

fn just_destruct<T: Destruct>(a: T, b: T, c: T) {
    //           -----------
    // This function only requests "Destruct" capability.

    std::mem::drop(a);   // OK
    std::mem::forget(b); // ERROR: `T: Forget` required
    let x = c;           // OK
}

The borrow checker would require “dropped” values implement Destruct

We would modify the drop function to require only T: Destruct:

fn drop<T: Destruct>(t: T) {}

We would also extend the borrow checker so that when it sees a value being dropped (i.e., because it went out of scope), it would require the Destruct bound.

That means that if you have a value whose type is only Move, you cannot “drop” it:

fn just_move<T: Move>(a: T, b: T, c: T) {
    //           -----------
    // This function only requests "Move" capability.

    std::mem::drop(a);   // ERROR: `T: Destruct` required
    std::mem::forget(b); // ERROR: `T: Forget` required
    let x = c;           // OK
}                        // ERROR: `x` is being dropped, but `T: Destruct`

This means that if you have only a Move bound, you must move anything you own if you want to return from the function. For example:

fn return_ok<T: Move>(a: T) -> T {
    a // OK
}

If you have a function that does not move, you’ll get an error:

fn return_err<T: Move>(a: T) -> T {
} // ERROR: `a` does not implement `Destruct`

It’s worth pointing out that this will be annoying as all get out in the face of panics:

fn return_err<T: Move>(a: T) -> T {
    // ERROR: If a panic occurs, `a` would be dropped, but `T` not implement `Destruct`
    forbid_env_var();

    a
} 

fn forbid_env_var() {
    if std::env::var("BAD").is_ok() {
        panic!("Uh oh: BAD cannot be set");
    }
}

I’m ok with this, but it is going to put pressure on better ways to rule out panics statically.

Const (and later async) variants of Destruct

In fact, we are already doing something much like this destruct check for const functions. Right now if you have a const fn and you try to drop a value, you get an error:

const fn test<T>(t: T) {
} // ERROR!

Compiling that gives you the error:

error[E0493]: destructor of `T` cannot be evaluated at compile-time
 --> src/lib.rs:1:18
  |
1 | const fn test<T>(t: T) { }
  |                  ^       - value is dropped here
  |                  |
  |                  the destructor for this type cannot be evaluated in constant functions

This check is not presently taking place in borrow check but it could be.

The borrow checker would require “moved” values implement Move

The final part of the check would be requiring that “moved” values implement Move:

fn return_err<T: Pointee>(a: T) -> T {
    a // ERROR: `a` does not implement `Move`
}

You might think that having types that are !Move would replace the need for pin, but this is not the case. A pinned value is one that can never move again, whereas a value that is not Move can never be moved in the first place – at least once it is stored into a place.

I’m not sure if this part of the proposal makes sense, we could start by just having all types be Move, Destruct, or (the default) Forget.

Opting out from forget etc

The other part of the proposal is that you should be able to explicit “opt out” from being forgettable, e.g. by doing

struct MyType {}
impl Destruct for MyType {}

Doing this will limit the generics that can accept your type, of course.

Associated type bounds

The tough part with these “default bound” proposals is always associated type bounds. For backwards compatibility, we’d have to default to Forget but a lot of associated types that exist in the wild today shouldn’t really require Forget. For example a trait like Add should really just require Move for its return type:

trait Add<Rhs = Self> {
    type Output /* : Move */;
}

I am basically not too worried about this. It’s possible that we can weaken these bounds over time or through editions. Or, perhaps, add in some kind of edition-specific “alias” like

trait Add2025<Rhs = Self> {
    type Output: Move;
}

where Add2025 is implemented for everything that implements Add.

I am not sure exactly how to manage it, but we’ll figure it out – and in the meantime, most of the types that should not be forgettable are really just “guard” types that don’t have to flow through quite so many places.

Associated type bounds in closures

The one place that I think it is really imporatnt that we weaken the associated type bounds is with closures– and, fortunately, that’s a place we can get away with due to the way our “closure trait bound” syntax works. I feel like I wrote a post on this before, but I can’t find it now, but the short version is that, today, when you write F: Fn(), that means that the closure must return (). If you write F: Fn() -> T, then this type T must have been declared somewhere else, and so T will (independently from the associated type of the Fn trait) get a default Forget bound. So since the Fn associated type is not independently nameable in stable Rust, we can change its bounds, and code like this would continue to work unchanged:

fn foo<T, F>()
where
    F: Fn() -> T,
    //         - `T: Forget` still holds by default
{}

Frequently asked questions

How does this relate to the recent thread on internals?

Recently I was pointed at this internals thread for a “substructural type system” which likely has very similar capabilities. To be totally honest, though, I haven’t had time to read and digest it yet! I had this blog post like 95% done though so I figured I’d post it first and then go try and compare.

What would it mean for a struct to opt out of Move (e.g., by being only Pointee)?

So, the system as I described would allow for ‘unmoveable’ types (i.e., a struct that opts out from everything and only permits Pointee), but such a struct would only really be something you could store in a static memory location. You couldn’t put it on the stack because the stack must eventually get popped. And you couldn’t move it from place to place because, well, it’s immobile.

This seems like something that could be useful – e.g., to model “video RAM” or something that lives in a specific location in memory and cannot live anywhere else – but it’s not a widespread need.

How would you handle destructors with arguments?

I imagine something like this:

struct Transaction {
    data: Vec<u8>
}

/// Opt out from destruct
impl Move for Transaction { }

impl Transaction {
    // This is effectively a "destructor"
    pub fn complete(
        self, 
        connection: Connection,
    ) {
        let Transaction { data } = self;
    }
}

With this setup, any function that owns a Transaction must eventually invoke transaction.complete(). This is because no values of this type can be dropped, so they must be moved.

How does this relate to async drop?

This setup provides attacks a key problem that has blocked async drop in my mind, which is that types that are “async drop” do not have to implement “sync drop”. This gives the type system the ability to prevent them from being dropped in sync code, then, and it would mean that they can only be dropped in async drop. But there’s still lots of design work to be done there.

Why is the trait Destruct and not Drop?

This comes from the const generifs work. I don’t love it. But there is a logic to it. Right now, when you drop a struct or other value, that actually does a whole sequence of things, only one of which is running any Drop impl – it also (for example) drops all the fields in the struct recursively, etc. The idea is that “destruct” refers to this whole sequence.

How hard would this to be to prototype?

I…don’t actually think it would be very hard. I’ve thought somewhat about it and all of the changes seem pretty straightforward. I would be keen to support a lang-team experiment on this.

Does this mean we should have had leak?

The whole topic of destructors and leaks and so forth datesback to approximately Rust 1.0, when we discovered that, in fact, our abstraction for threads was unsound when combined with cyclic ref-counted boxes. Before that we hadn’t fully internalized that destructors are “opt-out methods”. You can read this blog post I wrote at the time. At the time, the primary idea was to have some kind of ?Leak bounds and it was tied to the idea of references (so that all 'static data was assumed to be “leakable”, and hence something you could put into an Rc). I… mostly think we made the right call at the time. I think it’s good that most of the ecosystem is interoperable and that Rc doesn’t require static bounds, and certainly I think it’s good that we moved to 1.0 with minimal disruption. In any case, though, I rather prefer this design to the ones that were under discussion at the time, in part because it also addresses the need for different kinds of destructors and for destructors with many arguments and so forth, which wasn’t something we thought about then.

Isn’t it confusing to have these “magic” traits that “opt out” from default bounds?

I think that specifying the bounds you want is inherently better than today’s ? design, both because it’s easier to understand and because it allows us to backwards compatibly add traits in between in ways that are not possible with the ? design.

However, I do see that having T: Move mean that T: Destruct does not hold is subtle. I wonder if we should adopt some kind of sigil or convention on these traits, like T: @Move or something. I don’t know! Something to consider.

This blog argues that we should make it ergonomic to be explicit. This wasn’t always my position, but after an impactful conversation with Josh Triplett, I’ve come around. I think it aligns with what I once called the soul of Rust: we want to be ergonomic, yes, but we want to be ergonomic while giving control¹.

I like Tyler Mandry’s Clarity of purpose contruction, “Great code brings only the important characteristics of your application to your attention”. The key point is that there is great code in which cloning and handles are important characteristics, so we need to make that code possible to express nicely. This is particularly true since Rust is one of the very few languages that really targets that kind of low-level, foundational code.

This does not mean we cannot (later) support automatic clones and handles. It’s inarguable that this would benefit clarity of purpose for a lot of Rust code. But I think we should focus first on the harder case, the case where explicitness is needed, and get that as nice as we can; then we can circle back and decide whether to also support something automatic. One of the questions for me, in fact, is whether we can get “fully explicit” to be nice enough that we don’t really need the automatic version. There are benefits from having “one Rust”, where all code follows roughly the same patterns, where those patterns are perfect some of the time, and don’t suck too bad² when they’re overkill.

“Rust should not surprise you.” (hat tip: Josh Triplett)

I mentioned this blog post resulted from a long conversation with Josh Triplett³. The key phrase that stuck with me from that conversation was: Rust should not surprise you. The way I think of it is like this. Every programmer knows what its like to have a marathon debugging session – to sit and state at code for days and think, but… how is this even POSSIBLE? Those kind of bug hunts can end in a few different ways. Occasionally you uncover a deeply satisfying, subtle bug in your logic. More often, you find that you wrote if foo and not if !foo. And occasionally you find out that your language was doing something that you didn’t expect. That some simple-looking code concealed a subltle, complex interaction. People often call this kind of a footgun.

Overall, Rust is remarkably good at avoiding footguns⁴. And part of how we’ve achieved that is by making sure that things you might need to know are visible – like, explicit in the source. Every time you see a Rust match, you don’t have to ask yourself “what cases might be missing here” – the compiler guarantees you they are all there. And when you see a call to a Rust function, you don’t have to ask yourself if it is fallible – you’ll see a ? if it is.⁵

Creating a handle can definitely “surprise” you

So I guess the question is: would you ever have to know about a ref-count increment? The trick part is that the answer here is application dependent. For some low-level applications, definitely yes: an atomic reference count is a measurable cost. To be honest, I would wager that the set of applications where this is true are vanishingly small. And even in those applications, Rust already improves on the state of the art by giving you the ability to choose between Rc and Arc and then proving that you don’t mess it up.

But there are other reasons you might want to track reference counts, and those are less easy to dismiss. One of them is memory leaks. Rust, unlike GC’d languages, has deterministic destruction. This is cool, because it means that you can leverage destructors to manage all kinds of resources, as Yehuda wrote about long ago in his classic ode-to-RAII entitled “Rust means never having to close a socket”. But although the points where handles are created and destroyed is deterministic, the nature of reference-counting can make it much harder to predict when the underlying resource will actually get freed. And if those increments are not visible in your code, it is that much harder to track them down.

Just recently, I was debugging Symposium, which is written in Swift. Somehow I had two IPCManager instances when I only expected one, and each of them was responding to every IPC message, wreaking havoc. Poking around I found stray references floating around in some surprising places, which was causing the problem. Would this bug have still occurred if I had to write .handle() explicitly to increment the ref count? Definitely, yes. Would it have been easier to find after the fact? Also yes.⁶

Josh gave me a similar example from the “bytes” crate. A Bytes type is a handle to a slice of some underlying memory buffer. When you clone that handle, it will keep the entire backing buffer around. Sometimes you might prefer to copy your slice out into a separate buffer so that the underlying buffer can be freed. It’s not that hard for me to imagine trying to hunt down an errant handle that is keeping some large buffer alive and being very frustrated that I can’t see explicitly in the where those handles are created.

A similar case occurs with APIs like like Arc::get_mut⁷. get_mut takes an &mut Arc<T> and, if the ref-count is 1, returns an &mut T. This lets you take a shareable handle that you know is not actually being shared and recover uniqueness. This kind of API is not frequently used – but when you need it, it’s so nice it’s there.

“What I love about Rust is its versatility: low to high in one language” (hat tip: Alex Crichton)

Entering the conversation with Josh, I was leaning towards a design where you had some form of automated cloning of handles and an allow-by-default lint that would let crates which don’t want that turn it off. But Josh convinced me that there is a significant class of applications that want handle creation to be ergonomic AND visible (i.e., explicit in the source). Low-level network services and even things like Rust For Linux likely fit this description, but any Rust application that uses get_mut or make_mut might also.

And this reminded me of something Alex Crichton once said to me. Unlike the other quotes here, it wasn’t in the context of ergonomic ref-counting, but rather when I was working on my first attempt at the “Rustacean Principles”. Alex was saying that he loved how Rust was great for low-level code but also worked well high-level stuff like CLI tools and simple scripts.

I feel like you can interpret Alex’s quote in two ways, depending on what you choose to emphasize. You could hear it as, “It’s important that Rust is good for high-level use cases”. That is true, and it is what leads us to ask whether we should even make handles visible at all.

But you can also read Alex’s quote as, “It’s important that there’s one language that works well enough for both” – and I think that’s true too. The “true Rust gestalt” is when we manage to simultaneously give you the low-level control that grungy code needs but wrapped in a high-level package. This is the promise of zero-cost abstractions, of course, and Rust (in its best moments) delivers.

The “soul of Rust”: low-level enough for a kernel, usable enough for a GUI

Let’s be honest. High-level GUI programming is not Rust’s bread-and-butter, and it never will be; users will never confuse Rust for TypeScript. But then, TypeScript will never be in the Linux kernel.

The goal of Rust is to be a single language that can, by and large, be “good enough” for both extremes. The goal is make enough low-level details visible for kernel hackers but do so in a way that is usable enough for a GUI. It ain’t easy, but it’s the job.

This isn’t the first time that Josh has pulled me back to this realization. The last time was in the context of async fn in dyn traits, and it led to a blog post talking about the “soul of Rust” and a followup going into greater detail. I think the catchphrase “low-level enough for a Kernel, usable enough for a GUI” kind of captures it.

Conclusion: Explicit handles should be the first step, but it doesn’t have to be the final step

There is a slight caveat I want to add. I think another part of Rust’s soul is preferring nuance to artificial simplicity (“as simple as possible, but no simpler”, as they say). And I think the reality is that there’s a huge set of applications that make new handles left-and-right (particularly but not exclusively in async land⁸) and where explicitly creating new handles is noise, not signal. This is why e.g. Swift⁹ makes ref-count increments invisible – and they get a big lift out of that!¹⁰ I’d wager most Swift users don’t even realize that Swift is not garbage-collected¹¹.

But the key thing here is that even if we do add some way to make handle creation automatic, we ALSO want a mode where it is explicit and visible. So we might as well do that one first.

OK, I think I’ve made this point 3 ways from Sunday now, so I’ll stop. The next few blog posts in the series will dive into (at least) two options for how we might make handle creation and closures more ergonomic while retaining explicitness.

This is pretty different from how AI tools work today, where everything is a monolith – if you want to change one piece, you’re stuck rebuilding the whole thing from scratch. SymmACP allows you to build out new features and modes of interactions in a layered, interoperable way. This post explains how SymmACP would work by walking through a series of examples.

Right now, SymmACP is just a thought experiment. I’ve sketched these ideas to the Zed folks, and they seemed interested, but we still have to discuss the details in this post. My plan is to start prototyping in Symposium – if you think the ideas I’m discussing here are exciting, please join the Symposium Zulip and let’s talk!

“Composable agents” let you build features independently and then combine them

I’m going to explain the idea of “composable agents” by walking through a series of features. We’ll start with a basic CLI agent¹ tool – basically a chat loop with access to some MCP servers so that it can read/write files and execute bash commands. Then we’ll show how you could add several features on top:

1. Addressing time-blindness by helping the agent know what time it is.
2. Injecting context and “personality” to the agent.
3. Spawning long-running, asynchronous tasks.
4. A copy of Q CLI’s /tangent mode that lets you do a bit of “off the books” work that gets removed from your history later.
5. Implementing Symposium’s interactive walkthroughs, which give the agent a richer vocabulary for communicating with you than just text.
6. Smarter tool delegation.

The magic trick is that each of these features will be developed as separate repositories. What’s more, they could be applied to any base tool you want, so long as it speaks SymmACP. And you could also combine them with different front-ends, such as a TUI, a web front-end, builtin support from Zed or IntelliJ, etc. Pretty neat.

My hope is that if we can centralize on SymmACP, or something like it, then we could move from everybody developing their own bespoke tools to an interoperable ecosystem of ideas that can build off of one another.

let mut SymmACP = ACP

SymmACP begins with ACP, so let’s explain what ACP is. ACP is a wonderfully simple protocol that lets you abstract over CLI agents. Imagine if you were using an agentic CLI tool except that, instead of communication over the terminal, the CLI tool communicates with a front-end over JSON-RPC messages, currently sent via stdin/stdout.

flowchart LR
    Editor <-.->|JSON-RPC via stdin/stdout| Agent[CLI Agent]
  

When you type something into the GUI, the editor sends a JSON-RPC message to the agent with what you typed. The agent responds with a stream of messages containing text and images. If the agent decides to invoke a tool, it can request permission by sending a JSON-RPC message back to the editor. And when the agent has completed, it responds to the editor with an “end turn” message that says “I’m ready for you to type something else now”.

sequenceDiagram
    participant E as Editor
    participant A as Agent
    participant T as Tool (MCP)
    
    E->>A: prompt("Help me debug this code")
    A->>E: request_permission("Read file main.rs")
    E->>A: permission_granted
    A->>T: read_file("main.rs")
    T->>A: file_contents
    A->>E: text_chunk("I can see the issue...")
    A->>E: text_chunk("The problem is on line 42...")
    A->>E: end_turn
  

Telling the agent what time it is

OK, let’s tackle our first feature. If you’ve used a CLI agent, you may have noticed that they don’t know what time it is – or even what year it is. This may sound trivial, but it can lead to some real mistakes. For example, they may not realize that some information is outdated. Or when they do web searches for information, they can search for the wrong thing: I’ve seen CLI agents search the web for “API updates in 2024” for example, even though it is 2025.

To fix this, many CLI agents will inject some extra text along with your prompt, something like <current-date date="2025-10-08" time="HH:MM:SS"/>. This gives the LLM the context it needs.

So how could use ACP to build that? The idea is to create a proxy. This proxy would wrap the original ACP server:

flowchart LR
    Editor[Editor/VSCode] <-->|ACP| Proxy[Datetime Proxy] <-->|ACP| Agent[CLI Agent]
  

This proxy will take every “prompt” message it receives and decorate it with the date and time:

sequenceDiagram
    participant E as Editor
    participant P as Proxy
    participant A as Agent
    
    E->>P: prompt("What day is it?")
    P->>A: prompt("<current-date .../> What day is it?")
    A->>P: text_chunk("It is 2025-10-08.")
    P->>E: text_chunk("It is 2025-10-08.")
    A->>P: end_turn
    P->>E: end_turn
  

Simple, right? And of course this can be used with any editor and any ACP-speaking tool.

Next feature: Injecting “personality” to the agent

Let’s look at another feature that basically “falls out” from ACP: injecting personality. Most agents give you the ability to configure “context” in various ways – or what Claude Code calls memory. This is useful, but I and others have noticed that if what you want is to change how Claude “behaves” – i.e., to make it more collaborative – it’s not really enough. You really need to kick off the conversation by reinforcing that pattern.

In Symposium, the “yiasou” prompt (also available as “hi”, for those of you who don’t speak Greek 😛) is meant to be run as the first thing in the conversation. But there’s nothing an MCP server can do to ensure that the user kicks off the conversation with /symposium:hi or something similar. Of course, if Symposium were implemented as an ACP Server, we absolutely could do that:

sequenceDiagram
    participant E as Editor
    participant P as Proxy
    participant A as Agent
    
    E->>P: prompt("I'd like to work on my document")
    P->>A: prompt("/symposium:hi")
    A->>P: end_turn
    P->>A: prompt("I'd like to work on my document")
    A->>P: text_chunk("Sure! What document is that?") 
    P->>E: text_chunk("Sure! What document is that?") 
    A->>P: end_turn
    P->>E: end_turn
  

Proxies are a better version of hooks

Some of you may be saying, “hmm, isn’t that what hooks are for?” And yes, you could do this with hooks, but there’s two problems with that. First, hooks are non-standard, so you have to do it differently for every agent.

The second problem with hooks is that they’re fundamentally limited to what the hook designer envisioned you might want. You only get hooks at the places in the workflow that the tool gives you, and you can only control what the tool lets you control. The next feature starts to show what I mean: as far as I know, it cannot readily be implemented with hooks the way I would want it to work.

Next feature: long-running, asynchronous tasks

Let’s move on to our next feature, long-running asynchronous tasks. This feature is going to have to go beyond the current capabilities of ACP into the expanded “SymmACP” feature set.

Right now, when the server invokes an MCP tool, it executes in a blocking way. But sometimes the task it is performing might be long and complicated. What you would really like is a way to “start” the task and then go back to working. When the task is complete, you (and the agent) could be notified.

This comes up for me a lot with “deep research”. A big part of my workflow is that, when I get stuck on something I don’t understand, I deploy a research agent to scour the web for information. Usually what I will do is ask the agent I’m collaborating with to prepare a research prompt summarizing the things we tried, what obstacles we hit, and other details that seem relevant. Then I’ll pop over to claude.ai or Gemini Deep Research and paste in the prompt. This will run for 5-10 minutes and generate a markdown report in response. I’ll download that and give it to my agent. Very often this lets us solve the problem.²

This research flow works well but it is tedious and requires me to copy-and-paste. What I would ideally want is an MCP tool that does the search for me and, when the results are done, hands them off to the agent so it can start processing immediately. But in the meantime, I’d like to be able to continue working with the agent while we wait. Unfortunately, the protocol for tools provides no mechanism for asynchronous notifications like this, from what I can tell.

SymmACP += tool invocations + unprompted sends

So how would I do it with SymmACP? Well, I would want to extend the ACP protocol as it is today in two ways:

1. I’d like the ACP proxy to be able to provide tools that the proxy will execute. Today, the agent is responsible for executing all tools; the ACP protocol only comes into play when requesting permission. But it’d be trivial to have MCP tools where, to execute the tool, the agent sends back a message over ACP instead.
2. I’d like to have a way for the agent to initiate responses to the editor. Right now, the editor always initiatives each communication session with a prompt; but, in this case, the agent might want to send messages back unprompted.

In that case, we could implement our Research Proxy like so:

sequenceDiagram
    participant E as Editor
    participant P as Proxy
    participant A as Agent
    
    E->>P: prompt("Why is Rust so great?")
    P->>A: prompt("Why is Rust so great?")
    A->>P: invoke tool("begin_research")
    activate P
    P->>A: ok
    A->>P: "I'm looking into it!"
    P->>E: "I'm looking into it!"
    A->>P: end_turn
    P->>E: end_turn

    Note over E,A: Time passes (5-10 minutes) and the user keeps working...
    Note over P: Research completes in background
    
    P->>A: <research-complete/>
    deactivate P
    A->>P: "Research says Rust is fast"
    P->>E: "Research says Rust is fast"
    A->>P: end_turn
    P->>E: end_turn
  

What’s cool about this is that the proxy encapsulates the entire flow: it knows how to do the research, and it manages notifying the various participants when the research completes. (Also, this leans on one detail I left out, which is that )

Next feature: tangent mode

Let’s explore our next feature, Q CLI’s /tangent mode. This feature is interesting because it’s a simple (but useful!) example of history editing. The way /tangent works is that, when you first type /tangent, Q CLI saves your current state. You can then continue as normal but when you next type /tangent, your state is restored to where you were. This, as the name suggests, lets you explore a side conversation without polluting your main context.

The basic idea for supporting tangent in SymmACP is that the proxy is going to (a) intercept the tangent prompt and remember where it began; (b) allow the conversation to continue as normal; and then (c) when it’s time to end the tangent, create a new session and replay the history up until the point of the tangent³.

SymACP += replay

You can almost implement “tangent” in ACP as it is, but not quite. In ACP, the agent always owns the session history. The editor can create a new session or load an older one; when loading an older one, the agent “replays” “replays” the events so that the editor can reconstruct the GUI. But there is no way for the editor to “replay” or construct a session to the agent. Instead, the editor can only send prompts, which will cause the agent to reply. In this case, what we want is to be able to say “create a new chat in which I said this and you responded that” so that we can setup the initial state. This way we could easily create a new session that contains the messages from the old one.

So how this would work:

sequenceDiagram
    participant E as Editor
    participant P as Proxy
    participant A as Agent
    
    E->>P: prompt("Hi there!")
    P->>A: prompt("Hi there!")

    Note over E,A: Conversation proceeds
    
    E->>P: prompt("/tangent")
    Note over P: Proxy notes conversation state
    P->>E: end_turn
    E->>P: prompt("btw, ...")
    P->>A: prompt("btw, ...")

    Note over E,A: Conversation proceeds
    
    E->>P: prompt("/tangent")
    
    P->>A: new_session
    P->>A: prompt("Hi there!")    
    Note over P,A: ...Proxy replays conversation...
  

Next feature: interactive walkthroughs

One of the nicer features of Symposium is the ability to do interactive walkthroughs. These consist of an HTML sidebar as well as inline comments in the code:

Right now, this is implemented by a kind of hacky dance:

• The agent invokes an MCP tool and sends it the walkthrough in markdown. This markdown includes commands meant to be placed on particular lines, identified not by line number (agents are bad at line numbers) but by symbol names or search strings.
• The MCP tool parses the markdown, determines the line numbers for comments, and creates HTML. It sends that HTML over IPC to the VSCode extension.
• The VSCode receives the IPC message, displays the HTML in the sidebar, and creates the comments in the code.

It works, but it’s a giant Rube Goldberg machine.

SymmACP += Enriched conversation history

With SymmACP, we would structure the passthrough mechanism as a proxy. Just as today, it would provide an MCP tool to the agent to receive the walkthrough markdown. It would then convert that into the HTML to display on the side along with the various comments to embed in the code. But this is where things are different.

Instead of sending that content over IPC, what I would want to do is to make it possible for proxies to deliver extra information along with the chat. This is relatively easy to do in ACP as is, since it provides for various capabilities, but I think I’d want to go one step further

I would have a proxy layer that manages walkthroughs. As we saw before, it would provide a tool. But there’d be one additional thing, which is that, beyond just a chat history, it would be able to convey additional state. I think the basic conversation structure is like:

• Conversation
  • Turn
    • User prompt(s) – could be zero or more
    • Response(s) – could be zero or more
    • Tool use(s) – could be zero or more

but I think it’d be useful to (a) be able to attach metadata to any of those things, e.g., to add extra context about the conversation or about a specific turn (or even a specific prompt), but also additional kinds of events. For example, tool approvals are an event. And presenting a walkthrough and adding annotations are an event too.

The way I imagine it, one of the core things in SymmACP would be the ability to serialize your state to JSON. You’d be able to ask a SymmACP paricipant to summarize a session. They would in turn ask any delegates to summarize and then add their own metadata along the way. You could also send the request in the other direction – e.g., the agent might present its state to the editor and ask it to augment it.

Enriched history would let walkthroughs be extra metadata

This would mean a walkthrough proxy could add extra metadata into the chat transcript like “the current walkthrough” and “the current comments that are in place”. Then the editor would either know about that metadata or not. If it doesn’t, you wouldn’t see it in your chat. Oh well – or perhaps we do something HTML like, where there’s a way to “degrade gracefully” (e.g., the walkthrough could be presented as a regular “response” but with some metadata that, if you know to look, tells you to interpret it differently). But if the editor DOES know about the metadata, it interprets it specially, throwing the walkthrough up in a panel and adding the comments into the code.

With enriched histories, I think we can even say that in SymmACP, the ability to load, save, and persist sessions itself becomes an extension, something that can be implemented by a proxy; the base protocol only needs the ability to conduct and serialize a conversation.

Final feature: Smarter tool delegation.

Let me sketch out another feature that I’ve been noodling on that I think would be pretty cool. It’s well known that there’s a problem that LLMs get confused when there are too many MCP tools available. They get distracted. And that’s sensible, so would I, if I were given a phonebook-size list of possible things I could do and asked to figure something out. I’d probably just ignore it.

But how do humans deal with this? Well, we don’t take the whole phonebook – we got a shorter list of categories of options and then we drill down. So I go to the File Menu and then I get a list of options, not a flat list of commands.

I wanted to try building an MCP tool for IDE capabilities that was similar. There’s a bajillion set of things that a modern IDE can “do”. It can find references. It can find definitions. It can get type hints. It can do renames. It can extract methods. In fact, the list is even open-ended, since extensions can provide their own commands. I don’t know what all those things are but I have a sense for the kinds of things an IDE can do – and I suspect models do too.

What if you gave them a single tool, “IDE operation”, and they could use plain English to describe what they want? e.g., ide_operation("find definition for the ProxyHandler that referes to HTTP proxies"). Hmm, this is sounding a lot like a delegate, or a sub-agent. Because now you need to use a second LLM to interpret that request – you probably want to do something like, give it a list of sugested IDE capabilities and the ability to find out full details and ask it to come up with a plan (or maybe directly execute the tools) to find the answer.

As it happens, MCP has a capability to enable tools to do this – it’s called (somewhat oddly, in my opinion) “sampling”. It allows for “callbacks” from the MCP tool to the LLM. But literally nobody implements it, from what I can tell.⁴ But sampling is kind of limited anyway. With SymmACP, I think you could do much more interesting things.

SymmACP.contains(simultaneous_sessions)

The key is that ACP already permits a single agent to “serve up” many simultaneous sessions. So that means that if I have a proxy, perhaps one supplying an MCP tool definition, I could use it to start fresh sessions – combine that with the “history replay” capability I mentioned above, and the tool can control exactly what context to bring over into that session to start from, as well, which is very cool (that’s a challenge for MCP servers today, they don’t get access to the conversation history).

sequenceDiagram
    participant E as Editor
    participant P as Proxy
    participant A as Agent
    
    A->>P: ide_operation("...")
    activate P
    P->>A: new_session
    activate P
    activate A
    P->>A: prompt("Using these primitive operations, suggest a way to do '...'")
    A->>P: ...
    A->>P: end_turn
    deactivate P
    deactivate A
    Note over P: performs the plan
    P->>A: result from tool
    deactivate P
  

Conclusion

Ok, this post sketched a variant on ACP that I call SymmACP. SymmACP extends ACP with

• the ability for either side to provide the initial state of a conversation, not just the server
• the ability for an “editor” to provide an MCP tool to the “agent”
• the ability for agents to respond without an initial prompt
• the ability to serialize conversations and attach extra state (already kind of present)

Most of these are modest extensions to ACP, in my opinion, and easily doable in a backwards fashion just by adding new capabilities. But together they unlock the ability for anyone to craft extensions to agents and deploy them in a composable way. I am super excited about this. This is exactly what I wanted Symposium to be all about.

It’s worth noting the old adage: “with great power, comes great responsibility”. These proxies and ACP layers I’ve been talking about are really like IDE extensions. They can effectively do anything you could do. There are obvious security concerns. Though I think that approaches like Microsoft’s Wassette are key here – it’d be awesome to have a “capability-based” notion of what a “proxy layer” is, where everything compiles to WASM, and where users can tune what a given proxy can actually do.

I plan to start sketching a plan to drive this work in Symposium and elsewhere. My goal is to have a completely open and interopable client, one that can be based on any agent (including local ones) and where you can pick and choose which parts you want to use. I expect to build out lots of custom functionality to support Rust development (e.g., explaining and diagnosting trait errors using the new trait solver is high on my list…and macro errors…) but also to have other features like walkthroughs, collaborative interaction style, etc that are all language independent – and I’d love to see language-focused features for other langauges, especially Python and TypeScript (because “the new trifecta”) and Swift and Kotlin (because mobile). If that vision excites you, come join the Symposium Zulip and let’s chat!

Appendix: A guide to the agent protocols I’m aware of

One question I’ve gotten when discussing this is how it compares to the other host of protocols out there. Let me give a brief overview of the related work and how I understand its pros and cons:

• Model context protocol (MCP): The queen of them all. A protocol that provides a set of tools, prompts, and resources up to the agent. Agents can invoke tools by supplying appropriate parameters, which are JSON. Prompts are shorthands that users can invoke using special commands like / or @, they are essentially macros that expand “as if the user typed it” (but they can also have parameters and be dynamically constructed). Resources are just data that can be requested. MCP servers can either be local or hosted remotely. Remote MCP has only recently become an option and auth in particular is limited.
  • Comparison to SymmACP: MCP provides tools that the agent can invoke. SymmACP builds on it by allowing those tools to be provided by outer layers in the proxy chain. SymmACP is oriented at controlling the whole chat “experience”.
• Zed’s Agent Client Protocol (ACP): The basis for SymmACP. Allows editors to create and manage sessions. Focused only on local sessions, since your editor runs locally.
  • Comparison to SymmACP: That’s what this post is all about! SymmACP extends ACP with new capabilities that let intermediate layers manipulate history, provide tools, and provide extended data upstream to support richer interaction patterns than jus chat. PS I expect we may want to support more remote capabilities, but it’s kinda orthogonal in my opinion (e.g., I’d like to be able to work with an agent running over in a cloud-hosted workstation, but I’d probably piggyback on ssh for that).
• Google’s Agent-to-Agent Protocol (A2A) and IBM’s Agent Communication Protocol (ACP)⁵: From what I can tell, Google’s “agent-to-agent” protocol is kinda like a mix of MCP and OpenAPI. You can ping agents that are running remotely and get them to send you “agent cards”, which describe what operations they can perform, how you authenticate, and other stuff like that. It looks to me quite similar to MCP except that it has richer support for remote execution and in particular supports things like long-running communication, where an agent may need to go off and work for a while and then ping you back on a webhook.
  • Comparison to MCP: To me, A2A looks like a variant of MCP that is more geared to remote execution. MCP has a method for tool discovery where you ping the server to get a list of tools; A2A has a similar mechanism with Agent Cards. MCP can run locally, which A2A cannot afaik, but A2A has more options about auth. MCP can only be invoked synchronously, whereas A2A supports long-running operations, progress updates, and callbacks. It seems like the two could be merged to make a single whole.
  • Comparison to SymmACP: I think A2A is orthogonal from SymmACP. A2A is geared to agents that provide services to one another. SymmACP is geared towards building new development tools for interacting with agents. It’s possible you could build something like SymmACP on A2A but I don’t know what you would really gain by it (and I think it’d be easy to do later).

The story thus far

For those of you who haven’t been following, there’s been an ongoing discussion about how best to have ergonomic ref counting:

• It began with the first Rust Project Goals program in 2024H2, where Jonathan Kelley from Dioxus wrote a thoughtful blog post about a path to high-level Rust that eventually became a 2024H2 project goal towards ergonomic ref-counting.
• I wrote a series of blog posts about a trait I called Claim.
• Josh and I talked and Josh opened RFC #3680, which proposed a use keyword and use || closures. Reception, I would say, was mixed; yes, this is tackling a real problem, but there were lots of concerns on the approach. I summarized the key points here.
• Santiago implemented experimental support for (a variant of) RFC #3680 as part of the 2025H1 project goal.
• I authored a 2025H2 project goal proposing that we create an alternative RFC focused on higher-level use-cases which prompted Josh and I have to have a long and fruitful conversation in which he convinced me that this was not the right approach.
• We had a lang-team design meeting on 2025-08-27 in which I presented this survey and summary of the work done thus far.
• And then at the RustConf 2025 Unconf we had a big group discussion on the topic that I found very fruitful, as well as various follow-up conversations with smaller groups.

This blog post is about “the trait”

The focus of this blog post is on one particular question: what should we call “The Trait”. In virtually every design, there has been some kind of trait that is meant to identify something. But it’s been hard to get a handle¹ on what precisely that something is. What is this trait for and what types should implement it? Some things are clear: whatever The Trait is, Rc<T> and Arc<T> should implement it, for example, but that’s about it.

My original proposal was for a trait named Claim that was meant to convey a “lightweight clone” – but really the trait was meant to replace Copy as the definition of which clones ought to be explicit². Jonathan Kelley had a similar proposal but called it Capture. In RFC #3680 the proposal was to call the trait Use.

The details and intent varied, but all of these attempts had one thing in common: they were very operational. That is, the trait was always being defined in terms of what it does (or doesn’t do) but not why it does it. And that I think will always be a weak grounding for a trait like this, prone to confusion and different interpretations. For example, what is a “lightweight” clone? Is it O(1)? But what about things that are O(1) with very high probability? And of course, O(1) doesn’t mean cheap – it might copy 22GB of data every call. That’s O(1).

What you want is a trait where it’s fairly clear when it should and should not be implemented and not based on taste or subjective criteria. And Claim and friends did not meet the bar: in the Unconf, several new Rust users spoke up and said they found it very hard, based on my explanations, to judge whether their types ought to implement The Trait (whatever we call it). That has also been a persitent theme from the RFC and elsewhere.

“Shouldn’t we call it share?” (hat tip: Jack Huey)

But really there is a semantic underpinning here, and it was Jack Huey who first suggested it. Consider this question. What are the differences between cloning a Mutex<Vec<u32>> and a Arc<Mutex<Vec<u32>>>?

One difference, of course, is cost. Cloning the Mutex<Vec<u32>> will deep-clone the vector, cloning the Arc will just increment a referece count.

But the more important difference is what I call “entanglement”. When you clone the Arc, you don’t get a new value – you get back a second handle to the same value.³

Entanglement changes the meaning of the program

Knowing which values are “entangled” is key to understanding what your program does. A big part of how the borrow checker⁴ achieves reliability is by reducing “entaglement”, since it becomes a relative pain to work with in Rust.

Consider the following code. What will be the value of l_before and l_after?

let l_before = v1.len();
let v2 = v1.clone();
v2.push(new_value);
let l_after = v1.len();

The answer, of course, is “depends on the type of v1”. If v1 is a Vec, then l_after == l_before. But if v1 is, say, a struct like this one:

struct SharedVec<T> {
    data: Arc<Mutex<Vec<T>>>
}

impl<T> SharedVec<T> {
    pub fn push(&self, value: T) {
        self.data.lock().unwrap().push(value);
    }

    pub fn len(&self) -> usize {
        self.data.lock().unwrap().len()
    }
}

then l_after == l_before + 1.

There are many types that act like a SharedVec: it’s true for Rc and Arc, of course, but also for things like Bytes and channel endpoints like Sender. All of these are examples of “handles” to underlying values and, when you clone them, you get back a second handle that is indistinguishable from the first one.

We have a name for this concept already: handles

Jack’s insight was that we should focus on the semantic concept (sharing) and not on the operational details (how it’s implemented). This makes it clear when the trait ought to be implemented. I liked this idea a lot, although I eventually decided I didn’t like the name Share. The word isn’t specific enough, I felt, and users might not realize it referred to a specific concept: “shareable types” doesn’t really sound right. But in fact there is a name already in common use for this concept: handles (see e.g. tokio::runtime::Handle).

This is how I arrived at my proposed name and definition for The Trait, which is Handle:⁵

/// Indicates that this type is a *handle* to some
/// underlying resource. The `handle` method is
/// used to get a fresh handle.
trait Handle: Clone {
    final fn handle(&self) -> Self {
        Clone::clone(self)
    }
}

We would lint and advice people to call handle

The Handle trait includes a method handle which is always equivalent to clone. The purpose of this method is to signal to the reader that the result is a second handle to the same underlying value.

Once the Handle trait exists, we should lint on calls to clone when the receiver is known to implement Handle and encourage folks to call handle instead:

impl DataStore {
    fn store_map(&mut self, map: &Arc<HashMap<...>>) {
        self.stored_map = map.clone();
        //                    -----
        //
        // Lint: convert `clone` to `handle` for
        // greater clarity.
    }
}