This blog is where I post up various half-baked ideas that I have.
All Posts
29 March 2023
Thoughts on async closures
I’ve been thinking about async closures and how they could work once we have static async fn in trait. Somewhat surprisingly to me, I found that async closures are a strong example for where async transformers could be an important tool. Let’s dive in! We’re going to start with the problem, then show why modeling async closures as “closures that return futures” would require some deep lifetime magic, and finally circle back to how async transformers can make all this “just work” in a surprisingly natural way.
(read more...)16 March 2023
Must move types
Rust has lots of mechanisms that prevent you from doing something bad. But, right now, it has NO mechanisms that force you to do something good1. I’ve been thinking lately about what it would mean to add “must move” types to the language. This is an idea that I’ve long resisted, because it represents a fundamental increase to complexity. But lately I’m seeing more and more problems that it would help to address, so I wanted to try and think what it might look like, so we can better decide if it’s a good idea.
-
Well, apart from the “must use” lint. ↩
15 March 2023
Temporary lifetimes
In today’s lang team design meeting, we reviewed a doc I wrote about temporary lifetimes in Rust. The current rules were established in a blog post I wrote in 2014. Almost a decade later, we’ve seen that they have some rough edges, and in particular can be a common source of bugs for people. The Rust 2024 Edition gives us a chance to address some of those rough edges. This blog post is a copy of the document that the lang team reviewed. It’s not a proposal, but it covers some of what works well and what doesn’t, and includes a few sketchy ideas towards what we could do better.
(read more...)12 March 2023
To async trait or just to trait
One interesting question about async fn in traits is whether or not we should label the trait itself as async. Until recently, I didn’t see any need for that. But as we discussed the question of how to enable “maybe async” code, we realized that there would be some advantages to distinguishing “async traits” (which could contain async functions) from sync traits (which could not). However, as I’ve thought about the idea more, I’m more and more of the mind that we should not take this step — at least not now. I wanted to write a blog post diving into the considerations as I see them now.
(read more...)03 March 2023
Trait transformers (send bounds, part 3)
I previously introduced the “send bound” problem, which refers to the need to add a Send bound to the future returned by an async function. This post continues my tour over the various solutions that are available. This post covers “Trait Transformers”. This proposal arose from a joint conversation with myself, Eric Holk, Yoshua Wuyts, Oli Scherer, and Tyler Mandry. It’s a variant of Eric Holk’s inferred async send bounds proposal as well as the work that Yosh/Oli have been doing in the keyword generics group. Those posts are worth reading as well, lots of good ideas there.1
-
I originally planned to have part 3 of this series simply summarize those posts, in fact, but I consider Trait Transformers an evolution of those ideas, and close enough that I’m not sure separate posts are needed. ↩
13 February 2023
Return type notation (send bounds, part 2)
In the previous post, I introduced the “send bound” problem, which refers to the need to add a Send bound to the future returned by an async function. I want to start talking about some of the ideas that have been floating around for how to solve this problem. I consider this a bit of an open problem, in that I think we know a lot of the ingredients, but there is a bit of a “delicate balance” to finding the right syntax and so forth. To start with, though, I want to introduce Return Type Notation, which is an idea that Tyler Mandry and I came up with for referring to the type returned by a trait method.
01 February 2023
Async trait send bounds, part 1: intro
Nightly Rust now has support for async functions in traits, so long as you limit yourself to static dispatch. That’s super exciting! And yet, for many users, this support won’t yet meet their needs. One of the problems we need to resolve is how users can conveniently specify when they need an async function to return a Send future. This post covers some of the background on send futures, why we don’t want to adopt the solution from the async_trait crate for the language, and the general direction we would like to go. Follow-up posts will dive into specific solutions.
20 January 2023
Rust in 2023: Growing up
When I started working on Rust in 2011, my daughter was about three months old. She’s now in sixth grade, and she’s started growing rapidly. Sometimes we wake up to find that her clothes don’t quite fit anymore: the sleeves might be a little too short, or the legs come up to her ankles. Rust is experiencing something similar. We’ve been growing tremendously fast over the last few years, and any time you experience growth like that, there are bound to be a few rough patches. Things that don’t work as well as they used to. This holds both in a technical sense — there are parts of the language that don’t seem to scale up to Rust’s current size — and in a social one — some aspects of how the projects runs need to change if we’re going to keep growing the way I think we should. As we head into 2023, with two years to go until the Rust 2024 edition, this is the theme I see for Rust: maturation and scaling.
(read more...)22 September 2022
Rust 2024...the year of everywhere?
I’ve been thinking about what “Rust 2024” will look like lately. I don’t really mean the edition itself — but more like, what will Rust feel like after we’ve finished up the next few years of work? I think the answer is that Rust 2024 is going to be the year of “everywhere”. Let me explain what I mean. Up until now, Rust has had a lot of nice features, but they only work sometimes. By the time 2024 rolls around, they’re going to work everywhere that you want to use them, and I think that’s going to make a big difference in how Rust feels.
(read more...)21 September 2022
Dyn async traits, part 9: call-site selection
After my last post on dyn async traits, some folks pointed out that I was overlooking a seemingly obvious possibility. Why not have the choice of how to manage the future be made at the call site? It’s true, I had largely dismissed that alternative, but it’s worth consideration. This post is going to explore what it would take to get call-site-based dispatch working, and what the ergonomics might look like. I think it’s actually fairly appealing, though it has some limitations.
(read more...)19 September 2022
What I meant by the "soul of Rust"
Re-reading my previous post, I felt I should clarify why I called it the “soul of Rust”. The soul of Rust, to my mind, is definitely not being explicit about allocation. Rather, it’s about the struggle between a few key values — especially productivity and versatility1 in tension with transparency. Rust’s goal has always been to feel like a high-level but with the performance and control of a low-level one. Oftentimes, we are able to find a “third way” that removes the tradeoff, solving both goals pretty well. But finding those “third ways” takes time — and sometimes we just have to accept a certain hit to one value or another for the time being to make progress. It’s exactly at these times, when we have to make a difficult call, that questions about the “soul of Rust” starts to come into play. I’ve been thinking about this a lot, so I thought I would write a post that expands on the role of transparency in Rust, and some of the tensions that arise around it.
-
I didn’t write about versatility in my original post: instead I focused on the hit to productivity. But as I think about it now, versatility is really what’s at play here — versatility really meant that Rust was useful for high-level things and low-level things, and I think that requiring an explicit dyn adaptor is unquestionably a hit against being high-level. Interestingly, I put versatility after transparency in the list, meaning that it was lower priority, and that seems to back up the decision to have some kind of explicit adaptor. ↩
18 September 2022
Dyn async traits, part 8: the soul of Rust
In the last few months, Tyler Mandry and I have been circulating a “User’s Guide from the Future” that describes our current proposed design for async functions in traits. In this blog post, I want to deep dive on one aspect of that proposal: how to handle dynamic dispatch. My goal here is to explore the space a bit and also to address one particularly tricky topic: how explicit do we have to be about the possibility of allocation? This is a tricky topic, and one that gets at that core question: what is the soul of Rust?
(read more...)18 August 2022
Come contribute to Salsa 2022!
Have you heard of the Salsa project? Salsa is a library for incremental computation – it’s used by rust-analyzer, for example, to stay responsive as you type into your IDE (we have also discussed using it in rustc, though more work is needed there). We are in the midst of a big push right now to develop and release Salsa 2022, a major new revision to the API that will make Salsa far more natural to use. I’m writing this blog post both to advertise that ongoing work and to put out a call for contribution. Salsa doesn’t yet have a large group of maintainers, and I would like to fix that. If you’ve been looking for an open source project to try and get involved in, maybe take a look at our Salsa 2022 tracking issue and see if there is an issue you’d like to tackle?
(read more...)27 June 2022
Many modes: a GATs pattern
As some of you may know, on May 4th Jack Huey opened a PR to stabilize an initial version of generic associated types. The current version is at best an MVP: the compiler support is limited, resulting in unnecessary errors, and the syntax is limited, making code that uses GATs much more verbose than I’d like. Nonetheless, I’m super excited, since GATs unlock a lot of interesting use cases, and we can continue to smooth out the rough edges over time. However, folks on the thread have raised some strong concerns about GAT stabilization, including asking whether GATs are worth including in the language at all. The fear is that they make Rust the language too complex, and that it would be better to just use them as an internal building block for other, more accessible features (like async functions and [return position impl trait in traits][RPITIT]). In response to this concern, a number of people have posted about how they are using GATs. I recently took some time to deep dive into these comments and to write about some of the patterns that I found there, including a pattern I am calling the “many modes” pattern, which comes from the chumsky parser combinator library. I posted about this pattern on the thread, but I thought I would cross-post my write-up here to the blog as well, because I think it’s of general interest.
(read more...)15 June 2022
What it feels like when Rust saves your bacon
You’ve probably heard that the Rust type checker can be a great “co-pilot”, helping you to avoid subtle bugs that would have been a royal pain in the !@#!$! to debug. This is truly awesome! But what you may not realize is how it feels in the moment when this happens. The answer typically is: really, really frustrating! Usually, you are trying to get some code to compile and you find you just can’t do it.
(read more...)13 June 2022
Async cancellation: a case study of pub-sub in mini-redis
Lately I’ve been diving deep into tokio’s mini-redis example. The mini-redis example is a great one to look at because it’s a realistic piece of quality async Rust code that is both self-contained and very well documented. Digging into mini-redis, I found that it exemplifies the best and worst of async Rust. On the one hand, the code itself is clean, efficient, and high-level. On the other hand, it relies on a number of subtle async conventions that can easily be done wrong – worse, if you do them wrong, you won’t get a compilation error, and your code will “mostly work”, breaking only in unpredictable timing conditions that are unlikely to occur in unit tests. Just the kind of thing Rust tries to avoid! This isn’t the fault of mini-redis – to my knowledge, there aren’t great alterantive patterns available in async Rust today (I go through some of the alternatives in this post, and their downsides).
(read more...)17 April 2022
Coherence and crate-level where-clauses
Rust has been wrestling with coherence more-or-less since we added methods; our current rule, the “orphan rule”, is safe but overly strict. Roughly speaking, the rule says that one can only implement foreign traits (that is, traits defined by one of your dependencies) for local types (that is, types that you define). The goal of this rule was to help foster the crates.io ecosystem — we wanted to ensure that you could grab any two crates and use them together, without worrying that they might define incompatible impls that can’t be combined. The rule has served us well in that respect, but over time we’ve seen that it can also have a kind of chilling effect, unintentionally working against successful composition of crates in the ecosystem. For this reason, I’ve come to believe that we will have to weaken the orphan rule. The purpose of this post is to write out some preliminary exploration of ways that we might do that.
(read more...)12 April 2022
Implied bounds and perfect derive
There are two ergonomic features that have been discussed for quite some time in Rust land: perfect derive and expanded implied bounds. Until recently, we were a bit stuck on the best way to implement them. Recently though I’ve been working on a new formulation of the Rust trait checker that gives us a bunch of new capabilities — among them, it resolved a soundness formulation that would have prevented these two features from being combined. I’m not going to describe my fix in detail in this post, though; instead, I want to ask a different question. Now that we can implement these features, should we?
(read more...)29 March 2022
dyn*: can we make dyn sized?
Last Friday, tmandry, cramertj, and I had an exciting conversation. We were talking about the design for combining async functions in traits with dyn Trait that tmandry and I had presented to the lang team on Friday. cramertj had an insightful twist to offer on that design, and I want to talk about it here. Keep in mind that this is a piece of “hot off the presses”, in-progress design and hence may easily go nowhere – but at the same time, I’m pretty excited about it. If it works out, it could go a long way towards making dyn Trait user-friendly and accessible in Rust, which I think would be a big deal.
09 February 2022
Dare to ask for more #rust2024
Last year, we shipped Rust 2021 and I have found the changes to be a real improvement in usability. Even though the actual changes themselves were quite modest, the combination of precise capture closure and simpler formatting strings (println!("{x:?}") instead of println!("{:?}", x)) is making a real difference in my “day to day” life.1 Just like NLL and the new module system from Rust 2018, I’ve quickly adapted to these new conventions. When I go back to older code, with its clunky borrow checker workarounds and format strings, I die a little inside.2
27 January 2022
Panics vs cancellation, part 1
One of the things people often complain about when doing Async Rust is cancellation. This has always been a bit confusing to me, because it seems to me that async cancellation should feel a lot like panics in practice, and people don’t complain about panics very often (though they do sometimes). This post is the start of a short series comparing panics and cancellation, seeking after the answer to the question “Why is async cancellation a pain point and what should we do about it?” This post focuses on explaining Rust’s panic philosophy and explaining why I see panics and cancellation as being quite analogous to one another.
(read more...)07 January 2022
Dyn async traits, part 7: a design emerges?
Hi all! Welcome to 2022! Towards the end of last year, Tyler Mandry and I were doing a lot of iteration around supporting “dyn async trait” – i.e., making traits that use async fn dyn safe – and we’re starting to feel pretty good about our design. This is the start of several blog posts talking about where we’re at. In this first post, I’m going to reiterate our goals and give a high-level outline of the design. The next few posts will dive more into the details and the next steps.
18 November 2021
Rustc Reading Club, Take 2
Wow! The response to the last Rustc Reading Club was overwhelming – literally! We maxed out the number of potential zoom attendees and I couldn’t even join the call! It’s clear that there’s a lot of demand here, which is great. We’ve decided to take another stab at running the Rustc Reading Club, but we’re going to try it a bit differently this time. We’re going to start by selecting a smaller group to do it a few times and see how it goes, and then decide how to scale up.
15 November 2021
CTCFT 2021-11-22 Agenda
The next “Cross Team Collaboration Fun Times” (CTCFT) meeting will take place next Monday, on 2021-11-22 at 11am US Eastern Time (click to see in your time zone). Note that this is a new time: we are experimenting with rotating in an earlier time that occurs during the European workday. This post covers the agenda. You’ll find the full details (along with a calendar event, zoom details, etc) on the CTCFT website.
(read more...)05 November 2021
View types for Rust
I wanted to write about an idea that’s been kicking around in the back of my mind for some time. I call it view types. The basic idea is to give a way for an &mut or & reference to identify which fields it is actually going to access. The main use case for this is having “disjoint” methods that don’t interfere with one another.
28 October 2021
Rustc Reading Club
Ever wanted to understand how rustc works? Me too! Doc Jones and I have been talking and we had an idea we wanted to try. Inspired by the very cool Code Reading Club, we are launching an experimental Rustc Reading Club. Doc Jones posted an announcement on her blog, so go take a look!
(read more...)15 October 2021
Dyn async traits, part 6
A quick update to my last post: first, a better way to do what I was trying to do, and second, a sketch of the crate I’d like to see for experimental purposes.
(read more...)14 October 2021
Dyn async traits, part 5
If you’re willing to use nightly, you can already model async functions in traits by using GATs and impl Trait — this is what the Embassy async runtime does, and it’s also what the real-async-trait crate does. One shortcoming, though, is that your trait doesn’t support dynamic dispatch. In the previous posts of this series, I have been exploring some of the reasons for that limitation, and what kind of primitive capabilities need to be exposed in the language to overcome it. My thought was that we could try to stabilize those primitive capabilities with the plan of enabling experimentation. I am still in favor of this plan, but I realized something yesterday: using procedural macros, you can ALMOST do this experimentation today! Unfortunately, it doesn’t quite work owing to some relatively obscure rules in the Rust type system (perhaps some clever readers will find a workaround; that said, these are rules I have wanted to change for a while).
(read more...)13 October 2021
CTCFT 2021-10-18 Agenda
The next “Cross Team Collaboration Fun Times” (CTCFT) meeting will take place next Monday, on 2021-10-18 (in your time zone)! This post covers the agenda. You’ll find the full details (along with a calendar event, zoom details, etc) on the CTCFT website.
07 October 2021
Dyn async traits, part 4
In the previous post, I talked about how we could write our own impl Iterator for dyn Iterator by adding a few primitives. In this post, I want to look at what it would take to extend that to an async iterator trait. As before, I am interested in exploring the “core capabilities” that would be needed to make everything work.
06 October 2021
Dyn async traits, part 3
In the previous “dyn async traits” posts, I talked about how we can think about the compiler as synthesizing an impl that performed the dynamic dispatch. In this post, I wanted to start explore a theoretical future in which this impl was written manually by the Rust programmer. This is in part a thought exercise, but it’s also a possible ingredient for a future design: if we could give programmers more control over the “impl Trait for dyn Trait” impl, then we could enable a lot of use cases.
(read more...)01 October 2021
Dyn async traits, part 2
In the previous post, we uncovered a key challenge for dyn and async traits: the fact that, in Rust today, dyn types have to specify the values for all associated types. This post is going to dive into more background about how dyn traits work today, and in particular it will talk about where that limitation comes from.
30 September 2021
Dyn async traits, part 1
Over the last few weeks, Tyler Mandry and I have been digging hard into what it will take to implement async fn in traits. Per the new lang team initiative process, we are collecting our design thoughts in an ever-evolving website, the async fundamentals initiative. If you’re interested in the area, you should definitely poke around; you may be interested to read about the MVP that we hope to stabilize first, or the (very much WIP) evaluation doc which covers some of the challenges we are still working out. I am going to be writing a series of blog posts focusing on one particular thing that we have been talking through: the problem of dyn and async fn. This first post introduces the problem and the general goal that we are shooting for (but don’t yet know the best way to reach).
16 September 2021
Rustacean Principles, continued
RustConf is always a good time for reflecting on the project. For me, the last week has been particularly “reflective”. Since announcing the Rustacean Principles, I’ve been having a number of conversations with members of the community about how they can be improved. I wanted to write a post summarizing some of the feedback I’ve gotten.
(read more...)15 September 2021
CTCFT 2021-09-20 Agenda
The next “Cross Team Collaboration Fun Times” (CTCFT) meeting will take place next Monday, on 2021-09-20 (in your time zone)! This post covers the agenda. You’ll find the full details (along with a calendar event, zoom details, etc) on the CTCFT website.
08 September 2021
Rustacean Principles
As the web site says, Rust is a language empowering everyone to build reliable and efficient software. I think it’s precisely this feeling of empowerment that people love about Rust. As wycats put it recently to me, Rust makes it “feel like things are possible that otherwise feel out of reach”. But what exactly makes Rust feel that way? If we can describe it, then we can use that description to help us improve Rust, and to guide us as we design extensions to Rust.
Besides the language itself, Rust is also an open-source community, one that prides itself on our ability to do collaborative design. But what do we do which makes us able to work well together? If we can describe that, then we can use those descriptions to help ourselves improve, and to instruct new people on how to better work within the community.
This blog post describes a project I and others have been working on called the Rustacean principles. This project is an attempt to enumerate the (heretofore implicit) principles that govern both Rust’s design and the way our community operates. The principles are still in draft form; for the time being, they live in the nikomatsakis/rustacean-principles repository.
(read more...)30 August 2021
Next CTCFT Meeting: 2021-09-20
Hold the date! The next Cross Team Collaboration Fun Times meeting will be 2021-09-20. We’ll be using the “Asia-friendly” time slot of 21:00 EST.
(read more...)12 July 2021
CTCFT 2021-07-19 Agenda
The next “Cross Team Collaboration Fun Times” (CTCFT) meeting will take place one week from today, on 2021-07-19 (in your time zone)! What follows are the abstracts for the talks we have planned. You’ll find the full details (along with a calendar event, zoom details, etc) on the CTCFT website.
18 June 2021
CTCFT Social Hour
Hey everyone! At the CTCFT meeting this Monday (2021-06-21), we’re going to try a “social hour”. The idea is really simple: for the hour after the meeting, we will create breakout rooms in Zoom with different themes. You can join any breakout room you like and hangout.
(read more...)14 June 2021
CTCFT 2021-06-21 Agenda
The second “Cross Team Collaboration Fun Times” (CTCFT) meeting will take place one week from today, on 2021-06-21 (in your time zone)! This post describes the main agenda items for the meeting; you’ll find the full details (along with a calendar event, zoom details, etc) on the CTCFT website.
26 May 2021
Edition: the song
You may have heard that the Rust 2021 Edition is coming. Along with my daughter Daphne, I have recorded a little song in honor of the occasion! The full lyrics are below – if you feel inspired, please make your own version!1 Enjoy!
-
OMG, that would be amazing. I’ll update the post with any such links I find. ↩
14 May 2021
CTCFTFTW
This Monday I am starting something new: a monthly meeting called the “Cross Team Collaboration Fun Times” (CTCFT)1. Check out our nifty logo2:
-
In keeping with Rust’s long-standing tradition of ridiculous acronyms. ↩
-
Thanks to @Xfactor521! 🙏 ↩
01 May 2021
[AiC] Vision Docs!
The Async Vision Doc effort has been going now for about 6 weeks. It’s been a fun ride, and I’ve learned a lot. It seems like a good time to take a step back and start talking a bit about the vision doc structure and the process. In this post, I’m going to focus on the role that I see vision docs playing in Rust’s planning and decision making, particularly as compared to RFCs.
(read more...)26 April 2021
Async Vision Doc Writing Sessions VII
My week is very scheduled, so I am not able to host any public drafting sessions this week – however, Ryan Levick will be hosting two sessions!
(read more...)19 April 2021
Async Vision Doc Writing Sessions VI
Ryan Levick and I are going to be hosting more Async Vision Doc Writing Sessions this week. We’re not organized enough to have assigned topics yet, so I’m just going to post the dates/times and we’ll be tweeting about the particular topics as we go.
(read more...)12 April 2021
Async Vision Doc Writing Sessions V
This is an exciting week for the vision doc! As of this week, we are starting to draft “shiny future” stories, and we would like your help! (We are also still working on status quo stories, so there is no need to stop working on those.) There will be a blog post coming out on the main Rust blog soon with all the details, but you can go to the “How to vision: Shiny future” page now.
(read more...)07 April 2021
Async Vision Doc Writing Sessions IV
My week is very scheduled, so I am not able to host any public drafting sessions this week – however, Ryan Levick will be hosting two sessions!
(read more...)02 April 2021
My "shiny future"
I’ve been working on the Rust project for just about ten years. The language has evolved radically in that time, and so has the project governance. When I first started, for example, we communicated primarily over the rust-dev mailing list and the #rust IRC channel. I distinctly remember coming into the Mozilla offices1 one day and brson excitedly telling me, “There were almost a dozen people on the #rust IRC channel last night! Just chatting! About Rust!” It’s funny to think about that now, given the scale Rust is operating at today.
-
Offices! Remember those? Actually, I’ve been working remotely since 2013, so to be honest I barely do. ↩
29 March 2021
Async Vision Doc Writing Sessions III
Ryan Levick and I are hosting a number of public drafting sessions scheduled this week. Some of them are scheduled early to cover a wider range of time zones.
(read more...)25 March 2021
Async Vision Doc Writing Sessions II
I’m scheduling two more public drafting sessions for tomorrow, Match 26th:
(read more...)22 March 2021
Async Vision Doc Writing Sessions
Hey folks! As part of the Async Vision Doc effort, I’m planning on holding two public drafting sessions tomorrow, March 23rd:
(read more...)30 December 2020
The more things change...
I’ve got an announcement to make. As of Jan 4th, I’m starting at Amazon as the tech lead of their new Rust team. Working at Mozilla has been a great experience, but I’m pretty excited about this change. It’s a chance to help shape what I hope to be an exciting new phase for Rust, where we grow from a project with a single primary sponsor (Mozilla) to an industry standard, supported by a wide array of companies. It’s also a chance to work with some pretty awesome people – both familiar faces from the Rust community1 and some new folks. Finally, I’m hoping it will be an opportunity for me to refocus my attention to some long-standing projects that I really want to see through.
-
I’ll let them make their own announcements. ↩
18 December 2020
Looking back on 2020
I wanted to write a post that looks back over 2020 from a personal perspective. My goal here is to look at the various initiatives that I’ve been involved in and try to get a sense for how they went, what worked and what didn’t, and also what that means for next year. This post is a backdrop for a #niko2021 post that I plan to post sometime before 2021 actually starts, talking about what I expect to be doing in 2021.
(read more...)11 December 2020
Rotating the compiler team leads
Since we created the Rust teams, I have been serving as lead of two teams: the compiler team and the language design team (I’ve also been a member of the core team, which has no lead). For those less familiar with Rust’s governance, the compiler team is focused on the maintenance and implementation of the compiler itself (and, more recently, the standard library). The language design team is focused on the design aspects. Over that time, all the Rust teams have grown and evolved, with the compiler team in particular being home to a number of really strong members.
(read more...)30 April 2020
Async interviews: my take thus far
The point of the async interview series, in the end, was to help figure out what we should be doing next when it comes to Async I/O. I thought it would be good then to step back and, rather than interviewing someone else, give my opinion on some of the immediate next steps, and a bit about the medium to longer term. I’m also going to talk a bit about what I see as some of the practical challenges.
(read more...)09 April 2020
Library-ification and analyzing Rust
I’ve noticed that the ideas that I post on my blog are getting much more “well rounded”. That is a problem. It means I’m waiting too long to write about things. So I want to post about something that’s a bit more half-baked – it’s an idea that I’ve been kicking around to create a kind of informal “analysis API” for rustc.
(read more...)10 March 2020
Async Interview #7: Withoutboats
Hello everyone! I’m happy to be posting a transcript of my [async interview] with withoutboats. This particularly interview took place way back on January 14th, but the intervening months have been a bit crazy and I didn’t get around to writing it up till now.
(read more...)11 February 2020
Async Interview #6: Eliza Weisman
Hello! For the latest async interview, I spoke with Eliza Weisman (hawkw, mycoliza on twitter). Eliza first came to my attention as the author of the tracing crate, which is a nifty crate for doing application level tracing. However, she is also a core maintainer of tokio, and she works at Buoyant on the linkerd system. linkerd is one of a small set of large applications that were build using 0.1 futures – i.e., before async-await. This range of experience gives Eliza an interesting “overview” perspective on async-await and Rust more generally.
(read more...)20 January 2020
Async Interview #5: Steven Fackler
Hello! For the latest async interview, I spoke with Steven Fackler (sfackler). sfackler has been involved in Rust for a long time and is a member of the Rust libs team. He is also the author of [a lot of crates], most notably tokio-postgres.
(read more...)13 January 2020
Async Interview #4: Florian Gilcher
Hello! For the latest async interview, I spoke with Florian Gilcher (skade). Florian is involved in the async-std project, but he’s also one of the founders of Ferrous Systems, a Rust consulting firm that also does a lot of trainings. In that capacity, he’s been teaching people to use async Rust now since Rust’s 1.0 release.
(read more...)09 January 2020
Towards a Rust foundation
In my #rust2020 blog post, I mentioned rather off-handedly that I think the time has come for us to talk about forming a Rust foundation. I wanted to come back to this topic and talk in more detail about what I think a Rust foundation might look like. And, since I don’t claim to have the final answer to that question by any means, I’d also like to talk about how I think we should have this conversation going forward.
(read more...)23 December 2019
Async Interview #3: Carl Lerche
Hello! For the latest async interview, I spoke with Carl Lerche (carllerche). Among many other crates1, Carl is perhaps best known as one of the key authors behind tokio and mio. These two crates are quite widely used through the async ecosystem. Carl and I spoke on December 3rd.
-
I think [loom] looks particularly cool. ↩
11 December 2019
Async Interview #2: cramertj, part 3
This blog post is continuing my conversation with cramertj. This will be the last post.
(read more...)10 December 2019
Async Interview #2: cramertj, part 2
This blog post is continuing my conversation with cramertj.
(read more...)09 December 2019
Async Interview #2: cramertj
For the second async interview, I spoke with Taylor Cramer – or
cramertj, as I’ll refer to him. cramertj is a member of the compiler
and lang teams and was – until recently – working on Fuchsia at
Google. He’s been a key player in Rust’s Async I/O design and in the
discussions around it. He was also responsible for a lot of the
implementation work to make async fn a reality.
03 December 2019
AiC: Improving the pre-RFC process
I want to write about an idea that Josh Triplett and I have been iterating on to revamp the lang team RFC process. I have written a draft of an RFC already, but this blog post aims to introduce the idea and some of the motivations. The key idea of the RFC is formalize the steps leading up to an RFC, as well as to capture the lang team operations around project groups. The hope is that, if this process works well, it can apply to teams beyond the lang team as well.
(read more...)02 December 2019
Rust 2020
Technically speaking, it’s past the deadline for #rust2020 posts, but I’m running late this year, and I’m going to post something anyway. In this post, I am focusing on what I see as the “largest scale” issues, and not on technical initiatives. If I have time, I will try to post a follow-up talking about some of the key technical initiatives that I think we should focus on as well.
(read more...)28 November 2019
Async Interview #1: Alex and Nick talk about async I/O and WebAssembly
Hello from Iceland! (I’m on vacation.) I’ve just uploaded the first of the Async Interviews to YouTube. It is a conversation with Alex Crichton (alexcrichton) and Nick Fitzgerald (fitzgen) about how WebAssembly and Rust’s Async I/O system interact. When you watch it, you will probably notice two things:
(read more...)22 November 2019
Announcing the Async Interviews
Hello all! I’m going to be trying something new, which I call the “Async Interviews”. These interviews are going to be a series of recorded video calls with various “luminaries” from Rust’s Async I/O effort. In each one, I’m going to be asking roughly the same question: Now that the async-await MVP is stable, what should we be doing next? After each call, I’ll post the recording from the interview, along with a blog post that leaves a brief summary.
(read more...)26 October 2019
why async fn in traits are hard
After reading boat’s excellent post on asynchronous destructors,
I thought it might be a good idea to write some about async fn in
traits. Support for async fn in traits is probably the single most
common feature request that I hear about. It’s also one of the more
complex topics. So I thought it’d be nice to do a blog post kind of
giving the “lay of the land” on that feature – what makes it
complicated? What questions remain open?
11 September 2019
AiC: Shepherds 3.0
I would like to describe an idea that’s been kicking around in my head. I’m calling this idea “shepherds 3.0” – the 3.0 is to distinguish it from the other places we’ve used the term in the past. This proposal actually supplants both of the previous uses of the term, replacing them with what I believe to be a preferred alternative (more on that later).
(read more...)10 July 2019
AiC: Unbounded queues and lang design
I have been thinking about how language feature development works in Rust1. I wanted to write a post about what I see as one of the key problems: too much concurrency in our design process, without any kind of “back-pressure” to help keep the number of “open efforts” under control. This setup does enable us to get a lot of things done sometimes, but I believe it also leads to a number of problems.
-
I’m coming at this from the perspective of the lang team, but I think a lot of this applies more generally. ↩
08 July 2019
Async-await status report #2
I wanted to give an update on the status of the “async-await foundations” working group. This post aims to cover three things:
(read more...)26 April 2019
AiC: Language-design team meta working group
On internals, I just announced the formation of the language-design team meta working group. The role of the meta working group is to figure out how other language-design team working groups should work. The plan is to begin by enumerating some of our goals – the problems we aim to solve, the good things we aim to keep – and then move on to draw up more details plans. I expect this discussion will intersect the RFC process quite heavily (at least when it comes to language design changes). Should be interesting! It’s all happening in the open, and a major goal of mine is for this to be easy to follow along with from the outside – so if talking about talking is your thing, you should check it out.
(read more...)22 April 2019
AiC: Collaborative summary documents
In my previous post, I talked about the idea of mapping the solution space:
(read more...)19 April 2019
AiC: Adventures in consensus
In the talk I gave at Rust LATAM, I said that the Rust project has always emphasized finding the best solution, rather than winning the argument. I think this is one of our deepest values. It’s also one of the hardest for us to uphold.
(read more...)15 April 2019
More than coders
Lately, the compiler team has been changing up the way that we work. Our goal is to make it easier for people to track what we are doing and – hopefully – get involved. This is an ongoing effort, but one thing that has become clear immediately is this: the compiler team needs more than coders.
(read more...)01 March 2019
Async-await status report
I wanted to post a quick update on the status of the async-await effort. The short version is that we’re in the home stretch for some kind of stabilization, but there remain some significant questions to overcome.
(read more...)22 February 2019
Rust lang team working groups
Now that the Rust 2018 edition has shipped, the language design team has been thinking a lot about what to do in 2019 and over the next few years. I think we’ve got a lot of exciting stuff on the horizon, and I wanted to write about it.
(read more...)29 January 2019
Salsa: Incremental recompilation
So for the last couple of months or so, I’ve been hacking in my spare time on this library named salsa, along with a number of awesome other folks. Salsa basically extracts the incremental recompilation techniques that we built for rustc into a general-purpose framework that can be used by other programs. Salsa is developing quickly: with the publishing of v0.10.0, we saw a big step up in the overall ergonomics, and I think the current interface is starting to feel very nice.
(read more...)21 January 2019
Polonius and the case of the hereditary harrop predicate
In my previous post about Polonius and subregion obligations, I mentioned that there needs to be a follow-up to deal with higher-ranked subregions. This post digs a bit more into what the problem is in the first place and sketches out the general solution I have in mind, but doesn’t give any concrete algorithms for it.
(read more...)17 January 2019
Polonius and region errors
Now that NLL has been shipped, I’ve been doing some work revisiting the Polonius project. Polonius is the project that implements the “alias-based formulation” described in my older blogpost. Polonius has come a long way since that post; it’s now quite fast and also experimentally integrated into rustc, where it passes the full test suite.
(read more...)07 January 2019
Rust in 2019: Focus on sustainability
To me, 2018 felt like a big turning point for Rust, and it wasn’t just the edition. Suddenly, it has become “normal” for me to meet people using Rust at their jobs. Rust conferences are growing and starting to have large number of sponsors. Heck, I even met some professional Rust developers amongst the parents at a kid’s birthday party recently. Something has shifted, and I like it.
(read more...)10 November 2018
After NLL: Moving from borrowed data and the sentinel pattern
Continuing on with my “After NLL” series, I want to look at another common error that I see and its solution: today’s choice is about moves from borrowed data and the Sentinel Pattern that can be used to enable them.
(read more...)08 November 2018
Splash 2018 Mid-Week Report
This week I’ve been attending SPLASH 2018. It’s already been quite an interesting week, and it’s only just begun. I thought I’d write up a quick report on some of the things that have been particularly interesting to me, and some of the ideas that they’ve sparked off.
(read more...)01 November 2018
After NLL: Interprocedural conflicts
In my previous post on the status of NLL, I promised to talk about “What is next?” for ownership and borrowing in Rust. I want to lay out the various limitations of Rust’s ownership and borrowing system that I see, as well as – where applicable – current workarounds. I’m curious to get feedback on which problems affect folks the most.
(read more...)31 October 2018
MIR-based borrowck is almost here
Now that the final Rust 2018 Release Candidate has shipped, I thought it would be a good idea to do another update on the state of the MIR-based borrow check (aka NLL). The [last update][] was in June, when we were still hard at work on getting things to work.
(read more...)27 September 2018
October Office Hour Slots
Just a quick note that the October 2018 office hour slots are now posted. If you’re having a problem with Rust, or have something you’d like to talk out, please sign up!
(read more...)24 September 2018
Office Hours #1: Cyclic services
This is a report on the second “office hours”, in which we discussed how to setup a series of services or actors that communicate with one another. This is a classic kind of problem in Rust: how to deal with cyclic data. Usually, the answer is that the cycle is not necessary (as in this case).
(read more...)21 September 2018
Office Hours #0: Debugging with GDB
This is a report on the first “office hours”, in which we discussed debugging Rust programs with gdb. I’m very grateful to Ramana Venkata for suggesting the topic, and to Tom Tromey, who joined in. (Tom has been doing a lot of the work of integrating rustc into gdb and lldb lately.)
(read more...)12 September 2018
Rust office hours
Hello, all! Beginning this Friday (in two days)1, I’m going to start an experiment that I call Rust office hours. The idea is simple: I’ve set aside a few slots per week to help people work through problems they are having learning or using Rust. My goal here is both to be of service but also to gain more insight into the kinds of things people have trouble with. No problem is too big or too small!2
02 September 2018
Rust pattern: Iterating an over a Rc>
This post examines a particular, seemingly simple problem: given
ownership of a Rc<Vec<u32>>, can we write a function that returns an
impl Iterator<Item = u32>? It turns out that this is a bit harder
than it might at first appear – and, as we’ll see, for good
reason. I’ll dig into what’s going on, how you can fix it, and how we
might extend the language in the future to try and get past this
challenge.
13 August 2018
Never patterns, exhaustive matching, and uninhabited types (oh my!)
One of the long-standing issues that we’ve been wrestling with in Rust is how to integrate the concept of an “uninhabited type” – that is, a type which has no values at all. Uninhabited types are useful to represent the “result” of some computation you know will never execute – for example, if you have to define an error type for some computation, but this particular computation can never fail, you might use an uninhabited type.
(read more...)20 June 2018
Proposal for a staged RFC process
I consider Rust’s RFC process one of our great accomplishments, but it’s no secret that it has a few flaws. At its best, the RFC offers an opportunity for collaborative design that is really exciting to be a part of. At its worst, it can devolve into bickering without any real motion towards consensus. If you’ve not done so already, I strongly recommend reading aturon’s excellent blog posts on this topic.
(read more...)15 June 2018
MIR-based borrow check (NLL) status update
I’ve been getting a lot of questions about the status of “Non-lexical lifetimes” (NLL) – or, as I prefer to call it these days, the MIR-based borrow checker – so I wanted to post a status update.
(read more...)27 April 2018
An alias-based formulation of the borrow checker
Ever since the Rust All Hands, I’ve been experimenting with an alternative formulation of the Rust borrow checker. The goal is to find a formulation that overcomes some shortcomings of the current proposal while hopefully also being faster to compute. I have implemented a prototype for this analysis. It passes the full NLL test suite and also handles a few cases – such as #47680 – that the current NLL analysis cannot handle. However, the performance has a long way to go (it is currently slower than existing analysis). That said, I haven’t even begun to optimize yet, and I know I am doing some naive and inefficient things that can definitely be done better; so I am still optimistic we’ll be able to make big strides there.
(read more...)24 April 2018
Rust pattern: Precise closure capture clauses
This is the second in a series of posts about Rust compiler errors. Each one will talk about a particular error that I got recently and try to explain (a) why I am getting it and (b) how I fixed it. The purpose of this series of posts is partly to explain Rust, but partly just to gain data for myself. I may also write posts about errors I’m not getting – basically places where I anticipated an error, and used a pattern to avoid it. I hope that after writing enough of these posts, I or others will be able to synthesize some of these facts to make intermediate Rust material, or perhaps to improve the language itself.
(read more...)16 April 2018
Rust pattern: Rooting an Rc handle
I’ve decided to do a little series of posts about Rust compiler errors. Each one will talk about a particular error that I got recently and try to explain (a) why I am getting it and (b) how I fixed it. The purpose of this series of posts is partly to explain Rust, but partly just to gain data for myself. I may also write posts about errors I’m not getting – basically places where I anticipated an error, and used a pattern to avoid it. I hope that after writing enough of these posts, I or others will be able to synthesize some of these facts to make intermediate Rust material, or perhaps to improve the language itself.
(read more...)09 February 2018
Maximally minimal specialization: always applicable impls
So aturon wrote this beautiful post about what a good week it has been. In there, they wrote:
(read more...)01 February 2018
In Rust, ordinary vectors are values
I’ve been thinking a lot about persistent collections lately and in particular how they relate to Rust, and I wanted to write up some of my observations.1
-
As it happens, the SLG solver that I wrote about before seems like it would really like to use persistent collections. ↩
31 January 2018
An on-demand SLG solver for chalk
In my last Chalk post, I talked about an experimental, SLG-based
solver that I wrote for Chalk. That particular design was based very
closely on the excellent paper
“Efficient top-down computation of queries under the well-founded semantics”, by W. Chen, T. Swift, and D. Warren. It
followed a traditional Prolog execution model: this has a lot of
strengths, but it probably wasn’t really suitable for use in rustc.
The single biggest reason for this was that it didn’t really know when
to stop: given a query like exists<T> { T: Sized }, it would happily
try to enumerate all sized types in the system. It was also pretty
non-obvious to me how to extend that system with things like
co-inductive predicates (needed for auto traits) and a few other
peculiarities of Rust.
09 January 2018
#Rust2018
As part of #Rust2018, I thought I would try to writeup my own (current) perspective. I’ll try to keep things brief.
(read more...)05 January 2018
Lessons from the impl period
So, as you likely know, we tried something new at the end of 2017. For roughly the final quarter of the year, we essentially stopped doing design work, and instead decided to focus on implementation – what we called the “impl period”. We had two goals for the impl period: (a) get a lot of high-value implementation work done and (b) to do that by expanding the size of our community, and making it easy for new people to get involved. To that end, we spun up about 40 working groups, which is really a tremendous figure when you think about it, each of which was devoted to a particular task.
(read more...)21 October 2017
Chalk meets SLG
For the last month or so, I’ve gotten kind of obsessed with exploring a new evaluation model for Chalk. Specifically, I’ve been looking at adapting the SLG algorithm, which is used in the XSB Prolog engine. I recently opened a PR that adds this SLG-based solver as an alternative, and this blog post is an effort to describe how that PR works, and explore some of the advantages and disadvantages I see in this approach relative to the current solver that I described in my previous post.
(read more...)12 September 2017
Cyclic queries in chalk
In my last post about chalk queries, I discussed how the query model in chalk. Since that writing, there have been some updates, and I thought it’d be nice to do a new post covering the current model. This post will also cover the tabling technique that scalexm implemented for handling cyclic relations and show how that enables us to implement implied bounds and other long-desired features in an elegant way. (Nice work, scalexm!)
(read more...)11 July 2017
Non-lexical lifetimes: draft RFC and prototype available
I’ve been hard at work the last month or so on trying to complete the non-lexical lifetimes RFC. I’m pretty excited about how it’s shaping up. I wanted to write a kind of “meta” blog post talking about the current state of the proposal – almost there! – and how you could get involved with helping to push it over the finish line.
(read more...)25 May 2017
Query structure in chalk
For my next post discussing chalk, I want to take kind of a different turn. I want to talk about the general struct of chalk queries and how chalk handles them right now. (If you’ve never heard of chalk, it’s sort of “reference implementation” for Rust’s trait system, as well as an attempt to describe Rust’s trait system in terms of its logical underpinnings; see this post for an introduction to the big idea.)
(read more...)02 May 2017
gnome-class: Integrating Rust and the GNOME object system
I recently participated in the GNOME / Rust “dev sprint” in Mexico City. (A thousand thanks to Federico and Joaquin for organizing!) While there I spent some time working on the gnome-class plugin. The goal of gnome-class was to make it easy to write GObject implementations in Rust which would fully interoperate with C code.
(read more...)23 April 2017
Unification in Chalk, part 2
In my previous post, I talked over the basics of how unification works and showed how that “mathematical version” winds up being expressed in chalk. I want to go a bit further now and extend that base system to cover associated types. These turn out to be a pretty non-trival extension.
(read more...)06 April 2017
Rayon 0.7 released
We just released Rayon 0.7. This is a pretty exciting release, because it marks the official first step towards Rayon 1.0. In addition, it marks the first release where Rayon’s parallel iterators reach “feature parity” with the standard sequential iterators! To mark the moment, I thought I’d post the release notes here on the blog:
(read more...)25 March 2017
Unification in Chalk, part 1
So in my first post on chalk, I mentioned that unification and normalization of associated types were interesting topics. I’m going to write a two-part blog post series covering that. This first part begins with an overview of how ordinary type unification works during compilation. The next post will add in associated types and we can see what kinds of mischief they bring with them.
(read more...)17 March 2017
The Lane Table algorithm
For some time now I’ve been interested in better ways to construct LR(1) parsers. LALRPOP currently allows users to choose between the full LR(1) algorithm or the LALR(1) subset. Neither of these choices is very satisfying:
(read more...)01 March 2017
Nested method calls via two-phase borrowing
In my previous post, I outlined a plan for non-lexical lifetimes. I wanted to write a follow-up post today that discusses different ways that we can extend the system to support nested mutable calls. The ideas here are based on some the ideas that emerged in a recent discussion on internals, although what I describe here is a somewhat simplified variant. If you want more background, it’s worth reading at least the top post in the thread, where I laid out a lot of the history here. I’ll try to summarize the key bits as I go.
(read more...)21 February 2017
Non-lexical lifetimes using liveness and location
At the recent compiler design sprint, we spent some time discussing non-lexical lifetimes, the plan to make Rust’s lifetime system significantly more advanced. I want to write-up those plans here, and give some examples of the kinds of programs that would now type-check, along with some that still will not (for better or worse).
(read more...)17 February 2017
Project idea: datalog output from rustc
I want to have a tool that would enable us to answer all kinds of queries about the structure of Rust code that exists in the wild. This should cover everything from synctactic queries like “How often do people write let x = if { ... } else { match foo { ... } }?” to semantic queries like “How often do people call unsafe functions in another module?” I have some ideas about how to build such a tool, but (I suspect) not enough time to pursue them. I’m looking for people who might be interested in working on it!
12 February 2017
Compiler design sprint summary
This last week we had the rustc compiler team design sprint. This was our second rustc compiler team sprint; the first one (last year) we simply worked on pushing various projects over the finish line (for example, in an epic effort, arielb1 completed dynamic drop during that sprint).
(read more...)01 February 2017
Unsafe code and shared references
In a previous post, I talked about a proposed approach to drafting the
unsafe code guidelines. Specifically, I want to the approach of having
an executable specification of Rust with additional checks that
will signal when undefined behavior has occurred. In this post, I want
to try to dive into that idea a bit more and give some more specifics
of the approach I have in mind. I’m going to focus on this post on the
matter of the proper use of shared references &T – I’ll completely
ignore &mut T for now, since those are much more complicated
(because they require a notion of uniqueness).
26 January 2017
Lowering Rust traits to logic
Over the last year or two (man, it’s scary how time flies), I’ve been doing quite a lot of thinking about Rust’s trait system. I’ve been looking for a way to correct a number of flaws and shortcomings in the current implementation, not the least of which is that it’s performance is not that great. But also, I’ve been wanting to get a relatively clear, normative definition of how the trait system works, so that we can better judge possible extensions. After a number of false starts, I think I’m finally getting somewhere, so I wanted to start writing about it.
(read more...)22 January 2017
Assigning blame to unsafe code
While I was at POPL the last few days, I was reminded of an idea regarding how to bring more struture to the unsafe code guidelines process that I’ve been kicking around lately, but which I have yet to write about publicly. The idea is fresh on my mind because while at POPL I realized that there is an interesting opportunity to leverage the “blame” calculation techniques from gradual typing research. But before I get to blame, let me back up and give some context.
(read more...)14 November 2016
Parallel iterators, part 3: Consumers
This post is the (long awaited, or at least long promised) third post in my series on Rayon’s parallel iterators. The previous two posts were some time ago, but I’ve been feeling inspired to push more on Rayon lately, and I remembered that I had never finished this blog post series.
(read more...)09 November 2016
Associated type constructors, part 4: Unifying ATC and HKT
This post is a continuation of my posts discussing the topic of associated type constructors (ATC) and higher-kinded types (HKT):
(read more...)04 November 2016
Associated type constructors, part 3: What higher-kinded types might look like
This post is a continuation of my posts discussing the topic of associated type constructors (ATC) and higher-kinded types (HKT):
(read more...)03 November 2016
Associated type constructors, part 2: family traits
Hello. This post is a continuation of my posts discussing the topic of associated type constructors (ATC) and higher-kinded types (HKT):
(read more...)02 November 2016
Associated type constructors, part 1: basic concepts and introduction
So for the end of last week, I was at Rust Belt Rust. This was awesome. And not only because the speakers and attendees at Rust Belt Rust were awesome, though they were. But also because it gave aturon, withoutboats, and I a chance to talk over a lot of stuff in person. We covered a lot of territory and so I wanted to do a series of blog posts trying to write down some of the things we were thinking so as to get other people’s input.
(read more...)01 November 2016
Switching to Jekyll
If you visit the site today, you’ll notice it looks quite a bit different. I’ve decided to switch from my old antiquated Octopress to a plain Jekyll-based one. The most immediate of this is that Rust code highlighting looks much better, and I get access to modern Github-flavored markdown. =) Since I understand plain Jekyll a bit more, I’ll hopefully also be able to customize the appearance somewhat – but for now I’m just going with the basic theme.
(read more...)24 October 2016
Supporting blanket impls in specialization
In my previous post, I talked about how we can separate out
specialization into two distinct concepts: reuse and override.
Doing so makes because the conditions that make reuse possible are
more stringent than those that make override possible. In this post,
I want to extend this idea to talk about a new rule for specialization
that allow overriding in more cases. These rules are a big enabler
for specialization, allowing it to accommodate many use cases that we
couldn’t handle before. In particular, they enable us to add blanket
impls like impl<T: Copy> Clone for T in a backwards compatible
fashion, though only under certain conditions.
02 October 2016
Observational equivalence and unsafe code
I spent a really interesting day last week at Northeastern University. First, I saw a fun talk by Philip Haller covering LaCasa, which is a set of extensions to Scala that enable it to track ownership. Many of the techniques reminded me very much of Rust (e.g., the use of “spores”, which are closures that can limit the types of things they close over); if I have time, I’ll try to write up a more detailed comparison in some later post.
(read more...)30 September 2016
Announcing intorust.com
For the past year or so, I and a few others have been iterating on some tutorial slides for learning Rust. I’ve given this tutorial here at the local Boston Rust Meetup a few times, and we used the same basic approach at RustConf; I’ve been pretty happy with the results. But until now it’s been limited to “in person” events.
(read more...)29 September 2016
Distinguishing reuse from override
In my previous post, I started discussing the idea of
intersection impls, which are a possible extension to
specialization. I am specifically looking at the idea of
making it possible to add blanket impls to (e.g.) implement Clone
for any Copy type. We saw that intersection impls, while useful, do
not enable us to do this in a backwards compatible way.
24 September 2016
Intersection Impls
As some of you are probably aware, on the nightly Rust builds, we currently offer a feature called specialization, which was defined in RFC 1210. The idea of specialization is to improve Rust’s existing coherence rules to allow for overlap between impls, so long as one of the overlapping impls can be considered more specific. Specialization is hotly desired because it can enable powerful optimizations, but also because it is an important component for modeling object-oriented designs.
(read more...)12 September 2016
Thoughts on trusting types and unsafe code
I’ve been thinking about the unsafe code guidelines a lot in the back
of my mind. In particular, I’ve been trying to think through what it
means to “trust types” – if you recall from the
Tootsie Pop Model (TPM) blog post, one of the key examples
that I was wrestling with was the RefCell-Ref example. I want to
revisit a variation on that example now, but from a different
angle. (This by the way is one of those “Niko thinks out loud” blog
posts, not one of those “Niko writes up a proposal” blog posts.)
18 August 2016
'Tootsie Pop' Followup
A little while back, I wrote up a tentative proposal I called the “Tootsie Pop” model for unsafe code. It’s safe to say that this model was not universally popular. =) There was quite a long and fruitful discussion on discuss. I wanted to write a quick post summarizing my main take-away from that discussion and to talk a bit about the plans to push the unsafe discussion forward.
(read more...)27 May 2016
The 'Tootsie Pop' model for unsafe code
In my previous post, I spent some time talking about the idea of unsafe abstractions. At the end of the post, I mentioned that Rust does not really have any kind of official guidelines for what kind of code is legal in an unsafe block and what is not.What this means in practice is that people wind up writing what “seems reasonable” and checking it against what the compiler does today. This is of course a risky proposition since it means that if we start doing more optimization in the compiler, we may well wind up breaking unsafe code (the code would still compile; it would just not execute like it used to).
(read more...)23 May 2016
Unsafe abstractions
The unsafe keyword is a crucial part of Rust’s design. For those not
familiar with it, the unsafe keyword is basically a way to bypass
Rust’s type checker; it essentially allows you to write something more
like C code, but using Rust syntax.
09 May 2016
Non-lexical lifetimes: adding the outlives relation
This is the third post in my series on non-lexical lifetimes. Here I want to dive into Problem Case #3 from the introduction. This is an interesting case because exploring it is what led me to move away from the continuous lifetimes proposed as part of RFC 396.
(read more...)04 May 2016
Non-lexical lifetimes based on liveness
In my previous post I outlined several cases that we would like to improve with Rust’s current borrow checker. This post discusses one possible scheme for solving those. The heart of the post is two key ideas:
(read more...)27 April 2016
Non-lexical lifetimes: introduction
Over the last few weeks, I’ve been devoting my free time to fleshing out the theory behind non-lexical lifetimes (NLL). I think I’ve arrived at a pretty good point and I plan to write various posts talking about it. Before getting into the details, though, I wanted to start out with a post that lays out roughly how today’s lexical lifetimes work and gives several examples of problem cases that we would like to solve.
(read more...)02 March 2016
Nice errors in LALRPOP
For the last couple of weeks, my mornings have been occupied with a pretty serious revamping of LALRPOP’s error message output. I will probably wind up doing a series of blog posts about the internal details of how it works, but I wanted to write a little post to advertise this work.
(read more...)25 February 2016
Parallel Iterators Part 2: Producers
This post is the second post in my series on Rayon’s parallel iterators. The goal of this series is to explain how parallel iterators are implemented internally, so I’m going to be going over a lot of details and giving a lot of little code examples in Rust. If all you want to do is use parallel iterators, you don’t really have to understand any of this stuff.
(read more...)19 February 2016
Parallel Iterators Part 1: Foundations
Since giving a talk about Rayon at the Bay Area Rust meetup, I’ve been working off and on on the support for parallel iterators. The basic idea of a parallel iterator is that I should be able to take an existing iterator chain, which operates sequentially, and easily convert it to work in parallel. As a simple example, consider this bit of code that computes the dot-product of two vectors:
(read more...)18 December 2015
Rayon: data parallelism in Rust
Over the last week or so, I’ve been working on an update to Rayon, my experimental library for data parallelism in Rust. I’m pretty happy with the way it’s been going, so I wanted to write a blog post to explain what I’ve got so far.
(read more...)08 October 2015
Virtual Structs Part 4: Extended Enums And Thin Traits
So, aturon wrote this interesting post on an alternative “virtual structs” approach, and, more-or-less since he wrote it, I’ve been wanting to write up my thoughts. I finally got them down.
(read more...)14 September 2015
LALRPOP
Around four years ago, when I had first decided to start at Mozilla research, I had planned to write an LR(1) parser generator. It seemed like a good way to get to know Rust. However, I found that newborns actually occupy somewhat more time than anticipated (read: I was lucky to squeeze in a shower), and hence that never came to pass.
(read more...)20 August 2015
Virtual Structs Part 3: Bringing Enums and Structs Together
So, in previous posts, I discussed the pros and cons of two different approaches to modeling variants: Rust-style enums and C++-style classes. In those posts, I explained why I see Rust enums and OO-style class hierarchies as more alike than different (I personally credit Scala for opening my eyes to this, though I’m sure it’s been understood by others for much longer). The key points were as follows:
(read more...)29 May 2015
Virtual Structs Part 2: Classes strike back
This is the second post summarizing my current thoughts about ideas related to “virtual structs”. In the last post, I described how, when coding C++, I find myself missing Rust’s enum type. In this post, I want to turn it around. I’m going to describe why the class model can be great, and something that’s actually kind of missing from Rust. In the next post, I’ll talk about how I think we can get the best of both worlds for Rust. As in the first post, I’m focusing here primarily on the data layout side of the equation; I’ll discuss virtual dispatch afterwards.
(read more...)05 May 2015
Virtual Structs Part 1: Where Rust's enum shines
One priority for Rust after 1.0 is going to be incorporating some kind of support for “efficient inheritance” or “virtual structs”. In order to motivate and explain this design, I am writing a series of blog posts examining how Rust’s current abstractions compare with those found in other languages.
(read more...)30 April 2015
A few more remarks on reference-counting and leaks
So there has been a lot of really interesting discussion in response to my blog post. I wanted to highlight some of the comments I’ve seen, because I think they raise good points that I failed to address in the blog post itself. My comments here are lightly edited versions of what I wrote elsewhere.
(read more...)29 April 2015
On reference-counting and leaks
What’s a 1.0 release without a little drama? Recently, we discovered that there was an oversight in one of the standard library APIs that we had intended to stabilize. In particular, we recently added an API for scoped threads – that is, child threads which have access to the stack frame of their parent thread.
(read more...)06 April 2015
Modeling graphs in Rust using vector indices
After reading nrc’s blog post about graphs, I felt inspired to
write up an alternative way to code graphs in Rust, based on vectors
and indicates. This encoding has certain advantages over using Rc
and RefCell; in particular, I think it’s a closer fit to Rust’s
ownership model. (Of course, it has disadvantages too.)
14 January 2015
Little Orphan Impls
We’ve recently been doing a lot of work on Rust’s orphan rules,
which are an important part of our system for guaranteeing trait
coherence. The idea of trait coherence is that, given a trait and
some set of types for its type parameters, there should be exactly one
impl that applies. So if we think of the trait Show, we want to
guarantee that if we have a trait reference like MyType : Show, we
can uniquely identify a particular impl. (The alternative to coherence
is to have some way for users to identify which impls are in scope at
any time. It has its own complications; if you’re curious for
more background on why we use coherence, you might find this
rust-dev thread from a while back to be interesting
reading.)
26 November 2014
Purging proc
The so-called “unboxed closure” implementation in Rust has reached the
point where it is time to start using it in the standard library. As
a starting point, I have a
pull request that removes proc from the language. I started
on this because I thought it’d be easier than replacing closures, but
it turns out that there are a few subtle points to this transition.
14 November 2014
Allocators in Rust
There has been a lot of discussion lately about Rust’s allocator story, and in particular our relationship to jemalloc. I’ve been trying to catch up, and I wanted to try and summarize my understanding and explain for others what is going on. I am trying to be as factually precise in this post as possible. If you see a factual error, please do not hesitate to let me know.
(read more...)30 September 2014
Multi- and conditional dispatch in traits
I’ve been working on a branch that implements both multidispatch (selecting the impl for a trait based on more than one input type) and conditional dispatch (selecting the impl for a trait based on where clauses). I wound up taking a direction that is slightly different from what is described in the trait reform RFC, and I wanted to take a chance to explain what I did and why. The main difference is that in the branch we move away from the crate concatenability property in exchange for better inference and less complexity.
(read more...)11 September 2014
Attribute and macro syntax
A few weeks back pcwalton introduced a PR that aimed to move the
attribute and macro syntax to use a leading @ sigil. This means that
one would write macros like:
09 July 2014
An experimental new type inference scheme for Rust
While on vacation, I’ve been working on an alternate type inference scheme for rustc. (Actually, I got it 99% working on the plane, and have been slowly poking at it ever since.) This scheme simplifies the code of the type inferencer dramatically and (I think) helps to meet our intutions (as I will explain). It is however somewhat less flexible than the existing inference scheme, though all of rustc and all the libraries compile without any changes. The scheme will (I believe) make it much simpler to implement to proper one-way matching for traits (explained later).
(read more...)06 July 2014
Implied bounds
I am on vacation for a few weeks. I wanted to take some time to jot down an idea that’s been bouncing around in my head. I plan to submit an RFC at some point on this topic, but not yet, so I thought I’d start out by writing a blog post. Also, my poor blog has been neglected for some time. Consider this a draft RFC. Some important details about references are omitted and will come in a follow-up blog post.
(read more...)14 May 2014
Follow up to Focusing on Ownership
This post withdrawn: it was posted by accident and was incomplete.
(read more...)13 May 2014
Focusing on ownership
Over time, I’ve become convinced that it would be better to drop the distinction between mutable and immutable local variables in Rust. Many people are highly skeptical, to say the least. I wanted to lay out my argument in public. I’ll give various motivations: a philosophical one, an eductional one, and a practical one, and also address the main defense of the current system. (Note: I considered submitting this as a Rust RFC, but decided that the tone was better suited to a blog post, and I don’t have the time to rewrite it now.)
(read more...)24 April 2014
Parallel pipelines for JS
I’ve been thinking about an alternative way to factor the PJS API.
Until now, we’ve had these methods like mapPar(), filterPar() and
so forth. They work mostly like their sequential namesakes but execute
in parallel. This API has the advantage of being easy to explain and
relatively clear, but it’s also not especially flexible nor elegant.
01 April 2014
Value types in JavaScript
Here is the current state of my thinking with respect to value types and value objects. Some of you may have seen Brendan’s slides where he discusses value objects. This post is about the same topic, but it is focused on just the initial part of the work – what it means to be a value object and how we could define value types and integrate them into the standard. I am not going to discuss new syntax or operators yet. I have thoughts on those too but I wanted to start by laying out the foundations.
(read more...)01 April 2014
Typed Objects Status Report
I recently wrote up a paper describing the current version of the Typed Objects API. Anyone who is interested in the current state of the art in that specification should take a look. It’s not too long and intended to be an easy read. This is just a draft copy, and feedback is naturally very welcome – in particular, I expect that before we submit it, the implementation section will change, since it will be much further along.
(read more...)28 February 2014
Rust RFC: Opt-in builtin traits
In today’s Rust, there are a number of builtin traits (sometimes
called “kinds”): Send, Freeze, Share, and Pod (in the future,
perhaps Sized). These are expressed as traits, but they are quite
unlike other traits in certain ways. One way is that they do not have
any methods; instead, implementing a trait like Freeze indicates
that the type has certain properties (defined below). The biggest
difference, though, is that these traits are not implemented manually
by users. Instead, the compiler decides automatically whether or not a
type implements them based on the contents of the type.
25 February 2014
Rust RFC: Stronger guarantees for mutable borrows
Today, if you do a mutable borrow of a local variable, you lose the ability to write to that variable except through the new reference you just created:
(read more...)04 February 2014
Closures and the borrow checker
I have been working on making the borrow checker treat closures in a sound way. I hope to land this patch very soon. I want to describe the impact of these changes and summarize what we might do in the future.
(read more...)09 January 2014
Rvalue lifetimes in Rust
I’ve been working on Issue #3511, which is an effort to rationalize the lifetimes of temporary values in Rust. This issue has been a thorn in the side of Rust users for a while, because the current lifetimes are rather haphazard and frequently too short. Some time ago, I did some thinking on this issue and then let it lie while other things took priority.
(read more...)05 January 2014
DST, Take 5
I believe I have come to the point where I am ready to make a final proposal for DST. Ironically, this proposal is quite similar to where I started, but somewhat more expansive. It seems to be one of those unusual cases where supporting more features actually makes things easier. Thanks to Eridius on IRC for pointing this out to me. I intend for this post to stand alone, so I’m going to start from the beginning in the description.
(read more...)12 December 2013
Structural arrays in Typed Objects
Dave Herman and I were tossing around ideas the other day for a revision of the typed object specification in which we remove nominal array types. The goal is to address some of the awkwardness that we have encountered in designing the PJS API due to nominal array types. I thought I’d try writing it out. This is to some extent a thought experiment.
(read more...)02 December 2013
Thoughts on DST, Part 4
Over the Thanksgiving break I’ve been devoting a lot of time to thinking about DST and Rust’s approach to vector and object types. As before, this is very much still churning in my mind so I’m just going to toss out some semi-structured thoughts.
(read more...)27 November 2013
Thoughts on DST, Part 3
After posting part 2 of my DST series, I realized that I had focusing too much on the pure “type system” aspect and ignoring some of the more…mundane semantics, and in particular the impact of monomorphization. I realize now that – without some further changes – we would not be able to compile and execute the second proposal (which I will dub statically sized typed (SST) from here on out). Let me first explain the problem and then show how my first thoughts on how it might be addressed.
(read more...)27 November 2013
Thoughts on DST, Part 2
In the previous post I elaborated a bit on DSTs and how they could be
created and used. I want to look a bit now at an alternate way to
support the combination of vector types and smart pointers (e.g.,
RC<[uint]>). This approach avoids the use of DSTs. We’ll see that it
also addresses some of the rough patches of DST, but doesn’t work
quite as well for object types.
26 November 2013
Thoughts on DST, Part 1
In the past, I’ve been quite the champion of dynamically sized types
(DST). Specifically what this means is that things like [T] and
Trait would be “types” in the Rust type system. Lately I’ve been
investing a lot of effort thinking through the ramifications of
offering better support for smart pointers, and in particular how this
interacts with dynamically sized types, and I am no longer persuaded
that DST offer the best way forward. I’m a bit unsure, though, and the
topic is complicated, so I wanted to stop and write up a short series
of posts laying out my thought process thus far. This post will
describe what it would mean to offer DST in more detail. I don’t plan
to give a lot of Rust background, since there’s enough to talk about.
22 November 2013
Optimizing SIMD, part 2
A quick follow-up to my previous post. The approach I suggested (“generate boxing instructions but bypass them when possible”) is in some sense pessimistic: we generate the instructions we need for the worst case and then cleanup. Like many problems in computer science, it has an optimistic dual. We could generate unboxed data and then insert boxes where needed. In fact, we have an existing mechanism for doing that, called the type policies. Basically, there is a phase where each MIR opcode goes through and examines the types of its inputs, attempting to reconcile those types with what it needs, either by boxing or unboxing as needed.
(read more...)21 November 2013
Optimizing SIMD
There is currently some ongoing effort to implement the
proposed JavaScript SIMD API in Firefox. The basic idea of the
API is to introduce explicit vector value types called float32x4 and
int32x4. These types fit into the typed objects hierarchy, so you
can create arrays of them, embed them in structs, and so forth.
20 November 2013
Parameter coercion in Rust
Alex Chrichton recently sent a message to the rust-dev mailing list discussing the fate of parameter coercion in Rust. I’ve been thinking about this for a while and feeling conflicted. As is my wont, I decided to try and write up a blog post explaining precisely what’s under debate and exporing the tradeoffs.
(read more...)14 November 2013
Treating vectors like any other container
Some astute comments on a recent thread to rust-dev got me
thinking about our approach to vectors. Until now, we have focused on
having built-in support for vectors via the vector type (~[T]) and
slice types (&[T]). However, another possible approach would be to
move vector support out of the language (almost) entirely and into
standard libraries. I wanted to write out a post exploring this idea;
I find it brings some simplifications and reduces the need for
DST. Seems like an idea worth considering. Consider this a thought
experiment, not exactly a proposal.
04 November 2013
Optimizing complex typed object assignments
I want to optimize assignments to struct-typed fields in typed objects. This post is an effort to work through my optimization plan.
(read more...)04 November 2013
Intermingled parameter lists, take 2
I got a lot of feedback on my post about intermingled parameter lists – most of it negative – and I’ve been thinking about the issue over the weekend. Truth is, I wasn’t terribly fond of the proposal myself – making the position in the list significant feels wrong – but I felt it was the least bad of the various options. However, I’ve had a change of heart, and thus have a new “least bad” proposal.
(read more...)29 October 2013
PJS Roadmap
I think someone reading this blog would be forgiven for thinking that I must spend most of my energy thinking about Rust. In fact I spend a good part of my working hours hammering on PJS. I thought I’d try to write up a bit of a preview of the things we are working on.
(read more...)29 October 2013
Intermingled parameter lists
I’ve been hard at work finishing up work on Rust’s
“new” syntax for lifetimes – I put “new” in quotes because
the syntax has been in use for some time, but in rustc itself the
change was only half-baked. In effect, the new syntax was a kind of
“bridge” to code that was designed for the older type system. This
resulted in some artificial limitations: for example, types could only
have a single lifetime parameter, and it had to be named 'self.
Under my pull request, these limitations are lifted. However, in the
process of implementing things, I realized one minor problem with the
new syntax that must be rectified. In this post I describe the problem
and my proposed solution.
24 October 2013
Iterators yielding mutable references, take 2
OK, after writing the post on iterators that yield mutable references, and discussing with some folk on IRC, I remembered something I had forgotten. There is actually a way to phrase the mutable vector iterator differently such that it is safe. Actually, the end result still has some unsafe code, but it takes the form of a simple helper function, and it’s quite plausible to imagine that code becoming safe eventually. Even better, the approach generalizes to other data structures.
(read more...)24 October 2013
Single inheritance
The following is a draft proposal to support a form of single inheritance, similar to that found in object-oriented languages. The goal is to enable servo to efficiently support structures like the DOM. The proposal is not completely rounded out, but I wanted to put it up in its current form so as to gather any comments.
(read more...)24 October 2013
Iterators yielding mutable references
There is a known bug with the borrowck rules that causes it to
be overly permissive. The fix is relatively simple but it
unfortunately affects some of our Iterator implementations,
specifically those iterators that iterate over &mut values. The
short version is that while it is possible to expose a safe
interface for iterating over &mut values, it is not possible to
implement such iterators without an unsafe block.
18 October 2013
Typed object handles
Yesterday Dmitry Lomov and I had a discussion about the typed objects API. Much of the discussion revolved around the specific issue of handles. In this post I will summarize the issues we discussed and review the various design options.
(read more...)10 October 2013
Fn types in Rust, take 3
As some of you may recall, the fate of function types in Rust has been somewhat uncertain. It all began when I realized that we could not soundly permit closures to recurse, which implies that closures must be tracked in a linear fashion. This caused interactions with the prior plans we had for dynamically sized types, and led to several alternative proposals. The most radical involved keeping only one closure type for borrowed closures and then using macros and object types to represent all other use cases.
(read more...)03 September 2013
Type specifications in Parallel JS
Since I last wrote, we’ve made great progress with the work on the Parallel JS and Typed Objects (nee Binary Data) implementation. In particular, as of this morning, preliminary support for typed objects has landed in Mozilla Nightly, although what’s currently checked in is not fully conformant with the current version of the standard (for this reason, support is limited to Nightly and not available in Aurora or Beta builds).
(read more...)19 July 2013
Integrating binary data and type inference in SpiderMonkey
Since the new version of PJS is going to be based on binary data, we are going to need to have a well-optimized binary data implementation. Nikhil Marathe has prepared an initial implementation, but it is limited to the interpreter. I am looking now at how to integrate binary data into the JIT. The goal is to have accesses get compiled to very efficient generated code. In this blog post, I specifically want to cover the plan for integrating our type inference with binary data.
(read more...)18 July 2013
Rust presentation at Northeastern
Today I had the honor of giving a presentation on Rust at the Northeastern University PL Seminar. It was a lot of fun, and I also had a lot of good conversations afterwards with some of the professors and PhD students there. For those who are interested, I am uploading the slides from my talk. The talk takes the same approach that I would like to use for the next Rust paper. It was kind of a dry run to see if that approach would be understandable and would flow logically; overall, I thought it worked reasonably well.
(read more...)11 June 2013
On the connection between memory management and data-race freedom
As I alluded in the previous post, I have noticed an interesting connection between memory management and data-race freedom. I want to take a moment to elaborate on this, becaause the connection was not obvious to me at first, but it ultimately drives a lot of the Rust design decisions.
(read more...)11 June 2013
Data Parallelism in Rust
Rust currently has very strong support for concurrency in the form of
actors which exchange messages and do not share memory. However, there
are many tasks for which actors are not a good fit. The unbounded
lifetime of actors means that they cannot safely access
stack-allocated memory from another task, even if it is
immutable. Actors cannot share memory except through the relatively
clumsy (and somewhat expensive) mechanism of Arc structures (which
stands for “atomic reference count”), meaning that if there are large
data structures they can be a pain to access. Arc is also
inapplicable to data structures that transition between mutable and
immutable and back again.
06 June 2013
Reducing DST Annotation
So Ben Blum has doing some investigation into the full
implications of the Sized bound that I proposed as part of the
dynamically sized types post. It’s clear that, if we change
nothing else, the impact of Sized will be somewhat greater than I
thought. He estimates somewhere around 40% of the files in libstd need
at least one Sized bound; the actual number may wind up being
somewhat higher.
03 June 2013
More on fns
I have been thinking about my previous proposal for fn types. I wanted to offer some refinements and further thoughts.
(read more...)30 May 2013
Removing procs
I’ve been thinking more about my proposal to split the current fn
type into fn and proc. I have come to the conclusion that
we just don’t need proc at all. I think we can get by with two types:
29 May 2013
Integrating binary data and PJs
We’ve been making a lot of conceptual progress with the PJS API that has not been written down anywhere, so I want to cover some of that work. This post focuses on the integration of parallel methods with the binary data API. It shows how the new API approach allows users to avoid allocation for higher efficiency.
(read more...)14 May 2013
Procedures, continued
So, I didn’t actually mean to post that previous post, I had intended to think more on the idea. But oh well, cat’s out of the bag. In any case, I’ve been thinking about the “closures” vs “procedures” idea that I jotted down there and decided to try and elaborate on it a bit more, since I find it has a lot of appeal. In particular I think that the current collection of closure types is addressing too many distinct use cases and the result is confusing.
(read more...)13 May 2013
Mutable fn alternatives
I’ve been thinking about what I wrote in my last post regarding
closures and I am beginning to change my opinion about the correct
solution. fn~ just seems so unfortunate. So, besides writing fn~,
what are the other options? I just thought I’d write down a few of
the other ideas I’ve come up with for later reference. Not saying any
of the ideas in this post are good yet.
13 May 2013
Recurring closures and dynamically sized types
I realized today that there is an unfortunate interaction between the proposal for dynamically sized types and closure types. In particular, in the case of the recurring closure, I described the soundness issues that arise in our language when closures are able to recurse.
(read more...)30 April 2013
Dynamically sized types, revisited
Recently, separate discussions with pnkfelix and graydon have prompted
me to think a bit about “dynamically sized types” once again. Those
who know Rust well know all about the sometimes annoying discrepancy
between a type like ~T (owned pointer to T) and ~[S] (owned
vector of S instances)—in particular, despite the visual
similarity, there is no type [S], so ~[S] is not an instance of
~T for any T. This design was the outcome of a lot of
back-and-forth and I think it has generally served us well, but I’ve
always had this nagging feeling that we can do better. Recently it
occurred to me how we could, though it’s not without its price.
30 April 2013
Parallelizable JavaScript Subset
I want to look at an interesting topic: what subset of JavaScript do we intend to support for parallel execution, and how long will it take to get that working? As my dear and loyal readers already know, our current engine supports a simple subset of JavaScript but we will want to expand it and make the result more predictable.
(read more...)30 April 2013
The Case of the Recurring Closure
Yesterday I realized that you can violate Rust’s memory safety
guarantees by using “stack closures”, meaning closures that are
allocated on the stack which have can refer to and manipulate the
local variables of the enclosing stack frame. Such closures are
ubiquitous in Rust, since every for loop makes use of them (and
virtually every higher-order function). Luckily, this hole can be
fixed with (I think) very little pain—in fact, I think fixing it
can also help us make other analyses a little less strict.
04 April 2013
Nested lifetimes
While working on issue #5656 I encountered an interesting problem that I had not anticipated. The result is a neat little extension to the region type system that increases its expressive power. The change is completely internal to the type rules and involves no user-visible syntax or anything like that, though there are some (basically nonsensical) programs that will no longer compile. Anyway I found it interesting and thought I would share.
(read more...)04 April 2013
A tour of the Parallel JS implementation (Part 2)
In my last post about ParallelJS, I discussed the ForkJoin()
intrinsic and showed how it was used to implement the parallel map
operation. Today I want to write about the high-level changes to
IonMonkey that are needed to support ForkJoin(). IonMonkey, of
course, is our JavaScript engine.
03 April 2013
Associated items continued
I want to [finish my discussion of associated items][pp] by taking a look at how they are handled in Haskell, and what that might mean in Rust.
(read more...)02 April 2013
Associated items
I’ve been doing a lot of thinking about Rust’s trait system lately. The current system is a bit uneven: it offers a lot of power, but the implementation is inconsistent and incomplete, and in some cases we haven’t thought hard enough about precisely what should be allowed and what should not. I’m going to write a series of posts looking at various aspects of the trait system and trying to suss out what we should be doing in each case. In particular I want to be sure that our trait design is forwards compatible: that is, I expect that we will defer final decisions about various aspects of the trait system until after 1.0, but we should look now and try to anticipate any future difficulties we may encounter.
(read more...)21 March 2013
Guaranteeing parallel execution
One common criticism of the work on ParallelJS is that the API itself does not guarantee parallel execution. Instead, our approach has been to offer methods whose definition makes parallel execution possible, but we have left it up to the engines to define the exact set of JavaScript that will be safe for parallel execution.
(read more...)20 March 2013
A tour of the Parallel JS implementation (Part 1)
I am going to write a series of blog posts giving a tour of the current Parallel JS implementation in SpiderMonkey. These posts are intended to serve partly as documentation for the code. The plan is to begin high level and work my way down to the nitty gritty details, so here we go!
(read more...)20 March 2013
Parallel JS lands
The first version of our work on ParallelJS has just been promoted to mozilla-central and thus will soon be appearing in a Nightly Firefox build near you. I find this pretty exciting. In honor of the occassion, I wanted to take a moment to step back and look both at what has landed now, what we expect to land soon, and the overall trajectory we are aiming for.
(read more...)26 February 2013
Splitting the PJs API
Lately, I’ve been thinking about the ParallelJS API that we want to expose. In particular, I’ve been considering offering methods on the normal array type for basic parallel operations. I think this opens up some interesting doors.
(read more...)22 February 2013
Interfacing with C functions in Rust
One of the things that I’ve been working on for some time now is the proper integration of C functions. As with virtually every other facet of the design of Rust, we’ve been slowly moving from a model where Rust tried to hide low-level details for you to one where Rust offers tight control over what’s going on, with the type system intervening only as needed to prevent segfaults or other strange behavior. This blog post details what I consider to be the best proposal so far; some of the finer points are a bit vague, however.
(read more...)17 January 2013
Destructors and finalizers in Rust
Rust features destructors and, as of this moment, they are simply not sound with respect to many other features of the language, such as borrowed and managed pointers. The problem is that destructors are granted unlimited access to arbitrary data, but the type system and runtime do not take that into account. I propose to fix this by limiting destructors to owned types, meaning types that don’t contain borrowed or managed pointers.
(read more...)16 January 2013
Revised for loop protocol
In Rust today, there is a
special for syntax designed to support interruptible loops.
Since we introduced it, this has proven to be a remarkable success.
However, I think we can improve it very slightly.
15 January 2013
Lifetime notation redux
In a previous post I outlined some of the options for updating our lifetime syntax. I want to revist those examples after having given the matter more thought, and also after some discussions in the comments and on IRC.
(read more...)03 January 2013
The case FOR deterministic results
In my last post, I made the case against having a deterministic semantics. I’ve gotten a fair amount of feedback saying that, for a Web API, introducing nondeterminism is a very risky idea. Certainly the arguments are strong. Therefore, I want to take a moment and make the case for determinism.
(read more...)02 January 2013
Deterministic or not?
One of the interesting questions with respect to Parallel JS is what the semantics ought to be if you attempt a parallel operation with a kernel function that has side-effects. There are basically three reasonable options:
(read more...)30 December 2012
Lifetime notation
I’ve been thinking for a while that our lifetime notation has too many defaults which can be more confusing than helpful. A recent spate of e-mails on rust-dev brought this back to my mind. I’ve been wanting to take a look at these defaults for a while, so I thought I’d write up a quick exploration of the “syntactic space”. A warning: this is not really an exciting post to read. I hope to have a few of those coming up very soon. This one is mostly just a list of syntactic options I wanted to document for future reference and to serve as a starting point for discussion.
(read more...)06 December 2012
Improving our parallel intrinsic
I mentioned in my previous post that we are using a single primitive parallel operation to implement PJs. It turns out that I am not very satisfied with what we currently have and I have been thinking about some alternatives. In this post I’ll describe briefly how things are setup, what problems I see, and then sketch out how I think we could improve it.
(read more...)05 December 2012
Self-hosted Parallel JS
The blog has been silent for a while. The reason is that I’ve been hard at work on Parallel JS. It’s come a long way: in fact, the goal is to land an initial version in the next couple weeks!
(read more...)18 November 2012
Imagine never hearing the phrase 'aliasable, mutable' again
I’ve been thinking of a radical change we could make to the treatment of mutability and borrowed pointers in Rust. The goal is to eliminate all of the error messages about “aliasable, mutable data” that the borrow checker currently issues. The idea is somewhat inspired by writing a recent paper on Rust’s current system—writing a paper on something never fails to get me thinking about how to improve it, though it sometimes fails to stimulate ideas which are actually good—and also somewhat inspired by recent conversations on IRC and in person.
(read more...)24 October 2012
Restrict pointers
I am considering whether we should add a way to borrow something but retain uniqueness. This would address a shortcoming of the borrowing system that has been bothering me for some time, and it would enable a few patterns that are difficult or awkward today.
(read more...)24 October 2012
Purity in Parallel JavaScript
I can’t believe I’m saying this, but I’ve started to think that
Parallel JS (nee Rivertrail) should not demand pure callbacks to
functions like map() and so forth. Rather it should just accept
arbitrary functions. Previously, I thought that it was important that
ParallelArray methods should only accept functions which, at least
in a perfect world, would be safely parallelizable. But I am no
longer so sure why that is an important goal. Here is my reasoning.
23 October 2012
Function and object types
My big goal for 0.5 is to straighten out our function types (yet again). I’ve been tossing the design for these over in my head since the summer and I wanted to lay out my plan. This is a variation of something that Ben Blum and I sketched out on a whiteboard.
(read more...)12 October 2012
Extending the definition of purity in Rust
In this post I propose an extension of Rust’s purity rules. The short
version is that pure functions would be allowed to mutate data owned
by their &mut parameters. This extends the current Rust purity
rules which allow pure functions to invoke impure closures so long as
they are an argument to the function. The principle is the same: pure
functions are functions whose side-effects can be completely
determined by examining their parameters (for the more formally minded
among you, this is effectively an effect-parametric system with very
lightweight notation). The rest of the post is an elaboration and
justification of this idea.
10 October 2012
Rivertrail
I have started work on implementing Rivertrail, Intel’s proposal for data parallelism in JS. I am excited about this project, it seems like it’s going to be great fun. The initial version that we produce is going to be focused on Intel’s specification, but I hope we can eventually combine it with the more general stuff I’ve been doing as part of PJs. There is an awful lot of overlap between the two, though also a few minor differences that will need to be ironed out.
(read more...)05 October 2012
Termination of trait matching
So, the condition that was supposed to ensure termination in my previous post is most certainly wrong. The idea was to prevent tautological impls like the following:
(read more...)04 October 2012
A postscript on traits and impls
I was thinking more about type classes as I walked down the street. In my prior post I wrote that the rules I proposed resulted in a system where traits loosely fit the following Haskell template:
(read more...)04 October 2012
Refining traits and impls
Currently, the Rust compiler accepts all manner of trait, impl, and bound declarations. In fact, it accepts plenty of declarations that later phases of the compiler are not sophisticated enough to handle. In other words, the syntax is writing checks the semantics can’t cash. (An aside: I just love saying that phrase for some perverse reason. I really wish however that checks, like rotary dial telephones, were something that younger people vaguely understood but which no longer had relevance in the modern era. The Swiss Einzahlungschein truly opened my eyes! Anyhow.)
(read more...)01 October 2012
Moves based on type
I have been trying to come up with a reasonable set of rules for deciding when a pattern binding ought to be a move and when it ought to be a copy and utterly failing. Simultaneously, pcwalton, brson, and I kind of simultaneously arrived at an alternate design that tries to simplify the copy/move distinction. I think that it also solves the question of when to copy/move pattern bindings in a nice way. Therefore, I wanted to write up this proposal.
(read more...)26 September 2012
Type system for borrowing permissions
Well, I have not done too well with my goal of reading a research paper a day on the train (actually my initial goal was two papers, but seeing as how I’ve failed so spectacularly, I’ve dialed it back some). However, I’ve decided to give it another go. I’ve bought a printer now so I can print papers out (double-sided, no less!) at home (I had initially planned to buy an iPad or something, but a decent printer is only $100, and paper is still nicer to read and write notes on…you do the math). As additional motivation, I’m working again on the paper on Rust’s new borrowed pointers and so I have to catch up on a lot of related work.
(read more...)15 September 2012
Rvalue lifetimes
We need to clarify our story on rvalue lifetimes. This is related to issue #3387 and also various recent and not-so-recent discussions on IRC.
(read more...)24 August 2012
Datasort refinements
One of the things that is sometimes frustrating in Rust is the inability to define a type that indicates some subset of enum variants. For example, it is very common to have a pattern like this:
(read more...)30 July 2012
Type inference in Spidermonkey
I’ve been slowly learning how type inference works in SpiderMonkey. As I understand it, SpiderMonkey’s type inference scheme is the brain child of one Brian “Hack-it”, coder extraordinaire. You may have seen a recent PLDI publication on the topic. You may, like me, have read that publication. You may, also like me, have walked away thinking, “um, I don’t really understand how that works.” In that case, dear reader, this blog post is for you. Well, actually, it’s for me, to try and document what I gleaned from a conversation or two. It it is almost certainly not entirely accurate and it may or may not be helpful.
(read more...)24 July 2012
Generalizing inherited mutability
I have been working on a change to the definition of mutability in Rust. This is a much smaller change than my previous thought experiments, which were aimed at achieving better parameterization (those are still percolating; I think the best approach is a modified version of the latest proposal where not all types have mutability but type parameters do…but that’s a problem for another day with many complications). The goal of these changes is to enable operations like “freeze” and “thaw”.
(read more...)19 July 2012
Yet another tutorial on borrowed pointers
Here is my latest stab at a tutorial on borrowed pointers. I know, I know, enough with the borrowed pointer tutorials already! Hopefully this will be my last post in this vein for a while. I am much happier with this version. It is still too long to serve as a chapter in the general Rust tutorial, but I think it’s more approachable than the previous attempt, which was more of a reference document. As always, feedback welcome! I have tried to incorporate what people wrote in the comments into this version.
(read more...)17 July 2012
About that 'tutorial'...
One thing I didn’t make clear regarding my last post: I am not especially satisfied with the way the “tutorial” was turning out. I use scare quotes here because I think it resembles a reference manual more than a tutorial. Nonetheless I think there are some sections that are quite good; and a document like it probably ought to exist. So I figured I’d post it anyhow and I can mine it for material later. I intend to start afresh however with something that dives at the core ideas, which I think are relatively simple.
(read more...)17 July 2012
Borrowed pointer tutorial
Here is a (much) more complete draft of the tutorial on borrowed pointers. It is becoming more in-depth than I intended. I hope to later extract a much shorter subset. But I thought I’d post what I’ve got so far.
(read more...)13 July 2012
Concurrent maps
I had a very interesting discussion with Sriram and Terrence (of Kilim and ANTLR fame, respectively—two smart dudes) yesterday. One of the things we talked about was adapting shared-memory data structures like concurrent hash maps into an actor setting.
(read more...)12 July 2012
Fn types
I am trying to mop up some of the remaining work for regions now. One of the big remaining areas is dealing with function and iface types. This proposal is certainly influenced by my previous proposals. However, we have backed away from the idea of dynamically-sized types for vectors and so I will do the same here.
(read more...)10 July 2012
Borrowed Pointer Tutorial
This is a draft of (the first section of) a new Rust tutorial on borrowed pointers (the official name for “regions”). Comments welcome.
(read more...)11 June 2012
HotPar
Last Thursday and Friday I had the good fortune of presenting a paper of mine at HotPar 2012. The paper is called Parallel Closures: A New Twist on an Old Idea; it basically describes what has evolved to become the PJs (Parallel JavaScript) model, though it does so in the context of a static checker built in Java.
(read more...)10 June 2012
Unifying patterns in alts and lets
This is a proposal to unify the mechanics of alt and destructuring
assignment. It was born out of discussion between erickt, pcwalton,
and I amidst various bugs in the bug tracker but I wanted to float it
around to a larger audience. I’d like to discuss this on Tuesday,
because one of the logical next steps for the regions work is to begin
deciding precisely what to do about the types of identifiers in alts.
31 May 2012
Mutability
OK, I’ve been thinking more about the mutability issue and I think I have found a formulation that I am happy with. The basic idea is that we refactor types like so:
(read more...)30 May 2012
Mutability idea retracted
I have been thinking a bit more about the approach to mutability I recently discussed. It seemed a bit too good to be true (too clean) and I think I’ve realized a problem.
(read more...)29 May 2012
Simple effect system
Currently, Rust has an effect system but refuses to admit it. In an effort to broaden the set of things that can be safely done in the face of extant aliases into the heap, I have been experimenting with a lightweight extension to Rust’s system. So far I think it is promising but also no magic bullet.
(read more...)28 May 2012
Moving mutability into the type
I am dissatisfied with how mutability is treated in the Rust type
system. The current system is that a type is not prefixed mutable;
rather, lvalues are. That is, a type T is defined like so:
14 May 2012
Vectors, slices, and functions, oh my!
I wanted to bring together the various ideas around vectors and function types into one post. The goals of these changes are
(read more...)08 May 2012
Iface types
Yesterday I wrote about my scheme for paring down our set of function
types to one type, fn:kind(S) -> T. When I finished writing the
post, I was feeling somewhat uncertain about the merits of the idea,
but I’m feeling somewhat better about it today. I really like the
idea that top-level items have the type fn:kind(S) -> T and that you
therefore give them an explicit sigil to use them in an expression;
this allows us to remove the “bare function” type altogether without
any complex hacks in the inference scheme.
07 May 2012
Fn types
As you loyal readers know, I am on a quest to make the Rust type
system more orthogonal with respect to the kind of pointer in use,
by which I mean that I want to have the three pointer sigils (@,
&, and ~) indicate where memory is located and the other types
indicate what value is to be found at that memory. Right now there
are a few cases where we conflate the two things into one type. The
first, vectors and slices, I discused in a recent post. This post
discusses the second case: function and interface types.
05 May 2012
Borrowing errors
I implemented a simple, non-flow-sensitive version of the reference checker which I described in my previous post. Of course it does not accept the Rust codebase; however, the lack of flow-sensitivity is not the problem, but rather our extensive use of unique vectors. I thought I’d write a post first showing the problem that you run into and then the various options for solving it.
(read more...)01 May 2012
Borrowing
I’ve been working for the last few days on the proper safety conditions for borrowing. I am coming into a situation where I am not sure what would be the best approach. The question boils down to how coarse-grained and approximate our algorithm ought to be: in particular, ought it to be flow sensitive? But let me back up a bit, first, and provide a bit of background.
(read more...)27 April 2012
In favor of types of unknown size
I’m still thinking about vector and string types in Rust and I think I’ve decided what I feel is the best approach. I thought I’d summarize it here and make the case for it. If you don’t know what I’m talking about, see this post for more background. I’ll forward this to the mailing list as well; I’m sorry if it seems like I’m harping on this issue. I just think vectors and strings are kind of central data structures so we want them to be as nice as possible, both in terms of what you can do with them and in terms of the notations we use to work with them.
(read more...)25 April 2012
Permission regions for race-free parallelism
I’ve been making a point of reading academic papers on the train as I ride home. It’s so easy to get behind with the sheer quantity of work that is being produced. Anyway, it occurred to me that I ought to try and summarize the papers I read on this blog so that I can I remember my reactions to them.
(read more...)25 April 2012
References
I want to do an introduction to the regions system I’ve been working on. This is work-in-progress, so some of the details are likely to change. Also, I’m going to try some new terminology on for size: although it has a long history in the literature, I think the term “region” is not particularly accurate, so I am going to use the term “lifetime” or “pointer lifetime” and see how it fits.
(read more...)23 April 2012
Vectors, strings, and slices
We’ve been discussing a lot about how to manage vectors and strings in Rust. Graydon sent out an excellent proposal which allows for a great number of use cases to be elegant handled. However, I find the syntax somewhat misleading. I’ve proposed one alternative on the mailing list, but I now find I don’t like it, so I thought I’d brainstorm a bit and try to find something better.
(read more...)23 April 2012
On types and type schemes
After my recent dalliance in
Matters of a Truly Trivial Nature, I’d like to return to
Matters Most Deep and Profound. I’m running up against an interesting
question with regions that has to do with the nature of function types
like fn(&int): up until now, I’ve assumed that this refers to a
function that takes an integer pointer in some region that is
specified by the caller. That is, it is a kind of shorthand for a
type that might be written like fn<r>(&r.int), where the <r>
indicates that the function type is parameterized by the region r.
15 April 2012
Syntax matters...?
For a long time, it was considered fairly obvious, I think, that syntax didn’t really matter. It was just the surface skin over the underlying ideas. In recent times, though, the prevailing wisdom has reversed, and it is now quite common to hear people talk about how “syntax matters”.
(read more...)11 April 2012
DOA: Don't overabstract
I’d like to propose a term for code that has been “over-DRY’d” (dessicated?). I occasionally run across some method which just seems horribly complex. Reading it closer, it usually turns out that what happened is that two or three independent operations got collected into one subroutine. Perhaps they started out as doing almost the same thing—but before long, they diverged, and now the subroutine has grown a hundred parameters and has a control-flow path that requires a whiteboard and a ultra-super-fine-point marker to follow. But, just as often, you can tear this routine apart into two or three routines that read just fine, even if they share a line or two of code in common. So I’m going to start calling such routines “DOA”, though the acronym has a bit of a different expansion when used as an adjective.
(read more...)10 April 2012
Declared vs duckish typing
One of the questions in our object system is what precisely how “declared” we want things to be when it comes to interfaces and implementations. In a discussion on IRC, graydon suggested it’d be nice to have terms like “duck-typing” defined more precisely in a Rust syntax, and he is correct. So here is my effort.
(read more...)09 April 2012
Rust's object system
On the rust-dev mailing list, someone pointed out another
“BitC retrospective” post by Jonathon Shapiro concerning typeclasses.
The Rust object system provides interesting solutions to some of the
problems he raises. We also manage to combine traditional
class-oriented OOP with Haskell’s type classes in a way that feels
seamless to me. I thought I would describe the object system as I see
it in a post. However, it turns out that this will take me far too
long to fit into a single blog post, so I’m going to do a series.
This first one just describes the basics.
06 April 2012
For loops
First off, I want to welcome Brian Anderson to the Rust blog-o-sphere
(which so far consists primarily of myself). His first post
does a great job of explaining how to use the new for syntax that
was recently added to Rust: this syntax allows for break, ret, and
cont from within user-defined loops, which is very nice.
28 March 2012
Servo design
Yesterday we had a hackathon/meeting to discuss the overarching design of Servo, the project to build a next-generation rendering engine. We didn’t ultimately do much hacking (though we did a little), but mostly we tried to hammer out the big picture so that we can actually get to writing code. I wanted to try and write up what I understood as the consensus (for the moment, anyway).
(read more...)28 March 2012
Avoiding region explosion in Rust
pcwalton and I (but mostly pcwalton) have been hard at work implementing regions in Rust. We are hoping to use regions to avoid a lot of memory allocation overhead in the compiler—the idea is to use memory pools (a.k.a. arenas) so that we can cheaply allocate the data needed to process a given function and then release it all in one shot. It is well known that arenas are great fit for the memory allocation patterns of a compiler, which tend to produce a lot of data that lives for the duration of a pass but is not needed afterwards.
(read more...)03 March 2012
Progress on inlining
Cross-crate inlining has come a long way and is now basically functional (I have yet to write a comprehensive test suite, so I’m sure it will fail when exercising various corners of the language).
(read more...)25 February 2012
Serialization without type information via impls
My current implementation of the auto-serialization code generator requires full type information. This is a drag. First, macros and syntax extension currently run before the type checker, so requiring full type information prevents the auto-serialization code from being implemented in the compiler, as it should be. At first I wanted to change how the compiler works to provide type information, but after numerous discussions with pcwalton and dherman, I’ve come to the conclusion that this is a bad idea: it requires exposing an API for the AST and for type information and introduces numerous other complications.
(read more...)22 February 2012
Regions tradeoffs
In the last few posts I’ve been discussing various options for regions. I’ve come to see region support as a kind of continuum, where the current system of reference modes lies at one end and a full-blown region system with explicit parameterized types and user-defined memory pools lies at the other. In between there are various options. To better explore these tradeoffs, I wrote up a document that outlines various possible schemes and also details use cases that are enabled by these schemes. I don’t claim this to be a comprehensive list of all possible schemes, just the ones I’ve thought about so far. In some cases, the descriptions are quite hand-wavy. I also think some of them don’t hang together so well.
(read more...)18 February 2012
Versioning considered OK
Marijn pointed out to me that our current setup should avoid the worst of the versioning problems I was afraid of. In the snapshot, we package up a copy of the compiler along with its associated libraries, and use this compiler to produce the new compiler. The new compiler can then compilers its own target libraries, thus avoiding the need to interact with libraries produced by the snapshot.
(read more...)17 February 2012
CCI and versioning
I’ve been busily implementing the Cross-Crate Inlining stuff, but one area I haven’t looked at much is versioning. In particular, if we are going to be serializing the AST, we need a plan for what to do when the AST changes. Actually, if inlining were only to be used for performance, we wouldn’t really need to have a plan: we could just not inline when the AST appeared to be stored in some form we don’t understand. However, if we fully monomorphize, we will not have that luxury: without type descriptors, the only way to compile cross-crate, generic calls will be by inlining.
(read more...)16 February 2012
Returning refs
One commonly requested feature for regions is the ability to return
references to the inside of structures. I did not allow that in the
proposal in my previous post because I did not want to have any
region annotations beyond a simple &. I think, however, that if you
want to allow returning references to the interior of a parameter, you
need a way for the user to denote region names explicitly.
15 February 2012
Regions-lite...ish
I was talking to brson today about the possibility of moving Rust to a regions system. He pointed out that the complexity costs may be high. I was trying to make a slimmer version where explicit region names were never required. This is what I came up with. The truth is, it’s not that different from the original: adding back region names wouldn’t change much. But I’m posting it anyway because it includes a description of how to handle regions in types and I think it’s the most complete and correct proposal at the moment.
(read more...)14 February 2012
Using futures in the task API
Brian pointed out to me a nice solution to the Task API problem that I have overlooked, though it’s fairly obvious. Basically, I had rejected a “builder” style API for tasks because there is often a need for the child task to be able to send some data back to its parent after it has been spawned, and a builder API cannot easily accommodate this. Brian’s idea was to encapsulate these using futures. It’s still not perfect but it’s better I think and more composable than my first, limited proposal. It still requires that the actor pattern be a separate module.
(read more...)13 February 2012
Task API
One of the thorny API problems I’ve been thinking about lately is the task API for Rust. I originally had in mind this fancy and very flexible aproach based on bind. When I spelled it out I found it was very powerful and flexible but also completely unworkable in practice.
(read more...)09 February 2012
Auto-serialization in Rust
I’ve been working on implementing Cross-Crate Inlining. The
major task here is to serialize the AST. This is conceptually trivial
but in practice a major pain. It’s an interesting fact that the more
tightly you type your data, the more of a pain it (generally) is to
work with in a generic fashion. Of functional-ish languages that I’ve
used, Scala actually makes things relatively easy by using a
combination of reflection and dynamic typing (interfaces like
Product come to mind).
02 February 2012
Breaking out is hard to do
One of the things I’d like to do for the iteration library is settle on a convention for breaking and continuing within loops. There is a bug on this issue (#1619) and it seems like the general approach is clear but some of the particulars are less so. So I thought I’d try to enumerate how code will look under the various alternatives and then maybe we can settle on one: they’re all fairly similar. Who knows, maybe just writing things out will settle my mind.
(read more...)02 February 2012
Cross-crate inlining
Cross-crate inlining (CCI) refers to the ability to inline a function
across crate boundaries. In Rust, a “crate” is the unit of
compilation, rather than an individual file as in C or C++. A crate
basically corresponds to a single library or executable, but it may
contain any number of modules and source files internally. CCI is
important for performance due to the ubiquitous use of small methods
like vec::iter() in our source code. Such methods have proven to be
a very scalable way to define iteration abstracts, but performance is
currently somewhat lacking.
01 February 2012
Update
It’s been a while since I wrote anything on the blog! A lot has been going on in the meantime, both in Rust, parallel JavaScript, and personally…I hate to write a big update post but I gotta’ catch up somehow!
(read more...)11 January 2012
Proposed JS parallelism vs actors
In one of the comments on yesterday’s post, Tushar Pokle asked why I would champion my model over an Erlang model of strict data separation. There are several answers to this question. The simplest answer is that Web Workers already provide an actors model, though they do not make tasks particularly cheap (it’s possible to work around this by creating a fixed number of workers and sending tasks for them to execute).
(read more...)09 January 2012
Parallel Javascript
Lately the ideas for a parallel, shared memory JavaScript have begun to take shape. I’ve been discussing with various JavaScript luminaries and it seems like a design is starting to emerge. This post serves as a documentation of the basic ideas; I’m sure the details will change as we go along.
(read more...)29 December 2011
Composing blocks
The original Rust design included iterators very similar to Python’s generators. As I understand it, these were stripped out in favor of Ruby-esque blocks, partially because nobody could agree on the best way to implement iterators. I like blocks, but it seems like it’s more natural to compose iterators, so I wanted to think a bit about how one might use blocks to achieve similar things. I’m sure this is nothing new; there must be hundreds of libraries in Haskell that do the same things I’m talking about here.
(read more...)29 December 2011
Block sugar in expressions
UPDATE: I found some more complications. Updates inline.
(read more...)21 December 2011
Tone and criticism
So, I worry that my various posts about Rust give the impression that I’m dissatisfied with the language. It’s true that there are several things I’d like to change—and those are what I’ve focused on—but I want to clarify that I quite like Rust the way it is and I find the overall feel of the language to be very good. When it comes to the big decisions, I think Rust gets it right:
(read more...)20 December 2011
Dynamic race detection
In the context of thinking about parallelism for Rust, I have been reminded of an older idea I had for a lightweight, predictable dynamic race detection monitoring system based around block-scoped parallelism. I should think this would be suitable for (an extended version of) a dynamic language like Python, JavaScript, or Lua. I will write in a Python-like syntax since I know it best, but I am debating about exploring this for JavaScript.
(read more...)16 December 2011
Implementing unique closures
I landed a preliminary version of unique closures (which I am currently calling sendable fns) on the trunk last night. I wanted to briefly document what I did to alter the design of closures to get this working (of course there is a comment in the code too, but who reads that?).
(read more...)13 December 2011
Const vs Mutable
I keep thinking about parallel blocks although I know I probably shouldn’t; but so long as I write these notes while rustc builds, everybody wins, right?
(read more...)13 December 2011
Partially ordered unique closures
On a call with other Rust developers, I realized that I was thinking about unique closures all wrong. I had in mind a total ordering:
(read more...)12 December 2011
Challengines implementing unique closures
Update: See the recent post addressing the solution to this problem.
(read more...)09 December 2011
Pure blocks
I’ve been thinking a lot about “parallel blocks” recently and I am
beginning to think they can be made to work very simply. The main
thing that is needed is a type qualifier const that means
“read-only”. This would be a type prefix with very low precedence,
just like immutable and shared in D. The type const T
would refer to an instance of T that cannot be modified. This is a
deep property, so, given some record types defined like:
08 December 2011
Why not modes?
Marijn asked me what it is that I dislike about parameter modes. I thought I might as well explain here.
(read more...)08 December 2011
Sendable functions
One of the last remaining tasks for Rust 0.1 is to find a way to address issues #1128 and #1038. The key problem is that, right now, we can only spawn a task with a bare function, which is to say a function that carries no closure or environment. Due to the way that Rust is implemented, this even excludes generic functions. I have been wanting to lift this restriction but have been stymied by trying to make it accessible.
(read more...)07 December 2011
Rust usability problems
I recently implemented a new hashtable module for Rust. This was actually the first piece of code written in Rust which I started from scratch. I ran into some difficulties. Some of these are just things you have to get used to; some might be worth trying to correct. Just not entirely sure which problem falls into which category yet.
(read more...)06 December 2011
Coroutines for Rust
I have been thinking about unique closures, one of the last blocker items for the Rust 0.1 release. The original idea of a unique closure was that it should be a closure that can only access uniquely owned state, and which can therefore be sent to other tasks. However, I’ve come to have my doubts about this idea. In particular, a unique closure, like any closure, is a function, and can therefore be invoked many times: but this imposes some limits on what such a closure can safely do.
(read more...)05 December 2011
Statement-level parallelism
The primary means of parallel programming in Rust is tasks. Our task support is good: as good or better than any other language I’ve seen (good support for unique types and unique closures) but we have virtually no support for intra-task parallelism. The classic example is iterating over an array and processing each element in parallel. To be fair, this is a hard problem.
(read more...)03 December 2011
Rust without implicit copies
I just posted a draft of a proposal for Rust that aims to eliminate implicit copies. At the moment, it is not the final version; there are some flaws I need to correct. For one thing, I need to address implicit capturing of variables by lambdas.
(read more...)02 December 2011
Why case classes are better than variant types
One of the better features from functional programming languages are variant types (a.k.a. algebraic data types). Basically they are a way of enumerating a small set of possibilities and then making sure that you handle every possible case. However, in real world use variant types tend to run into a few annoying problems. While working on the Harmonic compiler, I found that Scala’s case classes addressed some of these shortcomings.
(read more...)