Async interviews: my take thus far
30 April 2020
The point of the async interview series, in the end, was to help figure out what we should be doing next when it comes to Async I/O. I thought it would be good then to step back and, rather than interviewing someone else, give my opinion on some of the immediate next steps, and a bit about the medium to longer term. I’m also going to talk a bit about what I see as some of the practical challenges.
Focus for the immediate term: interoperability and polish
At the highest level, I think we should be focusing on two things in the “short to medium” term: enabling interoperability and polish.
By interoperability, I mean the ability to write libraries and
frameworks that can be used with many different executors/runtimes.
Adding the Future
trait was a big step in this direction, but
there’s plenty more to go.
My dream is that eventually people are able to write portable async apps, frameworks, and libraries that can be moved easily between async executors. We won’t get there right away, but we can get closer.
By polish, I mean “small things that go a long way to improving quality of life for users”. These are the kinds of things that are easy to overlook, because no individual item is a big milestone.
Polish in the compiler: diagnostics, lints, smarter analyses
Most of the focus of wg-async-foundations recently has been on
polish work on the compiler, and we’ve made quite a lot of
progress. Diagnostics have notably improved, and we’ve been
working on inserting helpful suggestions, fixing compiler
bugs, and improving efficiency. One thing I’m especially excited
about is that we no longer rely on thread-local storage in the async fn
transformation, which means that async-await is now
compatible with #[no_std]
environments and hence embedded
development.
I want to give a ๐ “shout-out” ๐ to ๐ tmandry ๐ for leading this polish effort, and to point out that if you’re interested in contributing to the compiler, this is a great place to start! Here are some tips for how to get involved.
I think it’s also a good idea to be looking a bit more broadly. On
Zulip, for example, LucioFranco suggested that we could add a
lint to warn about things that should not be live across yields (e.g.,
lock guards), and I think that’s a great idea (there is a clippy
lint already, though it’s specific to MutexGuard
; maybe this should
just be promoted to the compiler and generalized).
Another, more challenging area is improving the precision of the
async-await transformation and analysis. Right now, for example, the
compiler “overapproximates” what values are live across a yield, which
sometimes yields spurious errors about whether a future needs to be
Send
or not. Fixing this is, um, “non-trivial”, but it would be a
major quality of life improvement.
Polish in the standard library: adding utilities
When it comes to polish, I think we can extend that focus beyond the
compiler, to the standard library and the language. I’d like to see
the stdlib include building blocks like async-aware mutexes and
channels, for example, as well as smaller utilities like
task::block_on
. YoshuaWuyts recently proposed adding some simple
constructors, like future::{pending, ready}
which I think could
fit in this category. A key constraint here is that these should be
libraries and APIs that are portable across all executors and
runtimes.
Polish in the language: async main, async drop
Polish extends to the language, as well. The idea here is to find
small, contained changes that fix specific pain points or limitations.
Adding async fn main
, as boats proposed, might be such an example
(and I rather like the idea of #[test]
that XAMPRocky proposed on
internals).
Another change I think makes sense is to support async destructors, and I would go further and adopt find some solution to the concerns about RAII and async that Eliza Weisman raised. In particular, I think we need some kind of (optional) callback for values that reside on a stack frame that is being suspended.
Supporting interoperability: the stream trait
Let me talk a bit about what we can do to support interoperability.
The first step, I think, is to do as Carl Lerche proposed and add
the Stream
trait into the standard library. Ideally, it would be
added in exactly the form that it takes in futures 0.3.4, so that we
can release a (minor) version of futures that simply re-exports the
stream trait from the stdlib.
Adding stream enables interoperability in the same way that adding
Future
did: one can now define libraries that produce streams, or
which operate on streams, in a completely neutral fashion.
But what about “attached streams”?
I said that I did not think adding Stream
to the standard library
would be controversial. This does not mean there aren’t any concerns.
cramertj, in particular, raised a concern about the desire for
“attached streams” (or “streaming streams”), as they are sometimes
called.
To review, today’s Stream
trait is basically the exact async analog
of Iterator
. It has a poll_next
method that tries to fetch the
next item. If the item is ready, then the caller of poll_next
gets
ownership of the item that was produced. This means in particular that
the item cannot be a reference into the stream itself. The same is
true of iterators today: iterators cannot yield references into
themselves (though they can yield references into the collection
that one is iterating over). This is both useful (it means that
generic callers can discard the iterator but keep the items that were
produced) and a limitation (it means that iterators/streams cannot
reuse some internal buffer between iterations).
We should not block progress on streams on GATs
I hear the concern about attached streams, but I don’t think it should block us from moving forward. There are a few reasons for this. The first is pragmatic: fully resolving the design details around attached streams will require not only GATs, but experience with GATs. This is going to take time and I don’t think we should wait. Just as iterators are used everywhere in their current form, there are plenty of streaming appplications for which the current stream trait is a good fit.
Symmetry between sync and async is a valuable principle
There is another reason I don’t think we should block progress on
attached streams. I think there is a lot of value to having symmetric
sync/async versions of things in the standard library. I think boats
had it right when they said that the guiding vision for Async I/O in
Rust should be that one can take sync code and make it async by adding
in async
and await
as necessary.
This isn’t to say that everything between sync and async must be the same. There will likely be things that only make sense in one setting or another. But I think that in cases where we see orthogonal problems – problems that are not really related to being synchronous or asynchronous – we should try to solve them in a uniform way.
In this case, the problem of “attached” vs “detached” is orthogonal from being async or sync. We want attached iterators just as much as we want attached streams – and we are making progress on the foundational features that will enable us to have them.
Once we have those features, we can design variants of Iterator
and
Stream
that support attached iterators/streams. Perhaps these
variants will deprecate the existing traits, or perhaps they will live
alongside them (or maybe we can even find a way to extend the existing
traits in place). I don’t know, but we’ll figure it out, and we’ll do
it for both sync and async applications, well, synchronously1.
Supporting interoperability: adding async read and write traits
I also think we should add AsyncRead
and AsyncWrite
to the
standard library, also in roughly the form they have today in
futures. In short, stable, interoperable traits for reading and writing enables
a whole lot of libraries and middleware. After all, the main reason
people are using async is to do I/O.
In contrast to Stream
, I do expect this to be controversial, for a
few reasons. But much like Stream
, I still think it’s the right
thing to do, and actually for much the same reasons.
First concern about async read: uninitialized memory
I know of two major concerns about adding AsyncRead
and
AsyncWrite
. The first is around uninitialized memory. Just like
its synchronous counterpart Read
, the AsyncRead
trait must be
given a buffer where the data will be written. And, just like
Read
, the trait currently requires that this buffer must be zeroed
or otherwise initialized.
You will probably recognize that this is another case of an “orthogonal problem”. Both the synchronous and asynchronous traits have the same issue, and I think the best approach is to try and solve it in an analogous way. Fortunately, sfackler has done just that. The idea that we discussed in our async interview is slowly making its way into RFC form.
So, in short, I think uninitialized memory is a “solved problem”, and moreover I think it was solved in the right way. Happy days.
Second concern about async read: io_uring
This is a relatively new thing, but a new concern about AsyncRead
and AsyncWrite
is that, fundamentally, they were designed around
epoll
-like interfaces. In these interfaces, you get a callback
when data is ready and then you can go and write that data into a
buffer. But in Linux 5.1 added a new interface, called io_uring
, and
it works differently. I won’t go into the details here, but boats
gives a good intro in their blog post introducing the iou
library.
My take here is somewhat similar to my take on why we should not block
streams on GATs: io_uring
is super promising, but it’s also super
new. We have very little experience trying to build futures atop
io_uring
. I think it’s great that people are experimenting, and I
think that we should encourage and spread those experiments. After
some time, I expect that “best practices” will start to emerge, and at
that time, we should try to codify those best practices into traits
that we can add to the standard library.
In the meantime, though, epoll is not going anywhere. There will
always be systems based on epoll that we will want to support, and we
know exactly how to do that, because we’ve spend years tinkering with
and experimenting with the AsyncRead
and AsyncWrite
. It’s time
to standardize them and to allow people to build I/O libraries based
on them. Once we know how best to handle io_uring
, we’ll integrate
that too.
All of that said, I would really like to learn more about io_uring
and what it might mean, since I’ve not dug that deeply here. Maybe a
good topic for a future async interview!
Looking further out
Looking further out, I think there are some bigger goals that we
should be thinking about. The largest is probably adding some form of
generator syntax. Anecdotally, I definitely hear about a fair
number of folks working with streams and encountering difficulties
doing so. As boats said, writing Stream
implementations is a
common reason that people have to interact directly with Pin
, and
that’s something we want to minimize. Further, in a synchronous
setting, generator syntax would also give us syntactic support for
writing iterators, which would benefit Rust overall. Enabling
support for async functions in traits would also be high on my list,
along with async closures. (The latter in particular would enable
us to bring in a lot more utility methods and combinators for futures
and streams, which would be great.)
I think though that it’s worth waiting a bit before we pursue these, for several reasons.
- Generator syntax would build on a
Stream
trait anyhow, so having that in the standard libary is an obvious first step. - There is ongoing work on GATs and chalk integration in the context of wg-traits, and we’re making quite rapid progress there. The above items all potentially interact with GATs in some way, and it’d be nice if we had more of an implementation available before we started in on them (though it may not be a hard requirement).
- Quite frankly, we don’t have the bandwidth. We need to work on building up an effective wg-async-foundations group before we can take on these sorts of projects. More on this point later.
Related and supporting efforts
There are a few pending features in the language team that I think may be pretty useful for async applications. I won’t go into detail here, but briefly:
impl Trait
everywhere – finishing up theimpl Trait
saga will enable us to encode some cases where async fn in traits might be nice, such as Tower’sService
trait;- GATs, obviously – GATs arise around a number of advanced features.
- procedural macros – we’ve been making slow and steady progress on
stabilizing bits and pieces of the procedural macro story, and I
think it’s a crucial enabler for async-related applications (and
many others). Things like the
#[runtime::main]
andasync-trait
crate are only possible because of the procedural macro support. Both Carl and Eliza brought up the importance of offering procedural macros in expression position without requiring things likeproc_macro_hack
.
I’ll write more about these points in other posts, though.
Summing up: the list
To summarize, here is my list of what I think we should be doing in “async land” as our next steps:
- Continued polish and improvements to the core compiler implementation.
- Lints for common “gotchas”, like
#[must_use]
to help identify “not yield safe” types. - Extend the stdlib with mutexes, channels,
task::block_on
, and other small utilities. - Extend the
Drop
trait with “lifecycle” methods (“async drop”). - Add
Stream
,AsyncRead
, andAsyncWrite
traits to the standard library.
To be clear, this is a proposal, and I am very much interested in feedback on it, and I wouldn’t surprised to add or remove a thing or two. However, it’s not an arbitrary proposal: It’s a proposal that I’ve given a fair amount of thought to, and I feel reasonably certain about it.
There are a few things I’d be particularly interested to get feedback on:
- If you maintain a library, what are some of the challenges you’ve encountered in making it operate generically across executors? What could help there?
- Do you have ideas for useful bits of polish? Are there small changes or stdlib additions that would make everyday life that much easier?
A challenge: growing an effective working group
I want to close with a few comments on organization. One of the things we’ve been trying to figure out is how best to organize ourselves and create a sustainable working group.
Thus far, tmandry has been doing a great job at organizing the polish work that has been our focus, and I think we’ve been making good progress there, although there’s always a need for more folks to help out. (Shameless plug: Here are some tips for how to get involved!)
If we want to go beyond polish and get back to adding things to the
standard library, especially things like the Stream
or AsyncRead
trait, we’re going to have to up our game. The same is true for some
of the more diverse tasks that fall under our umbrella, such as
maintaining the async book.
To do those tasks, we’re going to need more than coders. We need to take the time to draft designs, incorporate feedback, write the RFCs, and push things through to stabilization.
To be honest, I’m not entirely sure where that work is going to come
from – but I believe we can do it! If this is something you’re
interested in, definitely drop in the #wg-async-foundations
stream
on Zulip and say hello, and monitor the Inside Rust, as I expect
we’ll be posting updates there from time to time.
Comments?
As always, please leave comments in the async interviews thread
on users.rust-lang.org
.
Footnotes
I couldn’t resist. ↩︎