Merge pull request #27 from nikomatsakis/vision-document-pr

second draft of vision doc process

Author: nikomatsakis

rfc-drafts/stream.md

@@ -247,7 +247,7 @@ trait Stream {
 
 Unfortunately, async methods in traits are not currently supported,
 and there [are a number of challenges to be
-resolved](https://rust-lang.github.io/wg-async-foundations/design_notes/async_fn_in_traits.html)
+resolved](https://rust-lang.github.io/wg-async-foundations/design_docs/async_fn_in_traits.html)
 before they can be added. 
 
 Moreover, it is not clear yet how to make traits that contain async
@@ -309,7 +309,7 @@ design issues] to be resolved before they are added. Therefore, we've
 decided to enable progress on the stream trait by stabilizing a core,
 and to come back to the problem of extending it with combinators.
 
-[outstanding design issues]: https://rust-lang.github.io/wg-async-foundations/design_notes/async_closures.html
+[outstanding design issues]: https://rust-lang.github.io/wg-async-foundations/design_docs/async_closures.html
 
 This path does carry some risk. Adding combinator methods can cause
 existing code to stop compiling due to the ambiguities in method

src/SUMMARY.md

@@ -1,20 +1,29 @@
 # Summary
 
-- [Welcome](./welcome.md)
-- [Design notes](./design_notes.md)
-    - [Yield-safe lint](./design_notes/yield_safe.md)
-    - [Stream trait](./design_notes/stream.md)
-    - [Generator syntax](./design_notes/generator_syntax.md)
-    - [AsyncRead, AsyncWrite traits](./design_notes/async_read_write.md)
-    - [Async fn in traits](./design_notes/async_fn_in_traits.md)
-    - [Mutex (future-aware)](./design_notes/mutex.md)
-    - [Async aware channels](./design_notes/channels.md)
-    - [Async closures](./design_notes/async_closures.md)
-    - [Join combinator](./design_notes/join.md)
-    - [Select combinator](./design_notes/select.md)
-    - [Sink trait](./design_notes/sink_trait.md)
-    - [Async main](./design_notes/async_main.md)
-    - [Async drop](./design_notes/async_drop.md)
-       - [Async lifecycle](./design_notes/async_lifecycle.md)
-    - [Completion-based futures](./design_notes/completion_based_futures.md)
-
+- [👋🏽 Welcome](./welcome.md)
+- [🔮 The vision](./vision.md)
+   - [✏️ Design tenets for async](./vision/tenets.md)
+   - [🙋‍♀️ Cast of characters](./vision/characters.md)
+   - [😱 Status quo](./vision/status_quo.md)
+   - [✨ Shiny future](./vision/shiny_future.md)
+   - [📅 Roadmap for 2021](./vision/roadmap.md)
+   - [❓How to vision doc](./vision/how_to_vision_doc.md)
+- [🔍 Triage meetings](./triage.md)
+- [🔬 Design docs](./design_docs.md)
+    - [⚠️ Yield-safe lint](./design_docs/yield_safe.md)
+    - [☔ Stream trait](./design_docs/stream.md)
+    - [⚡ Generator syntax](./design_docs/generator_syntax.md)
+    - [📝 AsyncRead, AsyncWrite traits](./design_docs/async_read_write.md)
+    - [🧬 Async fn in traits](./design_docs/async_fn_in_traits.md)
+    - [🔒 Mutex (future-aware)](./design_docs/mutex.md)
+    - [📺 Async aware channels](./design_docs/channels.md)
+    - [📦 Async closures](./design_docs/async_closures.md)
+    - [🤝 Join combinator](./design_docs/join.md)
+    - [🤷‍♀️ Select combinator](./design_docs/select.md)
+    - [🚰 Sink trait](./design_docs/sink_trait.md)
+    - [🎇 Async main](./design_docs/async_main.md)
+    - [🗑️ Async drop](./design_docs/async_drop.md)
+       - [♻️ Async lifecycle](./design_docs/async_lifecycle.md)
+    - [⏳ Completion-based futures](./design_docs/completion_based_futures.md)
+- [💬 Conversations](./conversations.md)
+   - [🐦 2021-02-12 Twitter thread](./conversations/2021-02-12-Twitter-Thread.md)

src/conversations.md

@@ -0,0 +1,9 @@
+# 💬 Conversations
+
+This section contains notes and summaries from conversations that we have had with people are using Rust and async and describing their experiences. These conversations and links are used as "evidence" when building [the "status quo" section][sq].
+
+[sq]: vision/status_quo.md
+
+## Not exhaustive nor mandatory
+
+This section is not meant to be an "exhaustive list" of all sources. That would be impossible. Many conversations are short, not recorded, and hard to summaize. Others are subject to NDA. We certainly don't require that all claims in the [status quo][sq] section are backed by evidence found here. Still, it's useful to have a place to dump notes and things for future reference.

src/conversations/2021-02-12-Twitter-Thread.md

@@ -0,0 +1,50 @@
+# 🐦 2021-02-12 Twitter thread
+
+Notes taken from the thread in response to [Niko's tweet](https://twitter.com/nikomatsakis/status/1359454255971770372).
+
+* [Enzo](https://twitter.com/enzo_mdd/status/1359544617121820676)
+    * A default event loop. "choosing your own event loop" takes time, then you have to understand the differences between each event loop etc.
+    * Standard way of doing `for await (variable of iterable)` would be nice.
+    * Standard promise combinators.
+* [creepy_owlet](https://twitter.com/creepy_owlet/status/1359649695103131649)
+    * https://github.com/dtantsur/rust-osauth/blob/master/src/sync.rs
+* async trait --
+    * https://twitter.com/jcsp_tweets/status/1359820431151267843
+    * "I thought async was built-in"?
+    * nasty compiler errors
+    * [ownership puzzle](https://www.fpcomplete.com/blog/ownership-puzzle-rust-async-hyper/) blog post
+* [rubdos](https://twitter.com/rubdos/status/1359462402702606336)
+    * [blog post](https://www.rubdos.be/corona/qt/rust/tokio/actix/2020/05/23/actix-qt.html) describes integrating two event loops
+    * mentions desire for runtime independent libraries
+    * qt provides a mechanism to integrate one's own event loop
+    * [llvm bug](https://github.com/rust-lang/rust/issues/60605) generates invalid arm7 assembly
+* [alexmiberry](https://twitter.com/alexmiberry/status/1359559299161325581)
+    * kotlin/scala code, blocked by absence of async trait
+* helpful blog post
+    * [jamesmcm](http://jamesmcm.github.io/blog/2020/05/06/a-practical-introduction-to-async-programming-in-rust/)
+        * note that `join` and `Result` play poorly together
+            * [async-std version](https://github.com/jamesmcm/async-rust-example/blob/async-std/client_async/src/main.rs#L50-L59)
+            * [tokio version](https://github.com/jamesmcm/async-rust-example/blob/master/client_async/src/main.rs#L40-L61) has some wild "double question marks" -- I guess that spawn must be adding a layer of `Result`?
+    * the post mentions rayon but this isn't really a case where one ought to use rayon -- still, Rayon's APIs here are SO much nicer :)
+    * [rust aws and lambda](http://jamesmcm.github.io/blog/2020/04/19/data-engineering-with-rust-and-aws-lambda/#en)
+* [issue requiring async drop](https://github.com/jamesmcm/s3rename/issues/16)
+* [fasterthanlime](https://fasterthanli.me/articles/getting-in-and-out-of-trouble-with-rust-futures) -- 
+    * this post is amazing
+    * the discussion on Send bounds and the ways to debug it is great
+* [bridging different runtimes using GATs](https://github.com/thanethomson/async-channel-abs/blob/master/src/runtime.rs)
+* first server app, [great thread with problems](https://twitter.com/richardsabow/status/1345815109201842178)
+    * "I wasn't expecting that it will be easy but after Go and Node.js development it felt extremely hard to start off anything with Rust."
+    * "felt like I have to re-learn everything from scratch: structuring project and modules, dependency injection, managing the DB and of course dealing with concurrency"
+    * common thread: poor docs, though only somewhat in async libraries
+        * I had enums in the DB and it was a bit more complex to map them to my custom Rust enums but I succeeded with the help of a couple of blog posts – and not with Diesel documentation
+        * I used Rusoto for dealing with AWS services. It's also pretty straightforward and high quality package – but again the documentation was sooo poor. 
+* [implaustin](https://t.co/4rlyfUlFES?amp=1) wrote a [very nice post](https://t.co/4rlyfUlFES?amp=1) but it felt more like a "look how well this worked" post than one with actionable feedback
+    * "Async has worked well so far.  My top wishlist items are Sink and Stream traits in std.  It's quite difficult to abstract over types that asynchronously produce or consume values."
+    * "AsyncRead/AsyncWrite work fine for files, tcp streams, etc.  But once you are past I/O and want to pass around structs, Sink and Stream are needed.  One example of fragmentation is that Tokio channels used to implement the futures Sink/Stream traits, but no longer do in 1.0."
+    * "I usually use Sink/Stream to abstract over different async channel types.  Sometimes to hide the details of external dependencies from a task (e.g. where is this data going?).  And sometimes to write common utility methods."
+    * "One thing I can think of: there are still a lot of popular libraries that don't have async support (or are just getting there).  Rocket, Criterion, Crossterm's execute, etc."
+* [EchoRior](https://twitter.com/EchoRior/status/1359964691305496579):
+    * "I've written a bit of rust before, but rust is my first introduction to Async. My main gripes are that it's hard to figure our what the "blessed" way of doing async is. I'd love to see async included in the book, but I understand that async is still evolving too much for that."
+    * "Adding to the confusion: theres multiple executors, and they have a bit of lock in. Async libraries being dependent on which executor version I use is also confusing for newcomers. In other langs, it seems like one just uses everything from the stdlib and everything is compatible"
+    * "That kind of gave me a lot of hesitation/fomo in the beginning, because it felt like I had to make some big choices around my tech stack that I felt I would be stuck with later. I ended up chatting about this in the discord & researching for multiple days before getting started."
+    * "Also, due to there not being a "blessed" approach, I don't know if I'm working with some misconceptions around async in rust, and will end up discovering I will need to redo large parts of what I wrote."

src/design_docs.md

@@ -0,0 +1,19 @@
+# 🔬 Design documents
+
+The **design documents** (or "design docs", more commonly) describe potential designs. These docs vary greatly in terms of their readiness to be implemented:
+
+* Early on, they describe a vague idea for a future. Often this takes the shape of capturing constraints on the solution, rather than the solution itself.
+* When a feature is getting ready to ship, they can evolve into a full blown RFC, with links to tracking issues or other notes.
+
+## Early stage design docs
+
+In the early stages, design docs are meant to capture interesting bits of "async design space". They are often updated to capture the results of a fruitful conversation or thread which uncovered contraints or challenges in solving a particular problem. They will capture a combination of the following:
+
+* use cases;
+* interesting aspects to the design;
+* alternatives;
+* interactions with other features.
+
+## Late stage design docs
+
+As a design progresses, the doc should get more and more complete, until it becomes something akin to an RFC. (Often, at that point, we will expand the design document into a directory, adding an actual RFC draft and other contents; those things can live in this repo or elsewhere, depending.) Once we decide to put a design doc onto the roadmap, it will also contain links to tracking issues or other places to track the status.

src/design_docs/async_closures.md

@@ -0,0 +1 @@
+# 📦 Async closures

src/design_docs/async_drop.md

@@ -0,0 +1 @@
+# 🗑️ Async drop

src/design_notes/async_fn_in_traits.md renamed to src/design_docs/async_fn_in_traits.md

@@ -1,4 +1,4 @@
-# Async fn in traits
+# 🧬 Async fn in traits
 
 * [Why async fn in traits are hard][wafth]
 
@@ -81,3 +81,8 @@ trait MyMethod {
     async fn foo(&self);
 }
 ```
+
+## 🤔 Frequently Asked Questions
+
+* **What do people say about this to their friends on twitter?**
+    * (Explain your key points here)

src/design_docs/async_lifecycle.md

@@ -0,0 +1 @@
+# ♻️ Async lifecycle

src/design_docs/async_main.md

@@ -0,0 +1,7 @@
+# 🎇 Async main
+
+## What is it?
+
+## Motivation
+
+## Frequently Asked Questions