[IDEA] Expose low-level socket operations

[ntninja]

Currently mio exposes two sets of APIs for dealing with network operations:

1. The high-level TCP, UDP & UDS interface
2. The very low-level and platform-specific Unix fd polling and windows handle registration interfaces

However for many operations (such as maintaining a Bluetooth RFCOMM/SPP connection or talking the the kernel using netlink) that will never be part of mio core, currently the only option is to go "really low-level", and also unsafe on Windows, to implement these things. (Please, by all means, correct me if I'm wrong!)

In particular the following operations are asynchronously implementable in a safe and cross-platform way for any socket type:

• Connecting a socket: connect (ConnectEx on Windows)
• Reading data from socket: recv, recvfrom, recvmsg (WSARecv* on Windows)
• Writing data to socket: send, sendto, sendmsg (WSASend* on Windows)
• Accepting new client connections: accept (AcceptEx on Windows)

Looking a mio's current architecture implementing this would probably take the form of a bunch of new structs that take a socket fd/handle and each implement mio::Evented allowing them to be polled as usual, becoming ready when they have something to return. This whole process also happens to be at the essence of the TCP and UDP implementations already do internally anyways. The API can also easily be designed in a way that preserves the meaning of the interest bits by grouping together related operations, such as recv/send (what TCP currently does), recvfrom/sendto (like UDP) and possibly even connect/accept (as sockets become writable when they are fully connected and become "readable" when they have new incoming clients). Layering the current TCP/UDP/UDS on top of such an interface (or maybe even moving them to another crate because they are "too high-level" for mio core itself) would be quite trivial and would allow other, lesser-used protocols, to reuse the same facilities.

Do you agree that this is something that mio core should support (if patches, … are provided)?

[ntninja]

Just so you know I may actually implement this so it would be very useful to know whether it makes more sense to attempt to implement this on top of or in mio itself. Therefor I'd be very interested to hear your thoughts about this.

[carllerche]

I think what you are describing existed in a rougher form in past versions of Mio. Specifically Io which is currently private.

I think what you are describing is useful, but I am not sure if it is something that should be made public. The goal right now is to provide a compatibility layer to the OS's polling platform. TCP / UDP are also provided because they are so common, but I would like to limit additions as much as possible, especially if they can live in third party crates (in part so that I have less to maintain :)).

You should definitely give it a shot in a separate crate and we can evaluate if it makes sense to include after that.

[carllerche]

I'm going to close the issue for now. If you have more to discuss, you can continue here or open a new issue. Cheers.

[ntninja]

Thanks for your response! (Including the link to the private io module!)

That turned out way too long, so here's the TL;DR:

1. I reconsidered the design this thing should have and came to the conclusion that it would make more sense to expose StreamSocket, DatagramSocket, RawSocket and ListeningSocket (with APIs similar to TCPStream, UDPStream, ??? and TCPListener).
2. Reusing the current libstd-based code bits for socket-interaction would make sense, but cannot be done in practice, because the public API is limited to TCP & UDP over IP world
3. Would you accept a PR that rewrites/copies these parts from libstd and splits the TCP/UDS, UDP & TCPListener into the protocol-agnostic part mentioned above and their current implementation? And is it really necessary that everybody has to do this kind of thing, because in libstd socket=IP?

Since my opening post I've put some more thought into how it would make sense to implement this in the context of mio and came to the conclusion that it would probably be better to not expose the raw recv/send, recvfrom/sendto, … functions directly, but instead offer an a bit more high-level API for interacting with any kind of StreamSocket, DatagramSocket, RawSocket or ListeningSocket. Doing things this way means that it is not necessary for mio's source code to change in any significant way, while making Windows support and usage of the provided facilities for clients much simpler.

My attempt at designing an the public-facing API for the StreamSocket ended up with an API that was remarkably similar to that of the current mio::tcp::TCPStream, but with the following important changes:

1. No connect* methods (technically feasible, but currently somewhat out-of-scope)
2. No usage of std::net::TcpStream (since it is TCP only), instead a raw SOCKET/FD would be stored
3. All called wrapper functions from std::net::TcpStream have to replaced with their libc equivalents

It seems however that (2) and (3) are actually less of a requirement than it might seem at at first, because their implementation in libstd only does things that would work on any stream socket (not just TCP), unfortunately this fact is of no use in practice because (a) compatibility with non-TCP sockets is not guaranteed and (b) several of the exposed data types (for local_addr and peer_addr for instance) where written for a world where only IPv4 & IPv6 with TCP & UDP transports exist… (What would we have done if Rust had came into existence 10 years earlier without IPv6 support because it was "not important enough"!?)

My conclusion from this is that the main problem in supporting other socket types lies not so much in mio itself, but in the fact that the standard library divides the (networking) world into IPv4 & IPv6 with exactly one stream and datagram protocol (no exceptions allowed), which mio then faithfully re-exposes in an asynchronous way. I also noticed that you worked around this fact in the mio::uds code by implementing parts of the code on top of std::io::File and parts using libc directly (the code looks remarkably similar to the internal std::net code, btw 😉).

Is there any API of non-Inet sockets in libstd planned, or are we stuck with re-implementing whatever libstd does every time there is some socket-related usecase that isn't a perfect match for what it provides? The Rust RFC #923 seems to indicate that std::net was only ever designed for TCP & UDP based IP sockets and should probably have been named std::net::inet instead. (For which it is quite a well-done design.) If that is indeed the case then building on top of the standard library for this use-case was a bad idea from the start, I suppose?

[carllerche]

Sounds like you are making some progress. I am not really following everything, but as long as you can, I would try to experiment in a standalone crate. If you hit issues that prevent you from doing so (that require mio changes), open a new issue.

Cheers.

[ntninja]

Well, my idea would basically be to generalize the current TCP and UDP specific code to work for arbitrary stream and datagram sockets. This is a change much more in name and in the documented guarantees than in actual code. The main problem with this is that it would result in some, IMHO needless, duplication of code from libstd which only guarantees and in some places even considers support for IPv4 and IPv6 TCP and UDP sockets. Therefor I still need an answer to the questions in (TL;DR / 3):

• Would you accept a PR that copies the relevant socket-related code snippets from libstd and splits the current TCP/UDS, UDP & TCPListener into a protocol-agnostic part for stream/datagram sockets and a protocol-specific part?
• And is it really necessary that everybody has reimplement generic socket abstractions, because in libstd socket actually means IP-socket?

Thanks for your consideration!

[carllerche]

At this point, I would rather not extend the API surface. More API surface means more code to maintain :-)