Since it lives on top of UDP, I believe all you need is SOCK_DGRAM, right? The rest of QUIC can be in a userspace library ergonomically designed for your programming language e.g. https://github.com/quinn-rs/quinn - and can interoperate with others who have made different choices.
> Since it lives on top of UDP, I believe all you need is SOCK_DGRAM, right? The rest of QUIC can be in a userspace library ergonomically designed for your programming language e.g. (...)
I think that OP's point is that there's no standard equivalent of the BSD sockets API for writing programs that communicate over QUIC, which refers to the userspace library you've referred to.
A random project hosted in GitHub is not the same as a standard API.
Did they not answer that question? It uses the BSD sockets API with SOCK_DGRAM?
Right, that random project is not a standard API - its built using a standard API. You wouldn't expect BSD sockets to have HTTP built in... so you can find third-party random projects for HTTP implmented with BSD sockets just like you can find QUIC implmented with BSD sockets.
QUIC is roughly TCP-equivalent not HTTP-equivalent, and we do have a BSD sockets API for TCP. You might be thinking of HTTP/3 rather than QUIC; HTTP/3 actually is HTTP-equivalent.
You can turn the OP's question around. Every modern OS kernel provides an efficient, shared TCP stack. It isn't normal to implement TCP separately in each application or as a userspace library, although this is done occasionally. Yet we currently expect QUIC to be implemented separately in each application, and the mechanisms which are in the OS kernel for TCP are implemented in the applications for QUIC.
So why don't we implement TCP separately in each application, the way it's done with QUIC?
Although there were some advantages to this while the protocol was experimental and being stabilised, and for compatibility when running new applications on older OSes, arguably QUIC should be moved into the OS kernel to sit alongide TCP now that it's stable. The benefit of having Chrome, Firefox et al stabilise HTTP/3 and QUIC were good, but that potentially changes when the protocol is stable but there are thousands of applications, each with their own QUIC implementation doing congestion control differently, scheduling etc, and no cooperation with each other the way the OS kernel does with TCP streams from concurrent applications. Currently we are trending towards a mix of good and poor QUIC implementations on the network (in terms of things like congestion control and packet flow timing), rather than a few good ones as happens with TCP because modern kernels all have good quality implementations of TCP.
> QUIC is roughly TCP-equivalent not HTTP-equivalent, and we do have a BSD sockets API for TCP. You might be thinking of HTTP/3 rather than QUIC; HTTP/3 actually is HTTP-equivalent.
No, I understand QUIC is a transport and HTTP/3 is the next HTTP protocol that runs over QUIC. I was saying QUIC can be userspace just like HTTP is userspace over kernel TCP API. We haven't moved HTTP handling into the kernel so what makes QUIC special?
I think it is just too early to expect every operating system to have a standard API for this. We didn't have TCP api's built-in originally either.
> What makes TCP special that we put it in the kernel? A lot more of those answers apply to QUIC than to HTTP.
I mean, that seems like GP's point. There's no profound reason TCP should be implemented in the kernel, besides trying to fit every form of IO into Unix's "everything is a file descriptor" philosophy.
If QUIC works in userspace, that's fine. If it ain't broke, don't fix it.
Huh, you think so? I'd be a bit surprised if that was the intent, because TCP obviously is in the kernel, so if their point blatantly disagrees with preexisting decisions in every OS then that's something that needs to be addressed before we can talk about QUIC.
> I was saying QUIC can be userspace just like (...)
I think you're too hung up on "can" when that's way besides OP's point. The point is that providing access to fundamental features through a standard API is of critical importance.
If QUIC is already massively adopted them there is no reason whatsoever to not provide a standard API.
If QUIC was indeed developed to support changes then there is even fewer arguments to not provide a standard API.
Isn't the point of QUIC to offer high performance and flexibility at the same time? For these requirements, a one-size-fits-all API is rarely the way to go, so individual user-space implementations make sense. Compare this to file IO: For, many programs, the open/read/write/close FD API is sufficient, but if you require more throughput or control, its better to use a lower-level kernel interface and implement the missing functionality in user-space, tailored to your particular needs.
It occurs to me that QUIC could benefit from a single kernel-level coordinator that can be plugged for cooperation - for instance, a dynamic bandwidth-throttling implementation a la https://tripmode.ch/ for slower connections where the coordinator can look at pre-encryption QUIC headers, not just the underlying (encrypted) UDP packets. So perhaps I was hasty to say that you just need SOCK_DGRAM after all!
There are dozen of libs to do it right now but I expect ultimately distro folks will want to consolidate it all to avoid redundant dependencies, so we'll probably end up with openssl/gnutls/libressl doing it eventually and most apps using that.
Note there were talks to add it in-kernel like kTLS, but since the certificate handling is difficult that part will be offloaded to userspace as it is with kTLS -- you won't ever get an interface like connect() and forget about it.
"QUIC improves performance of connection-oriented web applications that are currently using TCP.[QUIC] is designed to obsolete TCP at the transport layer for many applications, thus earning the protocol the occasional nickname "TCP/2"."
Also taken from the wiki page:
"QUIC aims to be nearly equivalent to a TCP connection but with much-reduced latency."
Here is the only relevant bit regarding TLS:
"As most HTTP connections will demand TLS, QUIC makes the exchange of setup keys and supported protocols part of the initial handshake process. When a client opens a connection, the response packet includes the data needed for future packets to use encryption. This eliminates the need to set up the TCP connection and then negotiate the security protocol via additional packets."
[1] https://en.m.wikipedia.org/wiki/QUIC
The RFCs for QUIC (RFC 9000 and RFC 9001) mandate encryption.
Some random stackoverflow answer[1] claims there are implementations that ignore this and allow "QUIC without encryption", but I'd argue that it's not QUIC anymore -- in my opinion it'd be harmful to implement in the kernel.
Per the RFC, QUIC is literally defined as a transport protocol... Literally the first sentence of the overview is:
"QUIC is a secure general-purpose transport protocol."
More importantly, QUIC literally bakes in TLS into how it works which is a far cry away from replacing it.
"The QUIC handshake combines negotiation of cryptographic and transport parameters. QUIC integrates the TLS handshake [TLS13], although using a customized framing for protecting packets."
> So why don't we implement TCP separately in each application, the way it's done with QUIC?
Because library/dependency management sucked at the time, and OSes competed for developers by trying to offer more libraries; on the other side we were less worried about EEE because cross-platform codebases were rare and usually involved extensive per-OS code with #ifdefs or the like.
I don't think we would or should put TCP in the kernel if it was being made today.
> So why don't we implement TCP separately in each application, the way it's done with QUIC?
Because TCP also does multiplexing, which must be handled in some central coordinating service. QUIC doesn't suffer from that since it offloads that to UDP.
You wouldn't use a system TLS implementation either (well, technically SChannel/NetworkTransport exist, but you're vastly better off ignoring them).
That's going to happen regardless, that's just the nature of Ethernet, or even just existing in reality. TCP has to deal with the same thing. Either way the solution is the same: acks and retransmissions.
> Did they not answer that question? It uses the BSD sockets API with SOCK_DGRAM?
No, that does not answer the question, nor is it a valid answer to the question. Being able to send UDP datagrams is obviously not the same as establishing a QUIC connection. You're missing the whole point of the importance of having a standard API to establish network connections.
> Right, that random project is not a standard API - its built using a standard API.
Again, that's irrelevant and misses the whole point. Having access to a standard API to establish QUIC connections is as fundamental as having access to a standard API to establish a TCP connection.
> so you can find third-party random projects for HTTP (...)
The whole point of specifying standard APIs is to not have to search for and rely on random third-party projects to handle a fundamental aspect of your infrastructure.
Not having a system API is the entire point of QUIC. The only reason QUIC needs to exist is because the sockets API and the system TCP stacks are too ossified to be improved. If you move that boundary then QUIC will inevitably suffer from the same ossification that TCP displays today.
No, the reason QUIC exists is that TCP is ossified at the level of middle boxes on the internet. If it had been possible to modify TCP with just some changes in the Linux, BSD and Windows kernels, it would have been done.
"""
A middlebox is a computer networking device that transforms, inspects, filters, and manipulates traffic for purposes other than packet forwarding. Examples of middleboxes include firewalls, network address translators (NATs), load balancers, and deep packet inspection (DPI) devices.
"""
I don't know about that. Without middlebox problems we might have used SCTP as a basis and upgraded it. But it's so different from TCP that I doubt we would have done it as a modification of TCP.
The alternative is that either browsers will be the only users of QUIC - or that each application is required to bring its own QUIC implementation embedded into the binary.
If ossification was bad if every router and firewall has its own TCP stack, have fun in a world where every app has its own QUIC stack.
User-space apps have a lot more avenues for timely updates than middleboxes or kernel-space implementations do though, and developers have lots of experience with it. If middleboxes actually received timely updates and bugfixes, there would be no ossification in the first place, and a lot of other experiments would have panned out much better, much sooner than QUIC has (e.g. TCP Fast Open might not have been DOA.)
There's also a lot of work on interop testing for QUIC implementations; I think new implementations are strongly encouraged to join the effort: https://interop.seemann.io/
I am not seeing the problem with every participant linking in their own QUIC implementations. The problem of ossification is there is way too much policy hidden on the kernel side of the sockets API, and vanishingly few applications are actually trying to make contact with Mars, which is the use-case for which those policies are tuned.
There are a billion timers inside the kernel, not all of which can be changed. Some of them are #defined even.
In these days when machines are very large and always have several applications running, having an external network stack in the kernel violates the end-to-end principle. All of the policy about congestion, retrying, pacing, shaping, and flow control belong inside the application.
Can you point me to an example of a timer in the kernel that is not settable/tunable that should be? My experience in looking at such things suggests that most of the #defined bits are because RFCs define the protocol that way.
As for network stack per application: you're more than welcome to so so a myriad of ways - linux provides many different ways to pull raw IP, or raw ethernet into userspace (e.g. xdp, tun/tap devices, dpdk, and so on). It's not like you're being forced to use the kernel stack from lack of supported alternatives.
I wouldn't. I would write down the protocol in a good and extensible way, the first time. It's no good throwing something into the world with the assumption that you can fix the protocol later.
> The alternative is that either browsers will be the only users of QUIC - or that each application is required to bring its own QUIC implementation embedded into the binary.
This is already being done with event loops (libuv) and HTTP frameworks. I don't see why this would be a huge issue. It's also a boon for security and keeping software up-to-date because it's a lot easier to patch userspace apps than it is to roll out a new kernel patch across multiple kernels and force everyone to upgrade.
> because the sockets API and the system TCP stacks are too ossified to be improved
What part of the sockets API specifically do you think is ossified? Also, that doesn't seem to have kept the kernel devs from introducing new IO APIs like io_uring.
I think the point of QUIC is 'if the implementation other using is problematic, I can use my own. And no random middlebox will prevent me from doing so' instead of `everyone must bring their own QUIC implementation.`
There is a slight different here. It's the difference between 'the right to do' and 'the requirement to do'.
While at same time. You must use 'system tcp implementation' and you are not allowed to use custom one. Because even system allow it (maybe require root permission or something), the middlebox won't.
> Not having a system API is the entire point of QUIC. The only reason QUIC needs to exist is because the sockets API and the system TCP stacks are too ossified to be improved.
I don't think your take is correct.
The entire point if QUIC was that you could not change TCP without introducing breaking changes, not that there were system APIs for TCP.
Your point is also refuted by the fact that QUIC is built over UDP.
As far as I can tell there is no real impediment to provide a system API for QUIC.
It'll always be niche outside the browser until this exists.