Gemini
2.5 Flash
Upgrade
Show more
Conversation with Gemini analyze this text and implement the packets in dart: The QUIC packet headers
At this point, the QUIC session and the TLS security keys are known by the client and the server. They can start to exchange data. Before looking at how data is carried inside QUIC packets, it is interesting to explore in more details the packet headers that are placed inside each packet. QUIC uses variable length packet headers. Two different header formats exist. The first header format is the long header. This is the header used for the first packets of a QUIC connection.
Internet protocol specifications usually contain figures to represent the format of the packet headers. This graphical format is useful to get a quick grasp at a header containing fixed size fields. However, when a header contains several variable length fields, the graphical representation can become difficult to follow. The QUIC specification [25] uses the textual representation that was also used for the TLS protocol. As an example, let us consider the well-known TCP header. This header is graphically represented as shown in Fig. 22.
Fig. 22 Graphical representation of the TCP header
Listing 3 Textual representation of the TCP header
TCP Header Packet { Source Port (16), Destination Port (16), Sequence Number (32), Acknowledgment Number (32), Offset (4), Reserved (6), Urgent Flag (1), ACK Flag (1), Push Flag (1), RST Flag (1), SYN Flag (1), FIN Flag(1), Window (16), TCP Checksum (16), Urgent Pointer (16), TCP Options (..)}
The attentive reader will easily understand the correspondence between the two formats. When explaining QUIC, we use the textual representation while we stick to the graphical one for TCP.
Listing 4 shows the QUIC long header. It starts with one byte containing the header type in the most significant bit, two bits indicating the packet type and four bits that are specific to each packet packet. Then, 32 bits carry the QUIC version number. The current version of QUIC, defined in [25], corresponds to version 0x00000001. The header then contains the destination and source connection identifiers that were described previously and then a payload that is specific to each type.
Listing 4 The QUIC long header
Long Header Packet { Header Form (1) = 1, /* high order bit of the first byte / Fixed Bit (1) = 1, / second order bit of the first byte / Long Packet Type (2), / third and fourth high order bits of the first byte / Type-Specific Bits (4), / low order four bits of the first byte / Version (32), / 32 bits version number / Destination Connection ID Length (8), / 8 bits / Destination Connection ID (0..160), / variable number from 0 up to 160 bits / Source Connection ID Length (8), Source Connection ID (0..160), Type-Specific Payload (..), / variable length */}
Note
Encoding packet numbers
Most transport protocols use fixed fields to encode packet numbers or byte offsets. The size of this field is always a trade-off. On one hand, a small packet number field limits the per packet overhead. On the other hand, a large packet number space is required to ensure that two packets carrying different data do not use the same packet number. TCP uses a 32 bits sequence number field that indicates the position of the first byte of the payload in the bytestream. This 32 bits field became a concern as bandwidth increased to Gbps and beyond [32].
QUIC takes a different approach to sequence numbers. Each packet contains a per-packet sequence number. This number is encoded as a variable-length integer (varint). Such a varint has a length encoded in the two most significant bits of the first byte. If these bits are set to 00, then the varint is encoded in one byte and can contain values between 0
and 26−1
. If the two most significant bits are set to 01, the varint can encode values between 0
and 214−1
within two bytes. When the two high order bits are set to 11 the varint can encode values between 0
and 262−1
within four bytes.
There are other important differences between QUIC and other transport protocols when considering packet numbers. First, a QUIC sender must never reuse the same packet number for two different packets sent over a QUIC connection. If data needs to be retransmitted, it will be resent as a frame inside a new packet. Furthermore, since the largest possible packet number is 262−1
, a QUIC sender must close the corresponding connection once it has sent a QUIC packet carrying this packet number. This puts a restriction on the duration of QUIC connections. They cannot last forever in contrast to TCP connections such as those used to support BGP sessions between routers. An application that uses QUIC must be ready to restart a connection from time to time.
This long header is used for the Initial, Handhsake and Retry packets. Some of these packet types add new flags in the first byte and additional information after the connection identifiers. Listing 5 shows the long header of the Initial packet. It contains two bits in the first byte that indicate the length of the packet number field. The packet specific part contains an option token, a length field, a packet number and a payload. The token length, length and packet number are encoded using variable length integers.
Listing 5 The QUIC long header of the Initial packet
Initial Packet { Header Form (1) = 1, /* High order bit first byte / Fixed Bit (1) = 1, Long Packet Type (2) = 0, Reserved Bits (2), Packet Number Length (2), / Low order 2 bits of first byte */ Version (32), Destination Connection ID Length (8), Destination Connection ID (0..160), Source Connection ID Length (8), Source Connection ID (0..160), Token Length (i), Token (..), Length (i), Packet Number (8..32), Packet Payload (8..),}
The QUIC short header contains fewer fields. The most significant bit of the first byte is set to 1 to indicate that the packet carries a short header. The other flags will be discussed later. The two least significant bits of this byte contain the length of the packet number field. It is interesting to note that in contrast with the long header, there is no explicit indication of the length of the destination connection identifier. This connection identifier has been assigned by the host that receives this packet and it already knows the length of the connection identifiers that it uses.
Listing 6 The QUIC short header used by 1-RTT packets
1-RTT Packet { Header Form (1) = 0, /* High order bit of first byte / Fixed Bit (1) = 1, Spin Bit (1), Reserved Bits (2), Key Phase (1), Packet Number Length (2), / Low order bits of first byte */ Destination Connection ID (0..160), Packet Number (8..32), Packet Payload (8..),}
The short header format, depicted in Listing 6, is used by all QUIC packets once the session keys have been derived. This usually happens after one round-trip-time. These packets are called the 1-RTT packets in the QUIC specification [25]. This notation is used to emphasize the fact that QUIC also supports 0-RTT packets, i.e. packets that carry data and can be exchanged in parallel with the TLS handshake.
The provided text details the structure of QUIC (Quick UDP Internet Connections) packet headers, contrasting them with the more traditional TCP header. It describes two main QUIC header formats: the Long Header and the Short Header, and also explains the variable-length encoding of certain fields like connection IDs and packet numbers.
Here's an analysis of the text and how to implement the packet structures in Dart:
Analysis of QUIC Packet Headers Key Concepts Variable Length Headers: Unlike TCP's fixed-size header, QUIC headers use variable lengths, which can make graphical representation difficult. The specification uses a textual format to describe them. Long Header: Used for initial packets in a QUIC connection (e.g., Initial, Handshake, and Retry packets). Identified by the Header Form bit (most significant bit of the first byte) being 1. Includes a Version number, Destination Connection ID, and Source Connection ID. Specific packet types (like Initial packets) add further fields and flags. Short Header: Used for data exchange once session keys are established, typically after one Round-Trip Time (1-RTT packets). Identified by the Header Form bit being 0. Does not explicitly indicate the length of the destination connection ID, as the receiver already knows it. Includes a Destination Connection ID and Packet Number. Variable-Length Integers (Varints): QUIC uses a specialized encoding for fields like packet numbers, token length, and general lengths. This allows for efficient use of space: 00 prefix: 1-byte varint (values up to 2 6 −1) 01 prefix: 2-byte varint (values up to 2 14 −1) 10 prefix: 4-byte varint (values up to 2 30 −1) 11 prefix: 8-byte varint (values up to 2 62 −1) Packet Number Handling: QUIC packet numbers are never reused for different packets on the same connection. Retransmitted data is sent in new packets. Connections must close once the largest possible packet number (2 62 −1) is sent. Dart Implementation of QUIC Packet Headers To implement these in Dart, we'll create classes representing each header type. Since many fields are variable length and involve bit manipulation, we'll need helper functions for reading and writing these fields from a Uint8List (byte array).
We'll focus on the structure and placeholder methods for reading/writing, as a full implementation of varints and bit-level parsing would be extensive.
- QuicPacketHeader (Abstract Base Class) This abstract class will define common properties for all QUIC packet headers.
Dart
import 'dart:typed_data';
abstract class QuicPacketHeader { // Common properties (e.g., from the first byte) int headerForm; // 1 for Long Header, 0 for Short Header int fixedBit; // Should always be 1 // ... other common bits
QuicPacketHeader({required this.headerForm, required this.fixedBit});
// Method to parse the header from a byte stream factory QuicPacketHeader.parse(Uint8List data) { // Determine header type based on the first bit final firstByte = data[0]; final headerForm = (firstByte >> 7) & 0x01;
if (headerForm == 1) {
return QuicLongHeader.parse(data);
} else {
return QuicShortHeader.parse(data);
}
}
// Method to serialize the header to a byte stream Uint8List toBytes(); } 2. QuicLongHeader Dart
import 'dart:typed_data'; import 'package:collection/collection.dart'; // For deep equality checks if needed
class QuicLongHeader extends QuicPacketHeader { int longPacketType; int typeSpecificBits; int version; int destConnectionIdLength; Uint8List? destConnectionId; int sourceConnectionIdLength; Uint8List? sourceConnectionId; Uint8List? typeSpecificPayload; // This would be further parsed based on packet type
QuicLongHeader({ required int headerForm, required int fixedBit, required this.longPacketType, required this.typeSpecificBits, required this.version, required this.destConnectionIdLength, this.destConnectionId, required this.sourceConnectionIdLength, this.sourceConnectionId, this.typeSpecificPayload, }) : super(headerForm: headerForm, fixedBit: fixedBit);
factory QuicLongHeader.parse(Uint8List data) {
// This is a simplified parsing example.
// Real-world parsing would involve a ByteDataReader to handle variable lengths.
int offset = 0;
final firstByte = data[offset++];
final headerForm = (firstByte >> 7) & 0x01;
final fixedBit = (firstByte >> 6) & 0x01;
final longPacketType = (firstByte >> 4) & 0x03;
final typeSpecificBits = firstByte & 0x0F;
// Read 32-bit version (example)
final version = ByteData.view(data.buffer).getUint32(offset);
offset += 4;
final destConnectionIdLength = data[offset++];
Uint8List? destConnectionId;
if (destConnectionIdLength > 0) {
destConnectionId = data.sublist(offset, offset + destConnectionIdLength);
offset += destConnectionIdLength;
}
final sourceConnectionIdLength = data[offset++];
Uint8List? sourceConnectionId;
if (sourceConnectionIdLength > 0) {
sourceConnectionId = data.sublist(offset, offset + sourceConnectionIdLength);
offset += sourceConnectionIdLength;
}
// The rest is typeSpecificPayload, which would be parsed by specific packet types
final typeSpecificPayload = data.sublist(offset);
return QuicLongHeader(
headerForm: headerForm,
fixedBit: fixedBit,
longPacketType: longPacketType,
typeSpecificBits: typeSpecificBits,
version: version,
destConnectionIdLength: destConnectionIdLength,
destConnectionId: destConnectionId,
sourceConnectionIdLength: sourceConnectionIdLength,
sourceConnectionId: sourceConnectionId,
typeSpecificPayload: typeSpecificPayload,
);
}
@override Uint8List toBytes() { // Implement serialization logic here. // This would involve creating a ByteDataWriter and writing fields. // Example: final builder = BytesBuilder(); int firstByte = (headerForm << 7) | (fixedBit << 6) | (longPacketType << 4) | typeSpecificBits; builder.addByte(firstByte); builder.add(Uint8List(4)..buffer.asByteData().setUint32(0, version)); // Add version builder.addByte(destConnectionIdLength); if (destConnectionId != null) { builder.add(destConnectionId!); } builder.addByte(sourceConnectionIdLength); if (sourceConnectionId != null) { builder.add(sourceConnectionId!); } if (typeSpecificPayload != null) { builder.add(typeSpecificPayload!); } return builder.toBytes(); }
@override
String toString() {
return 'QuicLongHeader(headerForm: $headerForm, fixedBit: $fixedBit, longPacketType: $longPacketType, typeSpecificBits:
Dart
import 'dart:typed_data';
class QuicInitialPacketHeader extends QuicLongHeader { int reservedBits; int packetNumberLength; int tokenLength; // Variable-length integer Uint8List? token; int length; // Variable-length integer int packetNumber; // Variable-length integer Uint8List packetPayload;
QuicInitialPacketHeader({ required int headerForm, required int fixedBit, required int longPacketType, required this.reservedBits, required this.packetNumberLength, required int version, required int destConnectionIdLength, Uint8List? destConnectionId, required int sourceConnectionIdLength, Uint8List? sourceConnectionId, required this.tokenLength, this.token, required this.length, required this.packetNumber, required this.packetPayload, }) : super( headerForm: headerForm, fixedBit: fixedBit, longPacketType: longPacketType, // For Initial Packet, Type-Specific Bits are 2 Reserved + 2 Packet Number Length typeSpecificBits: (reservedBits << 2) | packetNumberLength, version: version, destConnectionIdLength: destConnectionIdLength, destConnectionId: destConnectionId, sourceConnectionIdLength: sourceConnectionIdLength, sourceConnectionId: sourceConnectionId, // In an Initial packet, the rest is the Type-Specific Payload // which includes Token Length, Token, Length, Packet Number, and Packet Payload // This would be assembled in toBytes() or parsed in factory. typeSpecificPayload: null, // Set to null here as it's broken down );
factory QuicInitialPacketHeader.parse(Uint8List data) {
// More complex parsing involving reading varints.
// This is a conceptual example. A full implementation would need
// a ByteDataReader that understands QUIC varints.
int offset = 0;
final firstByte = data[offset++];
final headerForm = (firstByte >> 7) & 0x01;
final fixedBit = (firstByte >> 6) & 0x01;
final longPacketType = (firstByte >> 4) & 0x03; // Should be 0 for Initial
final reservedBits = (firstByte >> 2) & 0x03;
final packetNumberLength = firstByte & 0x03; // Low order 2 bits
final version = ByteData.view(data.buffer).getUint32(offset);
offset += 4;
final destConnectionIdLength = data[offset++];
Uint8List? destConnectionId;
if (destConnectionIdLength > 0) {
destConnectionId = data.sublist(offset, offset + destConnectionIdLength);
offset += destConnectionIdLength;
}
final sourceConnectionIdLength = data[offset++];
Uint8List? sourceConnectionId;
if (sourceConnectionIdLength > 0) {
sourceConnectionId = data.sublist(offset, offset + sourceConnectionIdLength);
offset += sourceConnectionIdLength;
}
// Here, we'd use a function to read the varint for Token Length
// For simplicity, let's assume it's a fixed size for this example.
// In reality, you'd call a `readVarInt` helper.
final tokenLength = data[offset++]; // Placeholder for varint read
Uint8List? token;
if (tokenLength > 0) {
token = data.sublist(offset, offset + tokenLength);
offset += tokenLength;
}
// Read Length (varint)
final length = ByteData.view(data.buffer).getUint32(offset); // Placeholder for varint read
offset += 4; // Assuming 4 bytes for example
// Read Packet Number (varint, length indicated by packetNumberLength)
int packetNumber;
switch (packetNumberLength) {
case 0: // 1-byte
packetNumber = data[offset];
offset += 1;
break;
case 1: // 2-byte
packetNumber = ByteData.view(data.buffer).getUint16(offset);
offset += 2;
break;
case 2: // 4-byte
packetNumber = ByteData.view(data.buffer).getUint32(offset);
offset += 4;
break;
case 3: // 8-byte
// For simplicity, handling 8-byte (62-bit) varint as Dart's int supports it.
// Needs proper 62-bit handling for QUIC spec.
packetNumber = ByteData.view(data.buffer).getUint64(offset);
offset += 8;
break;
default:
throw Exception('Invalid packet number length');
}
final packetPayload = data.sublist(offset);
return QuicInitialPacketHeader(
headerForm: headerForm,
fixedBit: fixedBit,
longPacketType: longPacketType,
reservedBits: reservedBits,
packetNumberLength: packetNumberLength,
version: version,
destConnectionIdLength: destConnectionIdLength,
destConnectionId: destConnectionId,
sourceConnectionIdLength: sourceConnectionIdLength,
sourceConnectionId: sourceConnectionId,
tokenLength: tokenLength,
token: token,
length: length,
packetNumber: packetNumber,
packetPayload: packetPayload,
);
}
@override Uint8List toBytes() { final builder = BytesBuilder(); // Reconstruct the first byte with correct bits int firstByte = (headerForm << 7) | (fixedBit << 6) | (longPacketType << 4) | (reservedBits << 2) | packetNumberLength; builder.addByte(firstByte); builder.add(Uint8List(4)..buffer.asByteData().setUint32(0, version)); // Add version builder.addByte(destConnectionIdLength); if (destConnectionId != null) { builder.add(destConnectionId!); } builder.addByte(sourceConnectionIdLength); if (sourceConnectionId != null) { builder.add(sourceConnectionId!); }
// Add Token Length (varint)
builder.addByte(tokenLength); // Placeholder for varint write
if (token != null) {
builder.add(token!);
}
// Add Length (varint)
builder.add(Uint8List(4)..buffer.asByteData().setUint32(0, length)); // Placeholder for varint write
// Add Packet Number (based on packetNumberLength)
switch (packetNumberLength) {
case 0:
builder.addByte(packetNumber);
break;
case 1:
builder.add(Uint8List(2)..buffer.asByteData().setUint16(0, packetNumber));
break;
case 2:
builder.add(Uint8List(4)..buffer.asByteData().setUint32(0, packetNumber));
break;
case 3:
builder.add(Uint8List(8)..buffer.asByteData().setUint64(0, packetNumber));
break;
}
builder.add(packetPayload);
return builder.toBytes();
}
@override
String toString() {
return 'QuicInitialPacketHeader(headerForm: $headerForm, fixedBit: $fixedBit, longPacketType: $longPacketType, reservedBits: $reservedBits, packetNumberLength:
import 'dart:typed_data';
class QuicShortHeader extends QuicPacketHeader { int spinBit; int reservedBits; int keyPhase; int packetNumberLength; Uint8List? destConnectionId; // Length not explicitly in header, known by receiver int packetNumber; // Variable-length integer Uint8List packetPayload;
QuicShortHeader({ required int headerForm, required int fixedBit, required this.spinBit, required this.reservedBits, required this.keyPhase, required this.packetNumberLength, this.destConnectionId, required this.packetNumber, required this.packetPayload, }) : super(headerForm: headerForm, fixedBit: fixedBit);
factory QuicShortHeader.parse(Uint8List data, {int? destConnectionIdLength}) {
// Short header parsing assumes the receiver knows the destination connection ID length.
// This destConnectionIdLength would typically be pre-negotiated or known from connection state.
int offset = 0;
final firstByte = data[offset++];
final headerForm = (firstByte >> 7) & 0x01; // Should be 0 for Short Header
final fixedBit = (firstByte >> 6) & 0x01;
final spinBit = (firstByte >> 5) & 0x01;
final reservedBits = (firstByte >> 3) & 0x03;
final keyPhase = (firstByte >> 2) & 0x01;
final packetNumberLength = firstByte & 0x03; // Low order 2 bits
Uint8List? destConnectionId;
if (destConnectionIdLength != null && destConnectionIdLength > 0) {
destConnectionId = data.sublist(offset, offset + destConnectionIdLength);
offset += destConnectionIdLength;
} else {
// In a real scenario, if destConnectionIdLength is null or 0,
// it means the connection ID is omitted or its length is 0.
// For short headers, the length is implicit for the receiver.
}
int packetNumber;
switch (packetNumberLength) {
case 0: // 1-byte
packetNumber = data[offset];
offset += 1;
break;
case 1: // 2-byte
packetNumber = ByteData.view(data.buffer).getUint16(offset);
offset += 2;
break;
case 2: // 4-byte
packetNumber = ByteData.view(data.buffer).getUint32(offset);
offset += 4;
break;
case 3: // 8-byte
// For simplicity, handling 8-byte (62-bit) varint as Dart's int supports it.
// Needs proper 62-bit handling for QUIC spec.
packetNumber = ByteData.view(data.buffer).getUint64(offset);
offset += 8;
break;
default:
throw Exception('Invalid packet number length');
}
final packetPayload = data.sublist(offset);
return QuicShortHeader(
headerForm: headerForm,
fixedBit: fixedBit,
spinBit: spinBit,
reservedBits: reservedBits,
keyPhase: keyPhase,
packetNumberLength: packetNumberLength,
destConnectionId: destConnectionId,
packetNumber: packetNumber,
packetPayload: packetPayload,
);
}
@override Uint8List toBytes() { final builder = BytesBuilder(); // Reconstruct the first byte with correct bits int firstByte = (headerForm << 7) | (fixedBit << 6) | (spinBit << 5) | (reservedBits << 3) | (keyPhase << 2) | packetNumberLength; builder.addByte(firstByte); if (destConnectionId != null) { builder.add(destConnectionId!); }
// Add Packet Number (based on packetNumberLength)
switch (packetNumberLength) {
case 0:
builder.addByte(packetNumber);
break;
case 1:
builder.add(Uint8List(2)..buffer.asByteData().setUint16(0, packetNumber));
break;
case 2:
builder.add(Uint8List(4)..buffer.asByteData().setUint32(0, packetNumber));
break;
case 3:
builder.add(Uint8List(8)..buffer.asByteData().setUint64(0, packetNumber));
break;
}
builder.add(packetPayload);
return builder.toBytes();
}
@override
String toString() {
return 'QuicShortHeader(headerForm: $headerForm, fixedBit: $fixedBit, spinBit: $spinBit, reservedBits: $reservedBits, keyPhase: $keyPhase, packetNumberLength:
// Example of a conceptual readVarInt function // This would be much more complex to handle all cases correctly. int readVarInt(Uint8List data, int offset) { final firstByte = data[offset]; final prefix = (firstByte >> 6) & 0x03; if (prefix == 0x00) { return firstByte & 0x3F; // 6 bits } else if (prefix == 0x01) { return ByteData.view(data.buffer).getUint16(offset) & 0x3FFF; // 14 bits } else if (prefix == 0x10) { // This prefix is 2 in binary, which is 0x2. The text states 0x11, which would be 3. // There seems to be a slight discrepancy between the text and standard varint notation. // QUIC varints are 00, 01, 10, 11 for 1, 2, 4, 8 bytes respectively. // If it's 0x10 (binary 10), then it's 4 bytes, 30 bits. return ByteData.view(data.buffer).getUint32(offset) & 0x3FFFFFFF; // 30 bits } else if (prefix == 0x11) { // Binary 11 // For 8 bytes (62 bits) return ByteData.view(data.buffer).getUint64(offset) & 0x3FFFFFFFFFFFFFFF; // 62 bits } throw Exception('Invalid varint prefix'); }
// Similar writeVarInt function would be needed.
Byte Buffer Management: For efficient parsing and serialization, you'd typically use a custom ByteDataReader and ByteDataWriter that can read/write specific numbers of bits or variable-length fields and keep track of the current offset. The package:typed_data provides Uint8List and ByteData for byte-level operations, but you'd build higher-level abstractions on top of them.
Connection ID Length: For QuicShortHeader, the Destination Connection ID Length is not explicitly in the header. The receiver must know this length from the connection's established state. This implies that the parsing function for QuicShortHeader would need this length as an input parameter from the connection's context.
Packet Payload Parsing: The packetPayload fields in these classes are Uint8List. In a real QUIC implementation, this payload would contain QUIC frames (e.g., Stream, Ack, Crypto frames), which would require further parsing.
Error Handling: The provided parse methods have basic error handling, but a production-ready implementation would need extensive validation and error reporting according to the QUIC specification.
This Dart implementation provides a structural foundation for handling QUIC packet headers based on the provided text. The key to a complete and correct implementation lies in accurately handling the bit-level operations and variable-length integer encoding.
incorporate this note as well: The QUIC packet headers
At this point, the QUIC session and the TLS security keys are known by the client and the server. They can start to exchange data. Before looking at how data is carried inside QUIC packets, it is interesting to explore in more details the packet headers that are placed inside each packet. QUIC uses variable length packet headers. Two different header formats exist. The first header format is the long header. This is the header used for the first packets of a QUIC connection.
Internet protocol specifications usually contain figures to represent the format of the packet headers. This graphical format is useful to get a quick grasp at a header containing fixed size fields. However, when a header contains several variable length fields, the graphical representation can become difficult to follow. The QUIC specification [25] uses the textual representation that was also used for the TLS protocol. As an example, let us consider the well-known TCP header. This header is graphically represented as shown in Fig. 22.
Fig. 22 Graphical representation of the TCP header
Listing 3 Textual representation of the TCP header
TCP Header Packet { Source Port (16), Destination Port (16), Sequence Number (32), Acknowledgment Number (32), Offset (4), Reserved (6), Urgent Flag (1), ACK Flag (1), Push Flag (1), RST Flag (1), SYN Flag (1), FIN Flag(1), Window (16), TCP Checksum (16), Urgent Pointer (16), TCP Options (..)}
The attentive reader will easily understand the correspondence between the two formats. When explaining QUIC, we use the textual representation while we stick to the graphical one for TCP.
Listing 4 shows the QUIC long header. It starts with one byte containing the header type in the most significant bit, two bits indicating the packet type and four bits that are specific to each packet packet. Then, 32 bits carry the QUIC version number. The current version of QUIC, defined in [25], corresponds to version 0x00000001. The header then contains the destination and source connection identifiers that were described previously and then a payload that is specific to each type.
Listing 4 The QUIC long header
Long Header Packet { Header Form (1) = 1, /* high order bit of the first byte / Fixed Bit (1) = 1, / second order bit of the first byte / Long Packet Type (2), / third and fourth high order bits of the first byte / Type-Specific Bits (4), / low order four bits of the first byte / Version (32), / 32 bits version number / Destination Connection ID Length (8), / 8 bits / Destination Connection ID (0..160), / variable number from 0 up to 160 bits / Source Connection ID Length (8), Source Connection ID (0..160), Type-Specific Payload (..), / variable length */}
Note
Encoding packet numbers
Most transport protocols use fixed fields to encode packet numbers or byte offsets. The size of this field is always a trade-off. On one hand, a small packet number field limits the per packet overhead. On the other hand, a large packet number space is required to ensure that two packets carrying different data do not use the same packet number. TCP uses a 32 bits sequence number field that indicates the position of the first byte of the payload in the bytestream. This 32 bits field became a concern as bandwidth increased to Gbps and beyond [32].
QUIC takes a different approach to sequence numbers. Each packet contains a per-packet sequence number. This number is encoded as a variable-length integer (varint). Such a varint has a length encoded in the two most significant bits of the first byte. If these bits are set to 00, then the varint is encoded in one byte and can contain values between 0
and 26−1
. If the two most significant bits are set to 01, the varint can encode values between 0
and 214−1
within two bytes. When the two high order bits are set to 11 the varint can encode values between 0
and 262−1
within four bytes.
There are other important differences between QUIC and other transport protocols when considering packet numbers. First, a QUIC sender must never reuse the same packet number for two different packets sent over a QUIC connection. If data needs to be retransmitted, it will be resent as a frame inside a new packet. Furthermore, since the largest possible packet number is 262−1
, a QUIC sender must close the corresponding connection once it has sent a QUIC packet carrying this packet number. This puts a restriction on the duration of QUIC connections. They cannot last forever in contrast to TCP connections such as those used to support BGP sessions between routers. An application that uses QUIC must be ready to restart a connection from time to time.
This long header is used for the Initial, Handhsake and Retry packets. Some of these packet types add new flags in the first byte and additional information after the connection identifiers. Listing 5 shows the long header of the Initial packet. It contains two bits in the first byte that indicate the length of the packet number field. The packet specific part contains an option token, a length field, a packet number and a payload. The token length, length and packet number are encoded using variable length integers.
Listing 5 The QUIC long header of the Initial packet
Initial Packet { Header Form (1) = 1, /* High order bit first byte / Fixed Bit (1) = 1, Long Packet Type (2) = 0, Reserved Bits (2), Packet Number Length (2), / Low order 2 bits of first byte */ Version (32), Destination Connection ID Length (8), Destination Connection ID (0..160), Source Connection ID Length (8), Source Connection ID (0..160), Token Length (i), Token (..), Length (i), Packet Number (8..32), Packet Payload (8..),}
The QUIC short header contains fewer fields. The most significant bit of the first byte is set to 1 to indicate that the packet carries a short header. The other flags will be discussed later. The two least significant bits of this byte contain the length of the packet number field. It is interesting to note that in contrast with the long header, there is no explicit indication of the length of the destination connection identifier. This connection identifier has been assigned by the host that receives this packet and it already knows the length of the connection identifiers that it uses.
Listing 6 The QUIC short header used by 1-RTT packets
1-RTT Packet { Header Form (1) = 0, /* High order bit of first byte / Fixed Bit (1) = 1, Spin Bit (1), Reserved Bits (2), Key Phase (1), Packet Number Length (2), / Low order bits of first byte */ Destination Connection ID (0..160), Packet Number (8..32), Packet Payload (8..),}
The short header format, depicted in Listing 6, is used by all QUIC packets once the session keys have been derived. This usually happens after one round-trip-time. These packets are called the 1-RTT packets in the QUIC specification [25]. This notation is used to emphasize the fact that QUIC also supports 0-RTT packets, i.e. packets that carry data and can be exchanged in parallel with the TLS handshake.
Note
Coalescing packets
Besides the connection identifiers, another difference between the short and the long headers is the presence of the Packet Length field in the long header. This field might surprise the reader who is familiar with UDP since UDP also contains a Length field. As each QUIC packet is placed inside a UDP message, the QUIC Length field could seem redundant. This Length field was included in the QUIC long header to allow a QUIC sender to coalesce several consecutive and small packets inside a single UDP message. Some of the frames exchanged during the handshake are rather small. Sending a UDP message for each of these frames would be a waste of resources. Furthermore, a mix of Initial, Handshake and 0-RTT packets are exchanged when creating a QUIC connection. It would not be wise to require the utilization of one UDP message to send each of these packets. You might observe this optimization when analyzing packet traces collected on QUIC servers.
Let's break down the provided text and integrate the new "Coalescing packets" note into our understanding and Dart implementation plan.
Analysis of QUIC Packet Headers The provided text offers a detailed look into QUIC (Quick UDP Internet Connections) packet headers, highlighting their distinct features compared to traditional protocols like TCP.
Key Concepts from the Text Variable Length Headers: Unlike TCP's fixed-size header, QUIC employs variable-length headers. This approach makes graphical representation less intuitive, so the specification uses a textual format to describe them. Two Primary Header Formats: Long Header: Used for the initial packets of a QUIC connection, such as Initial, Handshake, and Retry packets. It's identifiable by the Header Form bit (the most significant bit of the first byte) being 1. This header includes a Version number, Destination Connection ID, and Source Connection ID. Specific packet types, like Initial packets, extend this base with additional fields and flags. Short Header: Utilized for data exchange once session keys are established, typically after one Round-Trip Time (1-RTT packets). It's identified by the Header Form bit being 0. A notable difference from the Long Header is the absence of an explicit destination connection ID length, as the receiver already knows this from the connection's context. Variable-Length Integers (Varints): QUIC uses a specialized, compact encoding for fields like packet numbers, token lengths, and general lengths. This encoding optimizes space: 00 prefix: 1-byte varint (values up to 2 6 −1) 01 prefix: 2-byte varint (values up to 2 14 −1) 10 prefix: 4-byte varint (values up to 2 30 −1) 11 prefix: 8-byte varint (values up to 2 62 −1) Packet Number Handling: A QUIC sender must never reuse the same packet number for different packets on the same connection. Retransmitted data is sent as new frames within new packets. QUIC connections have a finite lifespan; a sender must close the connection once the largest possible packet number (2 62 −1) has been sent. This contrasts with long-lived TCP connections. Coalescing Packets (New Note): The Length field in the Long Header is crucial for coalescing multiple small QUIC packets into a single UDP message. This optimization is particularly useful during the connection handshake (which involves Initial, Handshake, and 0-RTT packets) where individual frames might be small. By bundling them into one UDP datagram, QUIC reduces overhead and improves efficiency, especially in scenarios with many small data exchanges. This is a key distinction from UDP, where a length field might seem redundant. Dart Implementation of QUIC Packet Headers To implement these QUIC packet structures in Dart, we'll define classes for each header type and include placeholder methods for parsing and serialization. Handling variable-length fields and bit-level operations will require helper functions.
- QuicPacketHeader (Abstract Base Class) This abstract class provides a common interface for all QUIC packet headers.
Dart
import 'dart:typed_data';
abstract class QuicPacketHeader { // Common properties derived from the first byte final int headerForm; // 1 for Long Header, 0 for Short Header final int fixedBit; // Should always be 1 according to the spec
QuicPacketHeader({required this.headerForm, required this.fixedBit});
/// Parses a raw byte stream to determine and create the correct QUIC packet header. factory QuicPacketHeader.parse(Uint8List data, {int? shortHeaderDestConnectionIdLength}) { if (data.isEmpty) { throw ArgumentError('Packet data cannot be empty.'); } final firstByte = data[0]; final headerForm = (firstByte >> 7) & 0x01;
if (headerForm == 1) {
return QuicLongHeader.parse(data);
} else {
// For Short Headers, the destination connection ID length must be known
// from the connection's established state.
if (shortHeaderDestConnectionIdLength == null) {
throw ArgumentError('Destination Connection ID Length must be provided for Short Headers.');
}
return QuicShortHeader.parse(data, destConnectionIdLength: shortHeaderDestConnectionIdLength);
}
}
/// Serializes the header into a byte array. Uint8List toBytes(); } 2. QuicLongHeader This class represents the structure of a QUIC Long Header.
Dart
import 'dart:typed_data';
class QuicLongHeader extends QuicPacketHeader { final int longPacketType; final int typeSpecificBits; final int version; final int destConnectionIdLength; final Uint8List? destConnectionId; final int sourceConnectionIdLength; final Uint8List? sourceConnectionId; // Note: The text also mentions a 'Length (i)' field in the Initial packet's long header. // This is a crucial field for packet coalescing, but it's part of the Type-Specific Payload // for the Initial Packet type, so we'll handle it there. The general Long Header // itself doesn't explicitly list 'Length' as a top-level field outside of its // specific packet types.
QuicLongHeader({ required int headerForm, required int fixedBit, required this.longPacketType, required this.typeSpecificBits, required this.version, required this.destConnectionIdLength, this.destConnectionId, required this.sourceConnectionIdLength, this.sourceConnectionId, }) : super(headerForm: headerForm, fixedBit: fixedBit);
/// Parses a byte array into a QuicLongHeader object. factory QuicLongHeader.parse(Uint8List data) { if (data.length < 9) { // Minimum size: 1 byte for first byte + 4 for version + 2 for conn ID lengths + 2 for 0-length CIDs throw FormatException('Insufficient data for a complete Long Header.'); }
int offset = 0;
final firstByte = data[offset++];
final headerForm = (firstByte >> 7) & 0x01;
final fixedBit = (firstByte >> 6) & 0x01;
final longPacketType = (firstByte >> 4) & 0x03;
final typeSpecificBits = firstByte & 0x0F; // For Initial packet, this would be Reserved + Packet Number Length
// Read 32-bit version
final version = data.buffer.asByteData().getUint32(offset);
offset += 4;
final destConnectionIdLength = data[offset++];
Uint8List? destConnectionId;
if (destConnectionIdLength > 0) {
if (offset + destConnectionIdLength > data.length) {
throw FormatException('Malformed Long Header: Destination Connection ID length extends beyond data.');
}
destConnectionId = data.sublist(offset, offset + destConnectionIdLength);
offset += destConnectionIdLength;
}
final sourceConnectionIdLength = data[offset++];
Uint8List? sourceConnectionId;
if (sourceConnectionIdLength > 0) {
if (offset + sourceConnectionIdLength > data.length) {
throw FormatException('Malformed Long Header: Source Connection ID length extends beyond data.');
}
sourceConnectionId = data.sublist(offset, offset + sourceConnectionIdLength);
offset += sourceConnectionIdLength;
}
// The remainder of the data is the Type-Specific Payload.
// For a generic Long Header, we don't parse it here, but it would be passed
// to a more specific packet type parser (e.g., InitialPacketHeader).
// The text implies 'Type-Specific Payload' is the part after connection IDs.
return QuicLongHeader(
headerForm: headerForm,
fixedBit: fixedBit,
longPacketType: longPacketType,
typeSpecificBits: typeSpecificBits,
version: version,
destConnectionIdLength: destConnectionIdLength,
destConnectionId: destConnectionId,
sourceConnectionIdLength: sourceConnectionIdLength,
sourceConnectionId: sourceConnectionId,
);
}
@override Uint8List toBytes() { final builder = BytesBuilder(); int firstByte = (headerForm << 7) | (fixedBit << 6) | (longPacketType << 4) | typeSpecificBits; builder.addByte(firstByte); builder.add(Uint8List(4)..buffer.asByteData().setUint32(0, version)); builder.addByte(destConnectionIdLength); if (destConnectionId != null) { builder.add(destConnectionId!); } builder.addByte(sourceConnectionIdLength); if (sourceConnectionId != null) { builder.add(sourceConnectionId!); } return builder.toBytes(); }
@override
String toString() {
return 'QuicLongHeader(headerForm: $headerForm, fixedBit: $fixedBit, longPacketType: $longPacketType, typeSpecificBits:
Dart
import 'dart:typed_data';
// Helper for reading/writing QUIC variable-length integers (varints) // This is a simplified implementation for illustration. // A robust version would handle all 4 varint lengths and potential errors. class VarInt { static int read(Uint8List data, int offset) { if (offset >= data.length) { throw FormatException('Attempted to read varint beyond data bounds.'); } final firstByte = data[offset]; final prefix = (firstByte >> 6) & 0x03; int value; int bytesRead;
if (prefix == 0x00) { // 1-byte varint
value = firstByte & 0x3F;
bytesRead = 1;
} else if (prefix == 0x01) { // 2-byte varint
if (offset + 1 >= data.length) throw FormatException('Incomplete 2-byte varint.');
value = data.buffer.asByteData().getUint16(offset) & 0x3FFF;
bytesRead = 2;
} else if (prefix == 0x02) { // 4-byte varint
if (offset + 3 >= data.length) throw FormatException('Incomplete 4-byte varint.');
value = data.buffer.asByteData().getUint32(offset) & 0x3FFFFFFF;
bytesRead = 4;
} else if (prefix == 0x03) { // 8-byte varint
if (offset + 7 >= data.length) throw FormatException('Incomplete 8-byte varint.');
// Dart's int can handle up to 63 bits, so Uint64 works for 62-bit varint
value = data.buffer.asByteData().getUint64(offset) & 0x3FFFFFFFFFFFFFFF;
bytesRead = 8;
} else {
throw FormatException('Invalid varint prefix: $prefix');
}
return value;
}
static Uint8List write(int value) { final builder = BytesBuilder(); if (value < (1 << 6)) { // 1-byte builder.addByte(value & 0x3F); } else if (value < (1 << 14)) { // 2-byte builder.add(Uint8List(2)..buffer.asByteData().setUint16(0, value | 0x4000)); } else if (value < (1 << 30)) { // 4-byte builder.add(Uint8List(4)..buffer.asByteData().setUint32(0, value | 0x80000000)); } else if (value < (1 << 62)) { // 8-byte builder.add(Uint8List(8)..buffer.asByteData().setUint64(0, value | 0xC000000000000000)); } else { throw ArgumentError('Value $value is too large for a QUIC varint (max 2^62 - 1).'); } return builder.toBytes(); }
// Determine the number of bytes a varint will occupy when written. static int getLength(int value) { if (value < (1 << 6)) return 1; if (value < (1 << 14)) return 2; if (value < (1 << 30)) return 4; if (value < (1 << 62)) return 8; throw ArgumentError('Value $value is too large for a QUIC varint (max 2^62 - 1).'); } }
class QuicInitialPacketHeader extends QuicLongHeader { final int reservedBits; final int packetNumberLengthBits; // The 'Packet Number Length' in the first byte final int tokenLength; // This is a varint final Uint8List? token; final int length; // This is a varint, the total length of the packet number + payload final int packetNumber; // This is a varint, actual length determined by packetNumberLengthBits final Uint8List packetPayload;
QuicInitialPacketHeader({ required int headerForm, required int fixedBit, required int longPacketType, required this.reservedBits, required this.packetNumberLengthBits, // Using 'Bits' to distinguish from byte length required int version, required int destConnectionIdLength, Uint8List? destConnectionId, required int sourceConnectionIdLength, Uint8List? sourceConnectionId, required this.tokenLength, this.token, required this.length, required this.packetNumber, required this.packetPayload, }) : super( headerForm: headerForm, fixedBit: fixedBit, longPacketType: longPacketType, // For Initial Packet, Type-Specific Bits are 2 Reserved + 2 Packet Number Length typeSpecificBits: (reservedBits << 2) | packetNumberLengthBits, version: version, destConnectionIdLength: destConnectionIdLength, destConnectionId: destConnectionId, sourceConnectionIdLength: sourceConnectionIdLength, sourceConnectionId: sourceConnectionId, );
factory QuicInitialPacketHeader.parse(Uint8List data) { int offset = 0; final firstByte = data[offset++];
final headerForm = (firstByte >> 7) & 0x01;
final fixedBit = (firstByte >> 6) & 0x01;
final longPacketType = (firstByte >> 4) & 0x03; // Should be 0 for Initial Packet Type
final reservedBits = (firstByte >> 2) & 0x03;
final packetNumberLengthBits = firstByte & 0x03; // Low order 2 bits, indicates byte length
final version = data.buffer.asByteData().getUint32(offset);
offset += 4;
final destConnectionIdLength = data[offset++];
Uint8List? destConnectionId;
if (destConnectionIdLength > 0) {
destConnectionId = data.sublist(offset, offset + destConnectionIdLength);
offset += destConnectionIdLength;
}
final sourceConnectionIdLength = data[offset++];
Uint8List? sourceConnectionId;
if (sourceConnectionIdLength > 0) {
sourceConnectionId = data.sublist(offset, offset + sourceConnectionIdLength);
offset += sourceConnectionIdLength;
}
// Parse Token Length (varint)
final tokenLength = VarInt.read(data, offset);
offset += VarInt.getLength(tokenLength); // Advance by actual varint length
Uint8List? token;
if (tokenLength > 0) {
token = data.sublist(offset, offset + tokenLength);
offset += tokenLength;
}
// Parse Length (varint) - this is the total length of the Packet Number and Packet Payload
final length = VarInt.read(data, offset);
offset += VarInt.getLength(length);
// Parse Packet Number (length determined by packetNumberLengthBits)
final packetNumberByteLength = 1 << packetNumberLengthBits; // 0->1, 1->2, 2->4, 3->8 bytes
int packetNumber;
switch (packetNumberByteLength) {
case 1:
packetNumber = data[offset];
break;
case 2:
packetNumber = data.buffer.asByteData().getUint16(offset);
break;
case 4:
packetNumber = data.buffer.asByteData().getUint32(offset);
break;
case 8:
packetNumber = data.buffer.asByteData().getUint64(offset);
break;
default:
throw FormatException('Invalid packet number byte length derived from bits: $packetNumberByteLength');
}
offset += packetNumberByteLength;
final packetPayload = data.sublist(offset);
return QuicInitialPacketHeader(
headerForm: headerForm,
fixedBit: fixedBit,
longPacketType: longPacketType,
reservedBits: reservedBits,
packetNumberLengthBits: packetNumberLengthBits,
version: version,
destConnectionIdLength: destConnectionIdLength,
destConnectionId: destConnectionId,
sourceConnectionIdLength: sourceConnectionIdLength,
sourceConnectionId: sourceConnectionId,
tokenLength: tokenLength,
token: token,
length: length,
packetNumber: packetNumber,
packetPayload: packetPayload,
);
}
@override Uint8List toBytes() { final builder = BytesBuilder(); int firstByte = (headerForm << 7) | (fixedBit << 6) | (longPacketType << 4) | (reservedBits << 2) | packetNumberLengthBits; builder.addByte(firstByte); builder.add(Uint8List(4)..buffer.asByteData().setUint32(0, version)); builder.addByte(destConnectionIdLength); if (destConnectionId != null) { builder.add(destConnectionId!); } builder.addByte(sourceConnectionIdLength); if (sourceConnectionId != null) { builder.add(sourceConnectionId!); }
// Add Token Length (varint) and Token
builder.add(VarInt.write(tokenLength));
if (token != null) {
builder.add(token!);
}
// Add Length (varint)
builder.add(VarInt.write(length));
// Add Packet Number (based on packetNumberLengthBits)
final packetNumberByteLength = 1 << packetNumberLengthBits;
switch (packetNumberByteLength) {
case 1:
builder.addByte(packetNumber);
break;
case 2:
builder.add(Uint8List(2)..buffer.asByteData().setUint16(0, packetNumber));
break;
case 4:
builder.add(Uint8List(4)..buffer.asByteData().setUint32(0, packetNumber));
break;
case 8:
builder.add(Uint8List(8)..buffer.asByteData().setUint64(0, packetNumber));
break;
}
builder.add(packetPayload);
return builder.toBytes();
}
@override
String toString() {
return 'QuicInitialPacketHeader(headerForm: $headerForm, fixedBit: $fixedBit, longPacketType: $longPacketType, reservedBits: $reservedBits, packetNumberLengthBits:
Dart
import 'dart:typed_data';
class QuicShortHeader extends QuicPacketHeader { final int spinBit; final int reservedBits; final int keyPhase; final int packetNumberLengthBits; // The 'Packet Number Length' in the first byte final Uint8List? destConnectionId; // Length not explicitly in header, known by receiver final int packetNumber; // Variable-length integer final Uint8List packetPayload;
QuicShortHeader({ required int headerForm, required int fixedBit, required this.spinBit, required this.reservedBits, required this.keyPhase, required this.packetNumberLengthBits, this.destConnectionId, required this.packetNumber, required this.packetPayload, }) : super(headerForm: headerForm, fixedBit: fixedBit);
/// Parses a byte array into a QuicShortHeader object.
/// Requires destConnectionIdLength to be known by the receiver.
factory QuicShortHeader.parse(Uint8List data, {required int destConnectionIdLength}) {
int offset = 0;
final firstByte = data[offset++];
final headerForm = (firstByte >> 7) & 0x01; // Should be 0 for Short Header
final fixedBit = (firstByte >> 6) & 0x01;
final spinBit = (firstByte >> 5) & 0x01;
final reservedBits = (firstByte >> 3) & 0x03;
final keyPhase = (firstByte >> 2) & 0x01;
final packetNumberLengthBits = firstByte & 0x03; // Low order 2 bits
Uint8List? destConnectionId;
if (destConnectionIdLength > 0) {
if (offset + destConnectionIdLength > data.length) {
throw FormatException('Malformed Short Header: Destination Connection ID length extends beyond data.');
}
destConnectionId = data.sublist(offset, offset + destConnectionIdLength);
offset += destConnectionIdLength;
}
// Parse Packet Number (length determined by packetNumberLengthBits)
final packetNumberByteLength = 1 << packetNumberLengthBits;
int packetNumber;
switch (packetNumberByteLength) {
case 1:
packetNumber = data[offset];
break;
case 2:
packetNumber = data.buffer.asByteData().getUint16(offset);
break;
case 4:
packetNumber = data.buffer.asByteData().getUint32(offset);
break;
case 8:
packetNumber = data.buffer.asByteData().getUint64(offset);
break;
default:
throw FormatException('Invalid packet number byte length derived from bits: $packetNumberByteLength');
}
offset += packetNumberByteLength;
final packetPayload = data.sublist(offset);
return QuicShortHeader(
headerForm: headerForm,
fixedBit: fixedBit,
spinBit: spinBit,
reservedBits: reservedBits,
keyPhase: keyPhase,
packetNumberLengthBits: packetNumberLengthBits,
destConnectionId: destConnectionId,
packetNumber: packetNumber,
packetPayload: packetPayload,
);
}
@override Uint8List toBytes() { final builder = BytesBuilder(); int firstByte = (headerForm << 7) | (fixedBit << 6) | (spinBit << 5) | (reservedBits << 3) | (keyPhase << 2) | packetNumberLengthBits; builder.addByte(firstByte); if (destConnectionId != null) { builder.add(destConnectionId!); }
// Add Packet Number (based on packetNumberLengthBits)
final packetNumberByteLength = 1 << packetNumberLengthBits;
switch (packetNumberByteLength) {
case 1:
builder.addByte(packetNumber);
break;
case 2:
builder.add(Uint8List(2)..buffer.asByteData().setUint16(0, packetNumber));
break;
case 4:
builder.add(Uint8List(4)..buffer.asByteData().setUint32(0, packetNumber));
break;
case 8:
builder.add(Uint8List(8)..buffer.asByteData().setUint64(0, packetNumber));
break;
}
builder.add(packetPayload);
return builder.toBytes();
}
@override
String toString() {
return 'QuicShortHeader(headerForm: $headerForm, fixedBit: $fixedBit, spinBit: $spinBit, reservedBits: $reservedBits, keyPhase: $keyPhase, packetNumberLengthBits:
Analyze this part as well: 0-RTT data
Latency is a key concern for transport protocols. The QUIC/TLS handshake that we have described until now allows the client and the server to agree on security keys within one round-trip-time. However, one round-trip-time can be a long delay for some applications. To minimize the impact of the connection setup time, QUIC allows applications to exchange data during the QUIC/TLS handshake. Such data is called 0-RTT data. To ensure that 0-RTT data is exchanged securely, the client and the server must have previously agreed on a key which can be used to encrypt and authenticate the 0-RTT data. Such a 0-RTT key could either be a pre-shared key that the client and the server have shared without using the QUIC protocol or, and this is the most frequent solution, the key that they negotiated during a previous connection. In practice, the server enables 0-RTT by sending a TLS session ticket to the client [26]. A session ticket is an encrypted record that contains information that enables the server to recover all the state information about a session including its session keys. It is also linked to the client’s address to enable the server to verify that a given client reuses the tickets that it has received earlier. Usually, these tickets also contain an expiration date. The client places the received ticket in the TLS CLient Hello that it sends in the first Initial packet. It uses the pre-shared key corresponding to this ticket to encrypt data and place it in one or more 0-RTT packets. The server uses the information contained in the ticket to recover the key and decrypt the data of the 0-RTT packet. Listing 7 shows the format of QUIC’s 0-RTT packet.
Listing 7 The QUIC 0-RTT packet
0-RTT Packet { Header Form (1) = 1, /* High order bit of the first byte / Fixed Bit (1) = 1, Long Packet Type (2) = 1, Reserved Bits (2), Packet Number Length (2), / Low order bits of the first byte */ Version (32), Destination Connection ID Length (8), Destination Connection ID (0..160), Source Connection ID Length (8), Source Connection ID (0..160), Length (i), Packet Number (8..32), Packet Payload (8..),}
The main benefit of these 0-RTT packets is that the client can immediately send encrypted data while sending its Initial packet. This is illustrated in Fig. 23 where the frames encrypted with the 0-RTT keys are shown in italics. Note that some of these frames can span several packets. 0-RTT packets are only sent by the QUIC client since the server can send encrypted data immediately after having sent its Handshake frames. As explained earlier, the Initial packets are also encrypted but using keys derived from the connection identifiers.
Fig. 23 Simplified QUIC Handshake with 0-RTT data
Note
Replay attacks and 0-RTT packets
Thanks to the 0-RTT packets, a client can send encrypted data to the server before waiting for the secure handshake. This reduces the latency of the data transfer, but with one important caveat. QUIC does not provide any guarantee that 0-RTT data will not be replayed. QUIC’s 0-RTT data exchanged is intended for idempotent operations. A detailed discussion of the impact of replay attacks may be found in [33].
To understand the importance of these replay attacks, let us consider a simple HTTP GET request. Such a request could easily fit inside the 0-RTT packet and thus have lower latency. If a web browser uses it to request a static index.html file, there is no harm if the request is received twice by the server. However, if the GET request is part of a REST API and has side effects, then problems could occur depending on the type of side effect. Consider a REST API that allows a user to switch off the lights using his or her smartphone. Replaying this request two or three times will always result in the light being switched off. However, if the user requests to increase the room temperature by one °C, then multiple replays will obviously have different consequences.
Let's integrate the information about "0-RTT data" and "Replay attacks and 0-RTT packets" into our analysis and Dart implementation.
Analysis of 0-RTT Data in QUIC The provided text introduces 0-RTT (Zero Round-Trip Time) data in QUIC, a crucial feature designed to reduce latency by allowing clients to send encrypted application data immediately, without waiting for the full TLS handshake to complete.
Key Concepts from the Text Latency Reduction: 0-RTT data aims to minimize connection setup time by enabling data exchange during the initial QUIC/TLS handshake, potentially reducing the round-trip delay for applications. Secure Exchange: For 0-RTT data to be secure, the client and server must have a pre-agreed key. This key is most commonly derived from a previous connection where the server sent a TLS session ticket to the client. TLS Session Tickets: An encrypted record sent by the server to the client after a successful connection. Contains state information, including session keys, allowing the server to quickly resume a session without a full handshake. Linked to the client's IP address to prevent ticket theft and reuse by unauthorized parties. Typically includes an expiration date. Client's Role in 0-RTT: The client stores the received session ticket. In a subsequent connection, the client includes this ticket in its TLS Client Hello, which is sent within the first Initial packet. The client uses the pre-shared key (derived from the ticket) to encrypt application data. This encrypted data is then placed into one or more 0-RTT packets. Server's Role in 0-RTT: The server receives the TLS Client Hello with the session ticket. It uses the information in the ticket to recover the previously negotiated key. This key is then used to decrypt the 0-RTT data in the incoming packets. 0-RTT Packet Format: Uses the Long Header format. Identified by Long Packet Type (2) = 1. Includes fields like Header Form, Fixed Bit, Reserved Bits, Packet Number Length, Version, Destination Connection ID Length, Destination Connection ID, Source Connection ID Length, Source Connection ID, Length (i) (a varint indicating total packet length), Packet Number (8..32) (a varint), and Packet Payload (8..). Client-Only Feature (for sending): Only the QUIC client sends 0-RTT packets because the server can immediately send encrypted data (1-RTT data) after sending its Handshake frames. Coalescing: The Length (i) field in the 0-RTT packet (a type of Long Header) is crucial for coalescing, allowing multiple 0-RTT packets and other handshake packets (Initial, Handshake) to be bundled into a single UDP datagram. This reduces per-packet overhead. Replay Attacks and 0-RTT Packets The text highlights a critical security consideration for 0-RTT data: it does not provide protection against replay attacks.
The Caveat: While 0-RTT data reduces latency, there's no guarantee that the data won't be replayed by an attacker. Idempotent Operations: Because of the replay risk, QUIC's 0-RTT data is primarily intended for idempotent operations. An idempotent operation is one that produces the same result whether executed once or multiple times (e.g., requesting a static webpage). Non-Idempotent Operations Risk: For operations with side effects that are not idempotent (e.g., "increase room temperature by 1°C"), replaying the request can lead to unintended consequences (e.g., repeatedly increasing temperature). Developers must be aware of this limitation and design their applications accordingly. Dart Implementation of QUIC 0-RTT Packet Header We'll create a new class QuicZeroRTTPacketHeader that extends QuicLongHeader, similar to how QuicInitialPacketHeader was structured. It will incorporate the specific Long Packet Type and the Length field for coalescing.
- QuicPacketHeader (Abstract Base Class) and VarInt These remain the same as in the previous response, providing the foundational elements for all QUIC headers and variable-length integer handling.
Dart
import 'dart:typed_data';
// Helper for reading/writing QUIC variable-length integers (varints) class VarInt { static int read(Uint8List data, int offset) { if (offset >= data.length) { throw FormatException('Attempted to read varint beyond data bounds.'); } final firstByte = data[offset]; final prefix = (firstByte >> 6) & 0x03; int value; int bytesRead;
if (prefix == 0x00) { // 1-byte varint
value = firstByte & 0x3F;
bytesRead = 1;
} else if (prefix == 0x01) { // 2-byte varint
if (offset + 1 >= data.length) throw FormatException('Incomplete 2-byte varint.');
value = data.buffer.asByteData().getUint16(offset) & 0x3FFF;
bytesRead = 2;
} else if (prefix == 0x02) { // 4-byte varint
if (offset + 3 >= data.length) throw FormatException('Incomplete 4-byte varint.');
value = data.buffer.asByteData().getUint32(offset) & 0x3FFFFFFF;
bytesRead = 4;
} else if (prefix == 0x03) { // 8-byte varint
if (offset + 7 >= data.length) throw FormatException('Incomplete 8-byte varint.');
value = data.buffer.asByteData().getUint64(offset) & 0x3FFFFFFFFFFFFFFF;
bytesRead = 8;
} else {
throw FormatException('Invalid varint prefix: $prefix');
}
return value;
}
static Uint8List write(int value) { final builder = BytesBuilder(); if (value < (1 << 6)) { // 1-byte builder.addByte(value & 0x3F); } else if (value < (1 << 14)) { // 2-byte builder.add(Uint8List(2)..buffer.asByteData().setUint16(0, value | 0x4000)); } else if (value < (1 << 30)) { // 4-byte builder.add(Uint8List(4)..buffer.asByteData().setUint32(0, value | 0x80000000)); } else if (value < (1 << 62)) { // 8-byte builder.add(Uint8List(8)..buffer.asByteData().setUint64(0, value | 0xC000000000000000)); } else { throw ArgumentError('Value $value is too large for a QUIC varint (max 2^62 - 1).'); } return builder.toBytes(); }
static int getLength(int value) { if (value < (1 << 6)) return 1; if (value < (1 << 14)) return 2; if (value < (1 << 30)) return 4; if (value < (1 << 62)) return 8; throw ArgumentError('Value $value is too large for a QUIC varint (max 2^62 - 1).'); } }
abstract class QuicPacketHeader { final int headerForm; final int fixedBit;
QuicPacketHeader({required this.headerForm, required this.fixedBit});
factory QuicPacketHeader.parse(Uint8List data, {int? shortHeaderDestConnectionIdLength}) { if (data.isEmpty) { throw ArgumentError('Packet data cannot be empty.'); } final firstByte = data[0]; final headerForm = (firstByte >> 7) & 0x01;
if (headerForm == 1) {
final longPacketType = (firstByte >> 4) & 0x03;
switch (longPacketType) {
case 0: // Initial Packet Type
return QuicInitialPacketHeader.parse(data);
case 1: // 0-RTT Packet Type
return QuicZeroRTTPacketHeader.parse(data);
// Add cases for Handshake and Retry packets if needed
default:
throw FormatException('Unknown Long Packet Type: $longPacketType');
}
} else {
if (shortHeaderDestConnectionIdLength == null) {
throw ArgumentError('Destination Connection ID Length must be provided for Short Headers.');
}
return QuicShortHeader.parse(data, destConnectionIdLength: shortHeaderDestConnectionIdLength);
}
}
Uint8List toBytes(); } 2. QuicLongHeader (No changes from previous response, as it's a generic base for all long headers)
Dart
class QuicLongHeader extends QuicPacketHeader { final int longPacketType; final int typeSpecificBits; final int version; final int destConnectionIdLength; final Uint8List? destConnectionId; final int sourceConnectionIdLength; final Uint8List? sourceConnectionId;
QuicLongHeader({ required int headerForm, required int fixedBit, required this.longPacketType, required this.typeSpecificBits, required this.version, required int destConnectionIdLength, Uint8List? destConnectionId, required int sourceConnectionIdLength, Uint8List? sourceConnectionId, }) : destConnectionIdLength = destConnectionId?.length ?? destConnectionIdLength, // Ensure length matches ID if provided sourceConnectionIdLength = sourceConnectionId?.length ?? sourceConnectionIdLength, // Ensure length matches ID if provided destConnectionId = destConnectionId, sourceConnectionId = sourceConnectionId, super(headerForm: headerForm, fixedBit: fixedBit);
factory QuicLongHeader.parse(Uint8List data) { if (data.length < 9) { throw FormatException('Insufficient data for a complete Long Header.'); }
int offset = 0;
final firstByte = data[offset++];
final headerForm = (firstByte >> 7) & 0x01;
final fixedBit = (firstByte >> 6) & 0x01;
final longPacketType = (firstByte >> 4) & 0x03;
final typeSpecificBits = firstByte & 0x0F;
final version = data.buffer.asByteData().getUint32(offset);
offset += 4;
final destConnectionIdLength = data[offset++];
Uint8List? destConnectionId;
if (destConnectionIdLength > 0) {
if (offset + destConnectionIdLength > data.length) {
throw FormatException('Malformed Long Header: Destination Connection ID length extends beyond data.');
}
destConnectionId = data.sublist(offset, offset + destConnectionIdLength);
offset += destConnectionIdLength;
}
final sourceConnectionIdLength = data[offset++];
Uint8List? sourceConnectionId;
if (sourceConnectionIdLength > 0) {
if (offset + sourceConnectionIdLength > data.length) {
throw FormatException('Malformed Long Header: Source Connection ID length extends beyond data.');
}
sourceConnectionId = data.sublist(offset, offset + sourceConnectionIdLength);
offset += sourceConnectionIdLength;
}
return QuicLongHeader(
headerForm: headerForm,
fixedBit: fixedBit,
longPacketType: longPacketType,
typeSpecificBits: typeSpecificBits,
version: version,
destConnectionIdLength: destConnectionIdLength,
destConnectionId: destConnectionId,
sourceConnectionIdLength: sourceConnectionIdLength,
sourceConnectionId: sourceConnectionId,
);
}
@override Uint8List toBytes() { final builder = BytesBuilder(); int firstByte = (headerForm << 7) | (fixedBit << 6) | (longPacketType << 4) | typeSpecificBits; builder.addByte(firstByte); builder.add(Uint8List(4)..buffer.asByteData().setUint32(0, version)); builder.addByte(destConnectionIdLength); if (destConnectionId != null) { builder.add(destConnectionId!); } builder.addByte(sourceConnectionIdLength); if (sourceConnectionId != null) { builder.add(sourceConnectionId!); } return builder.toBytes(); }
@override
String toString() {
return 'QuicLongHeader(headerForm: $headerForm, fixedBit: $fixedBit, longPacketType: $longPacketType, typeSpecificBits:
Dart
class QuicInitialPacketHeader extends QuicLongHeader { final int reservedBits; final int packetNumberLengthBits; final int tokenLength; final Uint8List? token; final int length; // Total length of the packet number + payload (varint) final int packetNumber; final Uint8List packetPayload;
QuicInitialPacketHeader({ required int headerForm, required int fixedBit, required int longPacketType, required this.reservedBits, required this.packetNumberLengthBits, required int version, required int destConnectionIdLength, Uint8List? destConnectionId, required int sourceConnectionIdLength, Uint8List? sourceConnectionId, required this.tokenLength, this.token, required this.length, required this.packetNumber, required this.packetPayload, }) : super( headerForm: headerForm, fixedBit: fixedBit, longPacketType: longPacketType, typeSpecificBits: (reservedBits << 2) | packetNumberLengthBits, version: version, destConnectionIdLength: destConnectionIdLength, destConnectionId: destConnectionId, sourceConnectionIdLength: sourceConnectionIdLength, sourceConnectionId: sourceConnectionId, );
factory QuicInitialPacketHeader.parse(Uint8List data) { int offset = 0; final firstByte = data[offset++];
final headerForm = (firstByte >> 7) & 0x01;
final fixedBit = (firstByte >> 6) & 0x01;
final longPacketType = (firstByte >> 4) & 0x03;
final reservedBits = (firstByte >> 2) & 0x03;
final packetNumberLengthBits = firstByte & 0x03;
final version = data.buffer.asByteData().getUint32(offset);
offset += 4;
final destConnectionIdLength = data[offset++];
Uint8List? destConnectionId;
if (destConnectionIdLength > 0) {
destConnectionId = data.sublist(offset, offset + destConnectionIdLength);
offset += destConnectionIdLength;
}
final sourceConnectionIdLength = data[offset++];
Uint8List? sourceConnectionId;
if (sourceConnectionIdLength > 0) {
sourceConnectionId = data.sublist(offset, offset + sourceConnectionIdLength);
offset += sourceConnectionIdLength;
}
final tokenLength = VarInt.read(data, offset);
offset += VarInt.getLength(tokenLength);
Uint8List? token;
if (tokenLength > 0) {
token = data.sublist(offset, offset + tokenLength);
offset += tokenLength;
}
final length = VarInt.read(data, offset);
offset += VarInt.getLength(length);
final packetNumberByteLength = 1 << packetNumberLengthBits;
int packetNumber;
switch (packetNumberByteLength) {
case 1:
packetNumber = data[offset];
break;
case 2:
packetNumber = data.buffer.asByteData().getUint16(offset);
break;
case 4:
packetNumber = data.buffer.asByteData().getUint32(offset);
break;
case 8:
packetNumber = data.buffer.asByteData().getUint64(offset);
break;
default:
throw FormatException('Invalid packet number byte length derived from bits: $packetNumberByteLength');
}
offset += packetNumberByteLength;
final packetPayload = data.sublist(offset);
return QuicInitialPacketHeader(
headerForm: headerForm,
fixedBit: fixedBit,
longPacketType: longPacketType,
reservedBits: reservedBits,
packetNumberLengthBits: packetNumberLengthBits,
version: version,
destConnectionIdLength: destConnectionIdLength,
destConnectionId: destConnectionId,
sourceConnectionIdLength: sourceConnectionIdLength,
sourceConnectionId: sourceConnectionId,
tokenLength: tokenLength,
token: token,
length: length,
packetNumber: packetNumber,
packetPayload: packetPayload,
);
}
@override Uint8List toBytes() { final builder = BytesBuilder(); int firstByte = (headerForm << 7) | (fixedBit << 6) | (longPacketType << 4) | (reservedBits << 2) | packetNumberLengthBits; builder.addByte(firstByte); builder.add(Uint8List(4)..buffer.asByteData().setUint32(0, version)); builder.addByte(destConnectionIdLength); if (destConnectionId != null) { builder.add(destConnectionId!); } builder.addByte(sourceConnectionIdLength); if (sourceConnectionId != null) { builder.add(sourceConnectionId!); }
builder.add(VarInt.write(tokenLength));
if (token != null) {
builder.add(token!);
}
builder.add(VarInt.write(length));
final packetNumberByteLength = 1 << packetNumberLengthBits;
switch (packetNumberByteLength) {
case 1:
builder.addByte(packetNumber);
break;
case 2:
builder.add(Uint8List(2)..buffer.asByteData().setUint16(0, packetNumber));
break;
case 4:
builder.add(Uint8List(4)..buffer.asByteData().setUint32(0, packetNumber));
break;
case 8:
builder.add(Uint8List(8)..buffer.asByteData().setUint64(0, packetNumber));
break;
}
builder.add(packetPayload);
return builder.toBytes();
}
@override
String toString() {
return 'QuicInitialPacketHeader(headerForm: $headerForm, fixedBit: $fixedBit, longPacketType: $longPacketType, reservedBits: $reservedBits, packetNumberLengthBits:
Dart
class QuicZeroRTTPacketHeader extends QuicLongHeader { final int reservedBits; final int packetNumberLengthBits; final int length; // Total length of the packet number + payload (varint) final int packetNumber; final Uint8List packetPayload;
QuicZeroRTTPacketHeader({ required int headerForm, required int fixedBit, required int longPacketType, // Should be 1 for 0-RTT required this.reservedBits, required this.packetNumberLengthBits, required int version, required int destConnectionIdLength, Uint8List? destConnectionId, required int sourceConnectionIdLength, Uint8List? sourceConnectionId, required this.length, required this.packetNumber, required this.packetPayload, }) : super( headerForm: headerForm, fixedBit: fixedBit, longPacketType: longPacketType, // For 0-RTT Packet, Type-Specific Bits are 2 Reserved + 2 Packet Number Length typeSpecificBits: (reservedBits << 2) | packetNumberLengthBits, version: version, destConnectionIdLength: destConnectionIdLength, destConnectionId: destConnectionId, sourceConnectionIdLength: sourceConnectionIdLength, sourceConnectionId: sourceConnectionId, );
factory QuicZeroRTTPacketHeader.parse(Uint8List data) { int offset = 0; final firstByte = data[offset++];
final headerForm = (firstByte >> 7) & 0x01; // Should be 1
final fixedBit = (firstByte >> 6) & 0x01; // Should be 1
final longPacketType = (firstByte >> 4) & 0x03; // Should be 1 for 0-RTT
final reservedBits = (firstByte >> 2) & 0x03;
final packetNumberLengthBits = firstByte & 0x03;
final version = data.buffer.asByteData().getUint32(offset);
offset += 4;
final destConnectionIdLength = data[offset++];
Uint8List? destConnectionId;
if (destConnectionIdLength > 0) {
destConnectionId = data.sublist(offset, offset + destConnectionIdLength);
offset += destConnectionIdLength;
}
final sourceConnectionIdLength = data[offset++];
Uint8List? sourceConnectionId;
if (sourceConnectionIdLength > 0) {
sourceConnectionId = data.sublist(offset, offset + sourceConnectionIdLength);
offset += sourceConnectionIdLength;
}
// Parse Length (varint) - this is the total length of the Packet Number and Packet Payload
final length = VarInt.read(data, offset);
offset += VarInt.getLength(length);
// Parse Packet Number (length determined by packetNumberLengthBits)
final packetNumberByteLength = 1 << packetNumberLengthBits;
int packetNumber;
switch (packetNumberByteLength) {
case 1:
packetNumber = data[offset];
break;
case 2:
packetNumber = data.buffer.asByteData().getUint16(offset);
break;
case 4:
packetNumber = data.buffer.asByteData().getUint32(offset);
break;
case 8:
packetNumber = data.buffer.asByteData().getUint64(offset);
break;
default:
throw FormatException('Invalid packet number byte length derived from bits: $packetNumberByteLength');
}
offset += packetNumberByteLength;
final packetPayload = data.sublist(offset);
return QuicZeroRTTPacketHeader(
headerForm: headerForm,
fixedBit: fixedBit,
longPacketType: longPacketType,
reservedBits: reservedBits,
packetNumberLengthBits: packetNumberLengthBits,
version: version,
destConnectionIdLength: destConnectionIdLength,
destConnectionId: destConnectionId,
sourceConnectionIdLength: sourceConnectionIdLength,
sourceConnectionId: sourceConnectionId,
length: length,
packetNumber: packetNumber,
packetPayload: packetPayload,
);
}
@override Uint8List toBytes() { final builder = BytesBuilder(); int firstByte = (headerForm << 7) | (fixedBit << 6) | (longPacketType << 4) | (reservedBits << 2) | packetNumberLengthBits; builder.addByte(firstByte); builder.add(Uint8List(4)..buffer.asByteData().setUint32(0, version)); builder.addByte(destConnectionIdLength); if (destConnectionId != null) { builder.add(destConnectionId!); } builder.addByte(sourceConnectionIdLength); if (sourceConnectionId != null) { builder.add(sourceConnectionId!); }
builder.add(VarInt.write(length));
final packetNumberByteLength = 1 << packetNumberLengthBits;
switch (packetNumberByteLength) {
case 1:
builder.addByte(packetNumber);
break;
case 2:
builder.add(Uint8List(2)..buffer.asByteData().setUint16(0, packetNumber));
break;
case 4:
builder.add(Uint8List(4)..buffer.asByteData().setUint32(0, packetNumber));
break;
case 8:
builder.add(Uint8List(8)..buffer.asByteData().setUint64(0, packetNumber));
break;
}
builder.add(packetPayload);
return builder.toBytes();
}
@override
String toString() {
return 'QuicZeroRTTPacketHeader(headerForm: $headerForm, fixedBit: $fixedBit, longPacketType: $longPacketType, reservedBits: $reservedBits, packetNumberLengthBits:
Dart
class QuicShortHeader extends QuicPacketHeader { final int spinBit; final int reservedBits; final int keyPhase; final int packetNumberLengthBits; final Uint8List? destConnectionId; final int packetNumber; final Uint8List packetPayload;
QuicShortHeader({ required int headerForm, required int fixedBit, required this.spinBit, required this.reservedBits, required this.keyPhase, required this.packetNumberLengthBits, this.destConnectionId, required this.packetNumber, required this.packetPayload, }) : super(headerForm: headerForm, fixedBit: fixedBit);
factory QuicShortHeader.parse(Uint8List data, {required int destConnectionIdLength}) { int offset = 0; final firstByte = data[offset++];
final headerForm = (firstByte >> 7) & 0x01;
final fixedBit = (firstByte >> 6) & 0x01;
final spinBit = (firstByte >> 5) & 0x01;
final reservedBits = (firstByte >> 3) & 0x03;
final keyPhase = (firstByte >> 2) & 0x01;
final packetNumberLengthBits = firstByte & 0x03;
Uint8List? destConnectionId;
if (destConnectionIdLength > 0) {
if (offset + destConnectionIdLength > data.length) {
throw FormatException('Malformed Short Header: Destination Connection ID length extends beyond data.');
}
destConnectionId = data.sublist(offset, offset + destConnectionIdLength);
offset += destConnectionIdLength;
}
final packetNumberByteLength = 1 << packetNumberLengthBits;
int packetNumber;
switch (packetNumberByteLength) {
case 1:
packetNumber = data[offset];
break;
case 2:
packetNumber = data.buffer.asByteData().getUint16(offset);
break;
case 4:
packetNumber = data.buffer.asByteData().getUint32(offset);
break;
case 8:
packetNumber = data.buffer.asByteData().getUint64(offset);
break;
default:
throw FormatException('Invalid packet number byte length derived from bits: $packetNumberByteLength');
}
offset += packetNumberByteLength;
final packetPayload = data.sublist(offset);
return QuicShortHeader(
headerForm: headerForm,
fixedBit: fixedBit,
spinBit: spinBit,
reservedBits: reservedBits,
keyPhase: keyPhase,
packetNumberLengthBits: packetNumberLengthBits,
destConnectionId: destConnectionId,
packetNumber: packetNumber,
packetPayload: packetPayload,
);
}
@override Uint8List toBytes() { final builder = BytesBuilder(); int firstByte = (headerForm << 7) | (fixedBit << 6) | (spinBit << 5) | (reservedBits << 3) | (keyPhase << 2) | packetNumberLengthBits; builder.addByte(firstByte); if (destConnectionId != null) { builder.add(destConnectionId!); }
final packetNumberByteLength = 1 << packetNumberLengthBits;
switch (packetNumberByteLength) {
case 1:
builder.addByte(packetNumber);
break;
case 2:
builder.add(Uint8List(2)..buffer.asByteData().setUint16(0, packetNumber));
break;
case 4:
builder.add(Uint8List(4)..buffer.asByteData().setUint32(0, packetNumber));
break;
case 8:
builder.add(Uint8List(8)..buffer.asByteData().setUint64(0, packetNumber));
break;
}
builder.add(packetPayload);
return builder.toBytes();
}
@override
String toString() {
return 'QuicShortHeader(headerForm: $headerForm, fixedBit: $fixedBit, spinBit: $spinBit, reservedBits: $reservedBits, keyPhase: $keyPhase, packetNumberLengthBits:
Analyze this part now: Exchanging data over a QUIC connection
The data exchanged over is QUIC connection is organized in different streams. A stream is a sequence of bytes. QUIC supports both unidirectional and bidirectional streams. Both the client and the server can create new streams over a QUIC connection. Each stream is identified by a stream identifier. To prevent problems when the client and the server try to create a stream using the same identifier, QUIC restricts the utilization of the stream identifiers based its two low-order bits. A QUIC client can only create streams whose two low order bits are set to 00 (bidirectional stream) or 10 (unidirectional stream). Similarly, the low order bits of the server-initiated streams must be set to 01 (bidirectional stream) or 11 (unidirectional streams). The QUIC streams are created implicitly by sending data over the stream with the chosen identifier. The stream identifiers start at the minimum value, i.e. 0x00 to 0x03 for the respective types. If a host sends stream data for stream x before having sent data over the lower-numbered streams of that type, then those streams are implicitly created. The stream identifier is encoded using a variable length integer. The largest possible stream identifier is thus 262−1
.
QUIC places all data inside STREAM frames that are then placed inside QUIC packets. The structure of a STREAM frame is shown in Listing 8. This frame contains the following information :
the Type of the Stream frame 1
the identifier of the stream
the offset, i.e. the position of the first byte of the Stream data in the bytestream
the length of the data
the Stream Data
Listing 8 The QUIC STREAM frame
STREAM Frame { Type (i) = 0x08..0x0f, Stream ID (i), Offset (i), Length (i), Stream Data (..),}
The STREAM frame carries data, but it can also terminate the corresponding stream. The lowest order bit of the Type field acts as a FIN bit. When set to zero, it indicates that subsequent data will be sent over this stream. When set to one, it indicates that the STREAM frame contains the last bytes sent over that stream. The stream is closed once the last byte of the stream has been delivered to the user application. Once a QUIC stream has been closed, it cannot be reused again over this connection.
Using this information, the receiver can easily reassemble the data received over the different streams. As an illustration, let us consider a server that has created two streams (stream 1 and 5). The server sends ABCD.. over stream 1 and 123 over stream 5 and closes it after having sent the third digit. The data from these streams could be encoded as shown in Listing 9.
Listing 9 Data sent over two different streams
STREAM Frame { Type (i) = 0x0e, Stream ID = 1, Offset = 0 Length = 2 Stream Data = AB}STREAM Frame { Type (i) = 0x0e, Stream ID = 5, Offset = 0 Length = 1 Stream Data = 1}STREAM Frame { Type (i) = 0x0e, Stream ID = 1, Offset = 2 Length = 1 Stream Data = C}STREAM Frame { Type (i) = 0x0f, /* FIN bit is set, end of stream */ Stream ID = 5, Offset = 1 Length = 2 Stream Data = 23}STREAM Frame { Type (i) = 0x0e, Stream ID = 1, Offset = 3 Length = 4 Stream Data = DEFG}
The penultimate frame shown in Listing 9 has the FIN flag set. It marks the end of stream 1 that has transported three bytes in total. The FIN flag is the normal way to gracefully close a QUIC stream.
There are however cases where an application might need to cancel a stream abruptly without closing the corresponding connection. First, the sending side of a stream can decide to reset the stream. For this, it sends a RESET_STREAM frame that carries the identifier of the stream that is canceled. The receiving side of a stream might also need to stop receiving data over a given stream. Consider for example a web browser that has started to download the different images that compose a web while the user has already clicked on another page from the same server. The streams that correspond to these images become useless. In this case, our browser can send a STOP_SENDING frame to indicate that it discards the incoming data over the corresponding streams. Upon reception of this frame, the server sends a RESET_STREAM frame to indicate that the corresponding stream has been closed.
The provided text describes how data is exchanged over a QUIC connection, focusing on the concept of streams and the STREAM frame. It also touches upon stream management, including implicit creation, graceful closure with a FIN bit, and abrupt termination using RESET_STREAM and STOP_SENDING frames.
Data Exchange over a QUIC Connection QUIC organizes data exchange using streams, which are essentially ordered sequences of bytes. This offers a more granular control over data flow compared to traditional protocols.
QUIC Streams Stream Characteristics: Byte Sequences: Each stream acts as an independent sequence of bytes. Unidirectional and Bidirectional: QUIC supports both types of streams. A bidirectional stream allows data flow in both directions between client and server, while a unidirectional stream allows data flow in only one direction (either client-to-server or server-to-client). Initiation: Both the client and the server can initiate new streams. Implicit Creation: Streams are created implicitly when a host sends data on a stream ID that hasn't been used yet, or before sending data on lower-numbered streams of that type. Stream Identifiers (Stream IDs): Each stream has a unique identifier. These are variable-length integers, allowing for a vast range of possible stream IDs (up to 2 62 −1). ID Restrictions: To prevent collisions when both client and server try to create streams, QUIC enforces rules based on the two least significant bits of the stream ID: Client-Initiated: 00: Bidirectional stream 10: Unidirectional stream Server-Initiated: 01: Bidirectional stream 11: Unidirectional stream Starting IDs: Stream IDs begin at 0x00 to 0x03 for their respective types. The STREAM Frame All application data in QUIC is encapsulated within STREAM frames, which are then placed inside QUIC packets. This modular approach allows for flexible data transmission.
Structure of a STREAM Frame (Listing 8): STREAM Frame { Type (i) = 0x08..0x0f, /* A variable-length integer, indicating frame type and flags / Stream ID (i), / Variable-length integer for the stream identifier / Offset (i), / Variable-length integer, position of first byte in stream / Length (i), / Variable-length integer, length of the Stream Data / Stream Data (..), / The actual application data bytes */ } Frame Type and FIN Bit: The Type field of the STREAM frame is a variable-length integer whose lowest order bit acts as a FIN bit: FIN = 0 (Type 0x08 to 0x0E): Indicates that more data will follow on this stream. FIN = 1 (Type 0x0F): Indicates that this STREAM frame contains the last bytes of data for this stream. The stream is considered closed once all bytes up to this FIN bit are delivered to the application. Stream Closure: Once a QUIC stream is closed (via FIN bit or abrupt termination), it cannot be reused on the same connection. Data Reassembly: The Stream ID and Offset fields allow the receiver to correctly reassemble the data, even if frames arrive out of order or data for different streams is interleaved within the same or different packets. Stream Termination QUIC provides mechanisms for both graceful and abrupt stream termination.
Graceful Closure (FIN Bit): The primary method for gracefully closing a stream is by setting the FIN bit in the last STREAM frame sent on that stream. Abrupt Termination: RESET_STREAM Frame: A sender can use this frame to abruptly cancel a stream. It includes the Stream ID of the canceled stream. STOP_SENDING Frame: A receiver can send this frame to indicate that it no longer wishes to receive data on a particular stream (e.g., if the user navigates away from a page while images are still downloading). Upon receiving STOP_SENDING, the peer (sender) is expected to respond with a RESET_STREAM frame to confirm the closure of that stream. Dart Implementation Considerations for Streams and STREAM Frames Implementing QUIC streams and STREAM frames in Dart will involve defining classes for the STREAM frame and incorporating logic for managing stream IDs, offsets, and handling FIN bits.
- QuicStreamFrame Class This class will represent the STREAM frame itself, responsible for carrying the application data.
Dart
import 'dart:typed_data'; // Assuming VarInt helper is available from previous steps // import 'path/to/varint_helper.dart';
class QuicStreamFrame { final int type; // This is a varint, and its lowest bit is the FIN bit final int streamId; // Varint final int offset; // Varint final int length; // Varint, length of streamData final Uint8List streamData;
// Constructor QuicStreamFrame({ required this.type, required this.streamId, required this.offset, required this.length, required this.streamData, }) { if (streamData.length != length) { throw ArgumentError('StreamData length does not match specified length.'); } }
// Check if the FIN bit is set bool get isFinSet => (type & 0x01) == 0x01;
// Factory constructor for parsing from bytes factory QuicStreamFrame.parse(Uint8List data, int startOffset) { int currentOffset = startOffset;
// Read Type (varint)
final type = VarInt.read(data, currentOffset);
currentOffset += VarInt.getLength(type);
// Read Stream ID (varint)
final streamId = VarInt.read(data, currentOffset);
currentOffset += VarInt.getLength(streamId);
// Read Offset (varint)
final offset = VarInt.read(data, currentOffset);
currentOffset += VarInt.getLength(offset);
// Read Length (varint)
final length = VarInt.read(data, currentOffset);
currentOffset += VarInt.getLength(length);
// Read Stream Data
if (currentOffset + length > data.length) {
throw FormatException('Malformed STREAM frame: data length exceeds bounds.');
}
final streamData = data.sublist(currentOffset, currentOffset + length);
currentOffset += length;
return QuicStreamFrame(
type: type,
streamId: streamId,
offset: offset,
length: length,
streamData: streamData,
);
}
// Method to serialize the frame into bytes Uint8List toBytes() { final builder = BytesBuilder(); builder.add(VarInt.write(type)); builder.add(VarInt.write(streamId)); builder.add(VarInt.write(offset)); builder.add(VarInt.write(length)); builder.add(streamData); return builder.toBytes(); }
@override
String toString() {
return 'QuicStreamFrame(Type: 0x${type.toRadixString(16)}, Stream ID: $streamId, Offset: $offset, Length: $length, FIN:
Stream ID Generation: Logic to generate stream IDs based on whether it's client or server initiated, and whether it's unidirectional or bidirectional, ensuring the correct low-order bits. Stream State Tracking: Keeping track of each stream's highest received offset, whether it's gracefully closed (FIN received), or abruptly terminated. Reassembly Buffer: For each stream, maintaining a buffer to reassemble incoming Stream Data segments based on their Offset values. Handling RESET_STREAM and STOP_SENDING: Implementing logic to send/receive these control frames and update stream states accordingly. These would be separate frame types placed within QUIC packets, similar to how STREAM frames are. Example for Stream ID Generation (Conceptual) Dart
// Part of a hypothetical QuicConnection class class QuicStreamManager { int _clientBidirectionalStreamId = 0; // 00 int _clientUnidirectionalStreamId = 2; // 10 int _serverBidirectionalStreamId = 1; // 01 int _serverUnidirectionalStreamId = 3; // 11
int getNextClientBidirectionalStreamId() { final id = _clientBidirectionalStreamId; _clientBidirectionalStreamId += 4; // Increment by 4 to maintain '00' suffix return id; }
int getNextClientUnidirectionalStreamId() { final id = _clientUnidirectionalStreamId; _clientUnidirectionalStreamId += 4; // Increment by 4 to maintain '10' suffix return id; }
// ... similar for server-initiated streams if acting as a server } Integration with Packet Headers The QuicPacketHeader classes we defined earlier would contain a packetPayload (a Uint8List). In a real QUIC implementation, this packetPayload would be parsed into a sequence of QUIC frames. The QuicStreamFrame would be one type of frame that can be found in this payload.
Dart
// Conceptual snippet within a QuicPacket class QuicPacket { QuicPacketHeader header; List frames; // This list would hold various frame objects, including QuicStreamFrame
QuicPacket.fromBytes(Uint8List rawBytes, {int? shortHeaderDestConnectionIdLength}) { int offset = 0; header = QuicPacketHeader.parse(rawBytes, shortHeaderDestConnectionIdLength: shortHeaderDestConnectionIdLength); // After parsing header, the remaining bytes are frames offset += header.toBytes().length; // Get length of parsed header (this would be more complex with varints)
Uint8List frameData = rawBytes.sublist(offset);
frames = _parseFrames(frameData); // Call a function to parse individual frames
}
List _parseFrames(Uint8List data) { final List parsedFrames = []; int offset = 0; while (offset < data.length) { // Logic to determine frame type from first byte/varint of the frame // For this example, assuming all are STREAM frames for simplicity. // In reality, you'd read the frame type and dispatch to appropriate parser. try { final streamFrame = QuicStreamFrame.parse(data, offset); parsedFrames.add(streamFrame); offset += streamFrame.toBytes().length; // Advance offset by frame's total length } catch (e) { print('Error parsing frame: $e'); break; // Stop if a malformed frame is encountered } } return parsedFrames; }
// ... other methods } The concept of streams is fundamental to QUIC's multiplexing capabilities, allowing multiple independent data flows over a single connection, significantly enhancing efficiency and reducing head-of-line blocking.
Flow control in QUIC
Transport protocols usually allocate some resources to each established connection. Each QUIC connection requires memory to store its state, but also buffers to store the packets arrived out-of-order. In practice, the memory available for QUIC implementations is not unlimited and a QUIC receiver must control the amount of packets that the remote host can send at any time. QUIC supports flow control techniques that operate at different levels.
The first level is the connection level. During the handshake, each host can announce the maximum number of bytes that it agrees to receive initially on the connection using the initial_max_data transport parameter. This parameter contains the number of bytes that the sending host agrees to receive without further notice. If the connection uses more bytes than initially agreed, the receiver can update this limit by sending a MAX_DATA frame at any time. This frame contains a variable length integer that encodes the maximum amount of stream data that can be sent over the connection.
The utilization of different streams also consumes resources on a QUIC host. A receiver can also restrict the number of streams that the remote host can create. During the handshake, the initial_max_streams_bidi and initial_max_streams_uni transport parameters announce the maximum number of bidirectional and unidirectional streams that the receiving host can accept. This limit can be modified during the connection by sending a MAX_STREAMS frame that updates the limit.
Flow control can also take place at the stream level. During the handshake, several transport parameters allow the hosts to advertise the maximum number of bytes that they agree to receive on each stream. Different transport parameters are used to specify the limits that apply to the local/remote and unidirectional/bidirectional streams. These limits can be updated during the connection by sending MAX_STREAM_DATA frames. Each of these frames indicates the maximum amount of stream data that can be accepted on a given stream.
These limits restrict the number of streams that a host can create and the amount of bytes that it can send. If a host is blocked by any of these limits, it may sent a control frame to request the remote host to extend the limit. For each type of flow control, there is an associated control frame which can be used to request an extension of the limit.
A host should send a DATA_BLOCKED frame when it reaches the limit on the maximum amount of data set by the initial_max_data transport parameter or a previously received MAX_DATA frame. The DATA_BLOCKED frame contains the connection limit that caused the transmission to be blocked. In practice, a receiving host should increase the connection-level limit by sending MAX_DATA frames before reaching the limit. However, since this limit is function of the available memory, a host might not always be able to send a MAX_DATA frame. Fig. 25 provides an example packet flow with the utilization of these frames. We assume that the initial_max_data transport parameter was set to 100 bytes by the client during the handshake and the the server needs to send 900 bytes. The server creates a stream and sends 100 bytes in a 1-RTT packet carrying a STREAM frame. At this point, the server is blocked.
Fig. 25 QUIC uses DATA_BLOCKED frames when a connection’s flow control is blocked
The same applies with the STREAM_DATA_BLOCKED frame that is sent when a host reaching the per-stream limit. The STREAMS_BLOCKED frame is used when a host has reached the maximum number of established streams.
Note
Connections blocked by flow control are not totally idle
If a QUIC host detects that a connection is blocked by flow control, it should regularly send DATA_BLOCKED or STREAM_DATA_BLOCKED frames to request the remote host to extend the current limit. The QUIC specification [25] does not recommend precisely how often these frames should be sent when a connection is blocked by flow control. Experience with QUIC deployments will enable the QUIC developers to more precisely define how often these frames should be sent.
A measurement study [36] revealed that QUIC implementations used different strategies for flow control. They identified three main types of behaviors :
Static Flow Control: the receive buffer size stays unchanged and the receiver simply increases the maximum allowance linearly
Growing Flow Control: the size of the receive buffer increases over time as the connection progresses
Auto-tuning: the size of the receive buffer is adjusted dynamically based on the estimated bandwidth and round-trip times
In 2020, three quarter of the studied implementations used a Static Flow Control and only one used auto-tuning. Static Flow Control can work, but this is a waste of memory. For example, Google’s Chrome starts with a 15 MBytes receive buffer and updates it linearly [36]. This should be contrasted with TCP implementations that usually start with a window of a few tens of KBytes and adjust it with auto-tuning. Auto-tuning is a key feature of modern TCP implementations [37]. It is expected that QUIC implementations will include more advanced heuristics to tune their flow control mechanism.
Let's analyze the provided text on "Flow control in QUIC" and outline how to incorporate its concepts into a Dart implementation.
Analysis of Flow Control in QUIC The text highlights QUIC's multi-layered flow control mechanism, designed to manage resource allocation (primarily memory for state and out-of-order packets) and prevent a sender from overwhelming a receiver. This is crucial for maintaining connection stability and performance.
Levels of Flow Control QUIC implements flow control at three distinct levels:
Connection Level:
Initial Limit: Established during the handshake via the initial_max_data transport parameter. This sets the initial maximum number of bytes the receiver is willing to accept across the entire connection. Updates: The receiver can increase this limit at any time by sending a MAX_DATA frame. This frame contains a variable-length integer indicating the new absolute maximum amount of stream data that can be sent over the connection. Blocking: If the sender reaches this limit, it is blocked from sending further application data until the receiver extends the limit. Stream Creation Level:
Initial Limit: During the handshake, the receiver announces the maximum number of new streams it can accept, separately for bidirectional (initial_max_streams_bidi) and unidirectional (initial_max_streams_uni) streams. Updates: This limit can be updated during the connection by sending a MAX_STREAMS frame, which specifies the new maximum. Blocking: If a sender attempts to create more streams than allowed by this limit, it is blocked. Stream Level:
Initial Limit: Transport parameters advertise the initial maximum number of bytes the receiver is willing to accept on each individual stream. These limits can differ for local/remote and unidirectional/bidirectional streams. Updates: These per-stream limits are updated by the receiver sending MAX_STREAM_DATA frames. Each MAX_STREAM_DATA frame specifies the maximum amount of stream data that can be accepted on a given stream ID. Blocking: If a sender attempts to send more bytes on a specific stream than its allowed limit, it is blocked on that stream. Flow Control Frames QUIC uses specific control frames to manage these limits and signal when a sender is blocked:
MAX_DATA (from receiver to sender): Increases the connection-level flow control limit. MAX_STREAMS (from receiver to sender): Increases the stream creation limit (for either bidirectional or unidirectional streams). MAX_STREAM_DATA (from receiver to sender): Increases the per-stream flow control limit for a specific stream. DATA_BLOCKED (from sender to receiver): Sent by a sender when it's blocked by the connection-level flow control limit (initial_max_data or MAX_DATA). It includes the limit that caused the blocking. Receivers should ideally send MAX_DATA before the limit is reached. STREAM_DATA_BLOCKED (from sender to receiver): Sent by a sender when it's blocked by a per-stream flow control limit. STREAMS_BLOCKED (from sender to receiver): Sent by a sender when it's blocked from creating new streams (either bidirectional or unidirectional, depending on the type of limit reached). Flow Control Strategies The "Note" section on "Connections blocked by flow control are not totally idle" emphasizes that a blocked sender should periodically send DATA_BLOCKED or STREAM_DATA_BLOCKED frames. This serves as a persistent request for the receiver to extend the limit, even if the receiver hasn't sent an update.
The text also mentions different flow control strategies observed in QUIC implementations:
Static Flow Control: The receive buffer size remains constant, and the allowance is increased linearly. (e.g., Google Chrome's 15 MB initial buffer). This is simple but can be memory-inefficient. Growing Flow Control: The receive buffer size increases over time as the connection progresses. Auto-tuning: The receive buffer size is dynamically adjusted based on estimated bandwidth and round-trip times. This is more complex but more efficient, akin to modern TCP's congestion control. The expectation is that QUIC will move towards more advanced auto-tuning heuristics. Dart Implementation for QUIC Flow Control Implementing QUIC flow control in Dart requires managing state for these limits on both the sender and receiver sides, and defining classes for the flow control frames.
- Transport Parameters (for Initial Limits) These would typically be part of a QuicConnectionParameters or similar class exchanged during the handshake.
Dart
// Placeholder for initial transport parameters class QuicTransportParameters { final int initialMaxData; final int initialMaxStreamsBidi; final int initialMaxStreamsUni; // Per-stream limits would also be here, likely more complex structures final int initialMaxStreamDataBidiLocal; final int initialMaxStreamDataBidiRemote; final int initialMaxStreamDataUniLocal; final int initialMaxStreamDataUniRemote;
QuicTransportParameters({ this.initialMaxData = 0, // Default to 0, actual value set by spec/implementation this.initialMaxStreamsBidi = 0, this.initialMaxStreamsUni = 0, this.initialMaxStreamDataBidiLocal = 0, this.initialMaxStreamDataBidiRemote = 0, this.initialMaxStreamDataUniLocal = 0, this.initialMaxStreamDataUniRemote = 0, });
// Serialization/deserialization methods for transport parameters would go here // (beyond the scope of just flow control frames, but they set the initial state). } 2. Flow Control Frame Definitions These frames are distinct from STREAM frames and would be part of the packetPayload along with STREAM frames.
Dart
import 'dart:typed_data'; // Assume VarInt helper from previous steps is available // import 'path/to/varint_helper.dart';
// Abstract base class for all QUIC Frames abstract class QuicFrame { final int type; // Varint for frame type
QuicFrame(this.type);
Uint8List toBytes(); // Factory for parsing any frame type (would be implemented in a FrameParser utility) factory QuicFrame.parse(Uint8List data, int offset) { final frameType = VarInt.read(data, offset); switch (frameType) { case 0x01: // PING (example, not discussed but common control frame) // return QuicPingFrame.parse(data, offset); case 0x04: // MAX_DATA return QuicMaxDataFrame.parse(data, offset); case 0x05: // MAX_STREAM_DATA return QuicMaxStreamDataFrame.parse(data, offset); case 0x06: // MAX_STREAMS (Bidirectional) case 0x07: // MAX_STREAMS (Unidirectional) return QuicMaxStreamsFrame.parse(data, offset); case 0x14: // DATA_BLOCKED return QuicDataBlockedFrame.parse(data, offset); case 0x15: // STREAM_DATA_BLOCKED return QuicStreamDataBlockedFrame.parse(data, offset); case 0x16: // STREAMS_BLOCKED (Bidirectional) case 0x17: // STREAMS_BLOCKED (Unidirectional) return QuicStreamsBlockedFrame.parse(data, offset); case int _: // STREAM frames from 0x08 to 0x0F if (frameType >= 0x08 && frameType <= 0x0F) { return QuicStreamFrame.parse(data, offset); } break; default: // Handle unknown or unimplemented frame types throw FormatException('Unknown QUIC Frame Type: 0x${frameType.toRadixString(16)}'); } throw FormatException('Failed to parse QUIC Frame at offset $offset'); } }
// -------------------- Receiver-Sent Flow Control Frames --------------------
class QuicMaxDataFrame extends QuicFrame { static const int TYPE = 0x04; final int maximumData; // Varint
QuicMaxDataFrame({required this.maximumData}) : super(TYPE);
factory QuicMaxDataFrame.parse(Uint8List data, int offset) { int currentOffset = offset; final type = VarInt.read(data, currentOffset); currentOffset += VarInt.getLength(type); if (type != TYPE) throw FormatException('Invalid frame type for MaxData Frame.');
final maximumData = VarInt.read(data, currentOffset);
currentOffset += VarInt.getLength(maximumData);
return QuicMaxDataFrame(maximumData: maximumData);
}
@override Uint8List toBytes() { final builder = BytesBuilder(); builder.add(VarInt.write(type)); builder.add(VarInt.write(maximumData)); return builder.toBytes(); }
@override String toString() => 'MaxDataFrame(maxData: $maximumData)'; }
class QuicMaxStreamDataFrame extends QuicFrame { static const int TYPE = 0x05; final int streamId; // Varint final int maximumStreamData; // Varint
QuicMaxStreamDataFrame({required this.streamId, required this.maximumStreamData}) : super(TYPE);
factory QuicMaxStreamDataFrame.parse(Uint8List data, int offset) { int currentOffset = offset; final type = VarInt.read(data, currentOffset); currentOffset += VarInt.getLength(type); if (type != TYPE) throw FormatException('Invalid frame type for MaxStreamData Frame.');
final streamId = VarInt.read(data, currentOffset);
currentOffset += VarInt.getLength(streamId);
final maximumStreamData = VarInt.read(data, currentOffset);
currentOffset += VarInt.getLength(maximumStreamData);
return QuicMaxStreamDataFrame(streamId: streamId, maximumStreamData: maximumStreamData);
}
@override Uint8List toBytes() { final builder = BytesBuilder(); builder.add(VarInt.write(type)); builder.add(VarInt.write(streamId)); builder.add(VarInt.write(maximumStreamData)); return builder.toBytes(); }
@override String toString() => 'MaxStreamDataFrame(streamId: $streamId, maxStreamData: $maximumStreamData)'; }
class QuicMaxStreamsFrame extends QuicFrame { // Types: 0x06 for Bidirectional, 0x07 for Unidirectional final int maximumStreams; // Varint
QuicMaxStreamsFrame.bidi({required int maximumStreams}) : this._internal(0x06, maximumStreams); QuicMaxStreamsFrame.uni({required int maximumStreams}) : this._internal(0x07, maximumStreams);
QuicMaxStreamsFrame._internal(int type, this.maximumStreams) : super(type);
factory QuicMaxStreamsFrame.parse(Uint8List data, int offset) { int currentOffset = offset; final type = VarInt.read(data, currentOffset); currentOffset += VarInt.getLength(type); if (type != 0x06 && type != 0x07) throw FormatException('Invalid frame type for MaxStreams Frame.');
final maximumStreams = VarInt.read(data, currentOffset);
currentOffset += VarInt.getLength(maximumStreams);
return QuicMaxStreamsFrame._internal(type, maximumStreams);
}
@override Uint8List toBytes() { final builder = BytesBuilder(); builder.add(VarInt.write(type)); builder.add(VarInt.write(maximumStreams)); return builder.toBytes(); }
@override String toString() => 'MaxStreamsFrame(type: ${type == 0x06 ? 'Bidi' : 'Uni'}, maxStreams: $maximumStreams)'; }
// -------------------- Sender-Sent Flow Control Blocked Frames --------------------
class QuicDataBlockedFrame extends QuicFrame { static const int TYPE = 0x14; final int connectionLimit; // Varint, the limit that caused the blocking
QuicDataBlockedFrame({required this.connectionLimit}) : super(TYPE);
factory QuicDataBlockedFrame.parse(Uint8List data, int offset) { int currentOffset = offset; final type = VarInt.read(data, currentOffset); currentOffset += VarInt.getLength(type); if (type != TYPE) throw FormatException('Invalid frame type for DataBlocked Frame.');
final connectionLimit = VarInt.read(data, currentOffset);
currentOffset += VarInt.getLength(connectionLimit);
return QuicDataBlockedFrame(connectionLimit: connectionLimit);
}
@override Uint8List toBytes() { final builder = BytesBuilder(); builder.add(VarInt.write(type)); builder.add(VarInt.write(connectionLimit)); return builder.toBytes(); }
@override String toString() => 'DataBlockedFrame(limit: $connectionLimit)'; }
class QuicStreamDataBlockedFrame extends QuicFrame { static const int TYPE = 0x15; final int streamId; // Varint final int streamDataLimit; // Varint, the limit that caused the blocking on this stream
QuicStreamDataBlockedFrame({required this.streamId, required this.streamDataLimit}) : super(TYPE);
factory QuicStreamDataBlockedFrame.parse(Uint8List data, int offset) { int currentOffset = offset; final type = VarInt.read(data, currentOffset); currentOffset += VarInt.getLength(type); if (type != TYPE) throw FormatException('Invalid frame type for StreamDataBlocked Frame.');
final streamId = VarInt.read(data, currentOffset);
currentOffset += VarInt.getLength(streamId);
final streamDataLimit = VarInt.read(data, currentOffset);
currentOffset += VarInt.getLength(streamDataLimit);
return QuicStreamDataBlockedFrame(streamId: streamId, streamDataLimit: streamDataLimit);
}
@override Uint8List toBytes() { final builder = BytesBuilder(); builder.add(VarInt.write(type)); builder.add(VarInt.write(streamId)); builder.add(VarInt.write(streamDataLimit)); return builder.toBytes(); }
@override String toString() => 'StreamDataBlockedFrame(streamId: $streamId, limit: $streamDataLimit)'; }
class QuicStreamsBlockedFrame extends QuicFrame { // Types: 0x16 for Bidirectional, 0x17 for Unidirectional final int streamLimit; // Varint, the limit that caused the blocking for stream creation
QuicStreamsBlockedFrame.bidi({required int streamLimit}) : this._internal(0x16, streamLimit); QuicStreamsBlockedFrame.uni({required int streamLimit}) : this._internal(0x17, streamLimit);
QuicStreamsBlockedFrame._internal(int type, this.streamLimit) : super(type);
factory QuicStreamsBlockedFrame.parse(Uint8List data, int offset) { int currentOffset = offset; final type = VarInt.read(data, currentOffset); currentOffset += VarInt.getLength(type); if (type != 0x16 && type != 0x17) throw FormatException('Invalid frame type for StreamsBlocked Frame.');
final streamLimit = VarInt.read(data, currentOffset);
currentOffset += VarInt.getLength(streamLimit);
return QuicStreamsBlockedFrame._internal(type, streamLimit);
}
@override Uint8List toBytes() { final builder = BytesBuilder(); builder.add(VarInt.write(type)); builder.add(VarInt.write(streamLimit)); return builder.toBytes(); }
@override String toString() => 'StreamsBlockedFrame(type: ${type == 0x16 ? 'Bidi' : 'Uni'}, limit: $streamLimit)'; } 3. Flow Control State Management (Conceptual) This is where the actual logic for tracking current allowances, sending updates, and reacting to blocked signals would reside. This would likely be part of a QuicConnection or QuicFlowControlManager class.
Dart
class QuicFlowControlManager { // Connection-level int _localConnectionMaxData; // Max data we can receive int _remoteConnectionMaxData; // Max data remote can receive (set by remote's MAX_DATA frame) int _connectionDataSent = 0; // Total application data bytes sent int _connectionDataReceived = 0; // Total application data bytes received
// Stream creation limits int _localMaxStreamsBidi; int _localMaxStreamsUni; int _remoteMaxStreamsBidi; int _remoteMaxStreamsUni; int _activeStreamsBidi = 0; int _activeStreamsUni = 0;
// Per-stream limits (would need a map per stream ID) // Map<int, int> _localStreamMaxData = {}; // Max data we can receive on a stream // Map<int, int> _remoteStreamMaxData = {}; // Max data remote can receive on a stream // Map<int, int> _streamDataSent = {}; // Map<int, int> _streamDataReceived = {};
Function(QuicFrame) _sendFrameCallback; // Callback to send a frame
QuicFlowControlManager(QuicTransportParameters initialParams, this._sendFrameCallback) : _localConnectionMaxData = initialParams.initialMaxData, _remoteConnectionMaxData = 0, // This is updated by the remote's first MAX_DATA (or its initial_max_data) _localMaxStreamsBidi = initialParams.initialMaxStreamsBidi, _localMaxStreamsUni = initialParams.initialMaxStreamsUni, _remoteMaxStreamsBidi = 0, _remoteMaxStreamsUni = 0;
// --- Methods for Sender Side (what we are allowed to send) ---
void onRemoteMaxDataFrame(int maxData) { _remoteConnectionMaxData = maxData; // Log or react if we were previously blocked. // If _connectionDataSent is now < _remoteConnectionMaxData, we are unblocked. }
void onRemoteMaxStreamsFrame(int type, int maxStreams) { if (type == 0x06) { _remoteMaxStreamsBidi = maxStreams; } else { _remoteMaxStreamsUni = maxStreams; } // React if we were blocked from creating streams. }
void onRemoteMaxStreamDataFrame(int streamId, int maxStreamData) { // _remoteStreamMaxData[streamId] = maxStreamData; // React if this specific stream was blocked. }
bool canSendStreamData(int streamId, int length) { // Check connection-level limit if (_connectionDataSent + length > _remoteConnectionMaxData) { _sendFrameCallback(QuicDataBlockedFrame(connectionLimit: _remoteConnectionMaxData)); return false; // Blocked at connection level }
// Check per-stream limit (conceptual)
// if (_streamDataSent[streamId]! + length > _remoteStreamMaxData[streamId]!) {
// _sendFrameCallback(QuicStreamDataBlockedFrame(streamId: streamId, streamDataLimit: _remoteStreamMaxData[streamId]!));
// return false; // Blocked at stream level
// }
return true; // Can send
}
void onStreamDataSent(int streamId, int length) { _connectionDataSent += length; // _streamDataSent[streamId] = (_streamDataSent[streamId] ?? 0) + length; }
bool canCreateStream(bool isBidirectional) { if (isBidirectional) { if (_activeStreamsBidi >= _remoteMaxStreamsBidi) { _sendFrameCallback(QuicStreamsBlockedFrame.bidi(streamLimit: _remoteMaxStreamsBidi)); return false; } } else { if (_activeStreamsUni >= _remoteMaxStreamsUni) { _sendFrameCallback(QuicStreamsBlockedFrame.uni(streamLimit: _remoteMaxStreamsUni)); return false; } } return true; }
void onStreamCreated(bool isBidirectional) { if (isBidirectional) { _activeStreamsBidi++; } else { _activeStreamsUni++; } }
// --- Methods for Receiver Side (what we are expecting to receive) ---
void onStreamDataReceived(int streamId, int offset, int length) { _connectionDataReceived += length; // _streamDataReceived[streamId] = (_streamDataReceived[streamId] ?? 0) + length;
// Proactive flow control: send MAX_DATA/MAX_STREAM_DATA before limits are hit
if (_connectionDataReceived + (0.5 * _localConnectionMaxData) > _localConnectionMaxData) {
// Example: If we've received 50% of our capacity, extend it
_localConnectionMaxData += 10000; // Extend by some amount
_sendFrameCallback(QuicMaxDataFrame(maximumData: _localConnectionMaxData));
}
// Similar logic for per-stream data limits
}
void onDataBlockedFrame(int limit) { // Remote host is blocked by our connection limit. // This is a strong signal to increase _localConnectionMaxData and send MAX_DATA. _localConnectionMaxData += 20000; // React to blocking _sendFrameCallback(QuicMaxDataFrame(maximumData: _localConnectionMaxData)); }
void onStreamDataBlockedFrame(int streamId, int limit) { // Remote host is blocked by our per-stream limit on 'streamId'. // Increase _localStreamMaxData[streamId] and send MAX_STREAM_DATA. }
void onStreamsBlockedFrame(int type, int limit) { // Remote host is blocked by our stream creation limit. // Increase _localMaxStreamsBidi/_localMaxStreamsUni and send MAX_STREAMS. }
// Periodic check to send blocked frames if we are indeed blocked (as per the note) void periodicallyCheckBlockedStatus() { if (_connectionDataSent >= _remoteConnectionMaxData) { _sendFrameCallback(QuicDataBlockedFrame(connectionLimit: _remoteConnectionMaxData)); } // Similar checks for stream data and stream creation limits } } Integration with QuicPacket Parsing: The _parseFrames method within QuicPacket (from the previous response) would be updated to use the QuicFrame.parse factory, allowing it to correctly identify and parse all these different frame types.
Summary of Implementation Strategy: Define Frame Classes: Create distinct Dart classes for each flow control frame (MAX_DATA, MAX_STREAM_DATA, MAX_STREAMS, DATA_BLOCKED, STREAM_DATA_BLOCKED, STREAMS_BLOCKED), all extending a common QuicFrame abstract class. VarInt Helper: Continue to leverage the VarInt helper for reading/writing the variable-length integer fields within these frames. Flow Control Manager: Implement a QuicFlowControlManager to centralize the logic for: Tracking current allowed limits (local and remote). Incrementing byte/stream counters as data is sent/received. Deciding when to send MAX_DATA, MAX_STREAM_DATA, MAX_STREAMS frames to extend limits (proactive approach). Recognizing when we are blocked and sending DATA_BLOCKED, STREAM_DATA_BLOCKED, or STREAMS_BLOCKED frames. Reacting to incoming blocked frames by increasing local limits. Implementing the periodic sending of BLOCKED frames when stuck. Transport Parameter Integration: The initial_max_data and initial_max_streams values will initialize the flow control state. Packet/Frame Parsing: Ensure the main packet parsing logic correctly identifies and dispatches to the appropriate frame parsers for both STREAM frames and all the new flow control frames. This multi-layered approach to flow control makes QUIC highly adaptable and efficient, allowing fine-grained control over resource consumption and preventing congestion, while also providing mechanisms for transparently requesting more capacity when needed.
QUIC Loss Detection
As other transport protocols, QUIC includes mechanisms to detect transmission errors and packet losses. The transmission errors are usually detected by using the UDP checksum or at the datalink layer with the Wi-Fi or Ethernet CRCs. When a transmission error occurs, the corresponding packet is discarded and QUIC considers this error as a packet loss. Researchers have analyzed the performance of checksums and CRCs on real data [38].
Second, since QUIC used AEAD encryption schemes, all QUIC packets are authenticated and a receiver can leverage this AEAD to detect transmission errors that were undetected by the UDP checksum of the CRC of the lower layers. However, these undetected transmission errors are assumed to be rare and if QUIC a detects an invalid AEAD, it will consider that this error was caused by an attack and will stop the connection using a TLS alert [26].
There are several important differences between the loss detection and retransmission mechanisms used by QUIC and other transport protocols. First, QUIC packet numbers always increase monotonically over a QUIC connection. A QUIC sender never sends twice a packet with the same packet number over a given connection. QUIC encodes the packet numbers as variable length integers and it does not support wrap around in contrast with other transport protocols. The QUIC frames contain the valuable information that needs to be delivered reliably. If a QUIC packet is lost, the frames that it contained will be retransmitted in another QUIC packet that uses a different packet number. Thus, the QUIC packet number serves as a unique identifier of a packet. This simplifies some operations such as measuring the round-trip-time which is more difficult in protocols such as TCP when packets are transmitted [39].
Second, QUIC’s acknowledgments carry more information than the cumulative or selective acknowledgments used by TCP and related protocols. This enables the receiver to provide a more detailed view of the packets that it received. In contrast with TCP [11], once a receiver has reported that one packet was correctly received in an acknowledgment, the sender of that packet can discard the corresponding frames.
Third, a QUIC sender autonomously decides which frames it sends inside each packet. A QUIC packet may contain both data and control frames, or only data or only control information. If a QUIC packet is lost, the frames that it contained could be retransmitted in different packets. A QUIC implementation thus needs to buffer the frames and mark the in-flight ones to be able to retransmit them if the corresponding packet was lost.
Fourth, most QUIC packets are explicitly acknowledged. The only exception are the packets that only contain ACK, PADDING or CONNECTION_CLOSE frames. A packet that contains any other QUIC frame is called an ack-eliciting packet because its delivery will be confirmed by the transmission of an acknowledgment. A QUIC packet that carries both an ACK and a STREAM frame will thus be acknowledged.
With this in mind, it is interesting to look at the format of the QUIC acknowledgments and then analyze how they can be used. Listing 10 provides the format of an ACK frame. It can be sent at any time in a QUIC packet. Two types are used to distinguish between the acknowledgments that contain information about the received ECN flags (type 0x03) or only regular acknowledgments (type 0x02). The first information contained in the ACK frame is the largest packet number that is acknowledged by this ACK frame. This is usually the highest packet number received. The second information is the ACK delay. This is the delay in microseconds between the reception of the packet having the largest acknowledged number by the receiver and the transmission of the acknowledgment. This information is important to ensure that round-trip-times are accurately measured, even if a receiver delays acknowledgments. This is illustrated in Fig. 26. The ACK Range Count field contains the number of ACK ranges that are included in the QUIC ACK frame. This number can be set to zero if all packets were received in sequence without any gap. In this case, the First ACK Range field contains the number of the packet that arrived before the Largest Acknowledged packet number.
Listing 10 The QUIC ACK Frame
ACK Frame { Type (i) = 0x02..0x03, Largest Acknowledged (i), ACK Delay (i), ACK Range Count (i), First ACK Range (i), ACK Range (..) ..., [ECN Counts (..)],}
Fig. 26 Utilization of the QUIC ACK delay
An ACK frame contains 0 or more ACK Ranges. The format of an ACK range is shown in Listing 11. Each range indicates first the number of unacknowledged packets since the smallest acknowledged packet in the preceding range (or the first ACK range). The next field indicates the number of consecutive acknowledged packets.
Listing 11 A QUIC ACK range
ACK Range { Gap (i), ACK Range Length (i),}
As an example, consider a host that received the following QUIC packets: 3,4,6,7,8,9,11,14,16,18. To report all the received packets, it will generate the ACK frame shown in Listing 12.
Listing 12 Sample QUIC ACK Frame
ACK Frame { Type (i) = 0x02, Largest Acknowledged=18, ACK Delay=x, ACK Range Count=5, First ACK Range=0, ACK Range #0 [Gap=2, ACK Range Length=1], ACK Range #1 [Gap=2, ACK Range Length=1], ACK Range #2 [Gap=3, ACK Range Length=1], ACK Range #3 [Gap=2, ACK Range Length=4], ACK Range #4 [Gap=2, ACK Range Length=2]}
The QUIC specification recommends to send one ACK frame after having received two ack-eliciting packets. This corresponds roughly to TCP’s delayed acknowledgments strategy. However, there is ongoing work to allow the sender to provide more guidelines on when and how ACK frames should be sent [40].
Note
When should QUIC hosts send acknowledgments
A measurement study [36] analyzed how QUIC implementations generate acknowledgments. Two of the studied implementations sent acknowledgments every N packets (2 for one implementation and 10 for the other). Other implementations used ack frequencies that varied during the data transfer.
Fig. 27 Acknowledgment frequencies for different QUIC servers
The acknowledgment frequencies should be compared with TCP that usually acknowledges every second packet. It is likely that QUIC implementations will tune the generation of their acknowledgments in the coming years based on feedback from deployment.
It is interesting to observe that since the ACK frames are sent inside QUIC packets, they can also be acknowledged. Sending an ACK in response to another ACK could result in an infinite exchange of ACK frames. To prevent this problem, a QUIC sender cannot send an ACK frame in response to a non-eliciting QUIC packet and the ACK frames are one of the non-eliciting frame types. Note that if a receiver that receives many STREAM frames and thus sends many ACK frames wants to obtain information about the reception of its ACK frame, it can simply send one ACK frame inside a packet that contains an eliciting frame, e.g. a PING frame. This frame will trigger the receiver to acknowledge it and the previously sent ACK frames.
In contrast with other reliable transport protocols, QUIC does not use cumulative acknowledgments. As explained earlier, QUIC never retransmits a packet with the same packet number. When a packet is lost, this creates a gap that the receiver reports using an ACK Range. Such a gap will never be filled by retransmissions and obviously should not be reported by the receiver forever. In practice, a receiver will send the acknowledgment that corresponds to a given packet number several times and then will assume that the acknowledgment has been received. A receiver can also rely on other heuristics to determine that a given ACK Range should not be reported anymore. This is the case if the ACK frame was included in a packet that has been acknowledged by the other peer, but also when the gap was noticed several round-trip times ago.
QUIC also allows a receiver to send information about the ECN flags in the received packets. Two flags of the IP header [41] are reserved to indicate support for Explicit Congestion Notification. The QUIC ECN count field shown in Listing 13 contains three counters for the different values of the ECN flags. These counters are incremented upon the reception of each QUIC packet based on the values of the ECN flag of the received packet. Unfortunately, there are still many operational problems when using ECN in the global Internet [42]. Time will tell whether it is easier to deploy ECN with QUIC than with TCP.
Listing 13 A QUIC ECN Count
ECN Counts { ECT0 Count (i), ECT1 Count (i), ECN-CE Count (i),}
Note
QUIC also acknowledges control frames
Besides the STREAM frames that carry user data, QUIC uses several different frame types to exchange control information. These control frames, like the data frames, are ack-eliciting frames. This implies a host that receives such a frame needs to acknowledge it using an ACK frame.
Fig. 29 illustrates the beginning of a QUIC connection with the exchange of the Initial packets and the corresponding acknowledgments. The client sends its TLS Client Hello inside a CRYPTO frame in an Initial packet. This is the first packet sent by the client and thus its packet number is 0. The server replies with a TLS Server Hello inside a CRYPTO frame in an Initial packet. Since this is the first packet sent by the server, its packet number is also 0. The packet also contains an ACK frame that acknowledges the reception of the packet containing the TLS Client Hello.
The Handshake, 0-RTT and 1-RTT packets are acknowledged similarly using ACK frames. Handshake packets are acknowledged in other Handshake packets while 0-RTT and 1-RTT packets are acknowledged inside 1-RTT packets.
Fig. 29 QUIC also acknowledges Initial frames
Note
Not all QUIC servers use 0 as the packet number of their first Initial packet
The example shows a QUIC connection where the client sent its Initial packet with packet number 0 and the server also replied with a packet number set to 0. This is what most QUIC implementations do. However, the QUIC specification does not strictly requires this. In fact, facebook servers in October 2022 appear to use random packet numbers for the Initial packet that they sent in response to a client. This is probably use to detect or mitigate some forms of attacks since the client must receive the server’s Initial packet to be able to produce a valid acknowledgment.
To illustrate how QUIC uses acknowledgments, let us consider a simple QUIC connection. The client starts a QUIC connection with a new server, sends a request, receives a response and then closes the connection. There are no losses in this connection. Fig. 30 illustrates this connection.
Fig. 30 Acknowledgments in a short QUIC connection
The connection starts when the client sends an Initial packet containing a CRYPTO frame with the TLS Client Hello. The server replies with an Initial packet that contains an acknowledgment and a CRYPTO frame with the TLS Server Hello. The server then sends an Initial packet containing the TLS Encrypted Extensions. Since this is the first Initial packet, its packet number is set to 0. In practice, it is likely that the server will need to send several packets to carry the certificates contained in this packet. Note that the server cannot send more than 3 packets in response to the client’s Initial packet. This limit was introduced in the QUIC specification to restrict the ability of attackers to trigger DDoS attacks by sending spoofed packets to QUIC servers [25]. If the CRYPTO frame containing the certificates is too long, the server might need to wait for acknowledgments from the client before sending its last Handshake packets. The client confirms the reception of the server’s Initial packet by sending its last Initial packet that contains an ACK frame. This is the second Initial packet sent by the client and its packet number is thus 1. Since this packet only contains an ACK frame, the server does not return an acknowledgment to confirm its reception.
The client replies to the Handshake packet with another Handshake packet that contains a CRYPTO frame and acknowledges the Handshake packets sent by the server. The client’s Handshake packet is acknowledged by the server. The server starts the data exchange by sending a 1-RTT packet carrying one or more stream frames to create the required streams. Since this is the first 1-RTT packet sent by the server, its packet number is set to 0. The client then sends its request in a STREAM frame. The server replies with a 1-RTT packet that contains a STREAM frame with its response. The client ends the connection by sending a CONNECTION_CLOSE frame. In the example, the server replies with a CONNECTION_CLOSE frame, but the QUIC specification [25] only indicates that a host may respond with a CONNECTION_CLOSE in this case.
Let's analyze the provided text about "QUIC Loss Detection" and outline how to approach its implementation in Dart.
Analysis of QUIC Loss Detection The provided text details QUIC's mechanisms for detecting transmission errors and packet losses, emphasizing key differences from traditional transport protocols like TCP. It focuses heavily on packet numbers, acknowledgments, and their role in reliable delivery and RTT measurement.
Key Aspects of QUIC Loss Detection Error Detection:
Checksums/CRCs: Standard UDP checksums and lower-layer (Wi-Fi, Ethernet) CRCs are used for initial error detection. AEAD Authentication: QUIC leverages its AEAD (Authenticated Encryption with Associated Data) encryption. If a packet fails AEAD decryption/authentication, it's considered an integrity error. Crucially, if AEAD fails, QUIC assumes an attack and tears down the connection with a TLS alert, rather than treating it as a simple loss for retransmission. This distinguishes it from protocols that might retransmit on checksum failure. Packet Numbers and Retransmission:
Monotonically Increasing Packet Numbers: QUIC packet numbers always increase. A sender never reuses a packet number for a given connection. This is a fundamental difference from TCP, where retransmissions might use the same sequence number. No Wrap-Around: Packet numbers are variable-length integers and do not wrap around, simplifying RTT measurement and loss detection. Frame-Level Reliability: Unlike TCP, which retransmits segments, QUIC retransmits frames. If a QUIC packet is lost, the frames it contained are retransmitted in a new QUIC packet with a different packet number. This means the packet number uniquely identifies a specific transmission instance of a set of frames, not the data itself. Buffering Frames: A QUIC implementation must buffer the frames that are in-flight (sent but not yet acknowledged) to allow for retransmission if the packet carrying them is lost. Acknowledgments (ACK Frames):
Richer Information: QUIC's ACK frames provide more detailed information than TCP's cumulative or selective acknowledgments. This allows the receiver to convey a precise view of received packets. Discarding Acknowledged Frames: Once a receiver reports a packet as received in an ACK, the sender can safely discard the corresponding frames from its retransmission buffer. ACK Frame Format (Listing 10): Type (i) = 0x02..0x03: 0x02 for regular, 0x03 if ECN flags are present. Largest Acknowledged (i): Highest packet number received by the acknowledger. ACK Delay (i): Delay in microseconds between receiving the Largest Acknowledged packet and sending this ACK. This is vital for accurate RTT measurement, even with delayed acknowledgments. ACK Range Count (i): Number of subsequent ACK ranges. First ACK Range (i): Number of packets contiguous with Largest Acknowledged that were also received. ACK Range (..) ...: List of Gap and ACK Range Length (Listing 11) pairs to report received packets. [ECN Counts (..)]: Optional, if Type is 0x03. Contains ECT0 Count, ECT1 Count, ECN-CE Count for Explicit Congestion Notification. Acknowledgment Eliciting Packets:
Most QUIC packets are ack-eliciting, meaning their reception must trigger an ACK from the receiver. Exceptions: Packets containing only ACK, PADDING, or CONNECTION_CLOSE frames are non-ack-eliciting. They do not require a response. Avoiding ACK Storms: A sender will not send an ACK in response to a non-ack-eliciting packet. To get an ACK for an ACK frame, one must include it with an ack-eliciting frame (e.g., a PING frame). ACK Frame Generation Strategy:
Delayed ACKs: The specification recommends sending an ACK after receiving two ack-eliciting packets, similar to TCP's delayed ACK strategy. Variability: Implementations can vary their ACK frequency (e.g., every N packets, or dynamically). Future Tuning: It's expected that ACK generation heuristics will be refined based on deployment experience. Non-Cumulative Acknowledgments:
QUIC does not use cumulative acknowledgments. Gaps reported by ACK ranges will never be filled by retransmissions (since retransmissions get new packet numbers). Gap Management: Receivers need to stop reporting old gaps. This can happen if the ACK frame itself is acknowledged, or if the gap was noticed several RTTs ago. ECN Support:
QUIC ACK frames can carry ECN (Explicit Congestion Notification) information, specifically counts of ECT0, ECT1, and ECN-CE marked packets. This allows congestion signals to be conveyed without packet loss. Acknowledging Control Frames:
All ack-eliciting frames, including control frames (e.g., CRYPTO, PING, STREAM, CONNECTION_CLOSE unless it's the only frame) trigger acknowledgments. This ensures reliable delivery of control information. Initial Connection Handshake Acknowledgment Example (Fig. 29, 30):
Client sends Initial (PN=0) with CRYPTO (Client Hello). Server replies with Initial (PN=0) containing ACK (PN=0 of client) and CRYPTO (Server Hello). Server sends more Initial packets (e.g., PN=1, PN=2) with CRYPTO (Certificates, etc.). Server must manage DDoS risk by limiting initial packets. Client sends Initial (PN=1) containing ACK of server's Initial packets. Since this client packet contains only an ACK frame, the server does not acknowledge it back. This flow continues for Handshake, 0-RTT, and 1-RTT packets. Notable Detail: First Initial Packet Number While commonly both client and server start their Initial packet numbers at 0, the QUIC specification does not strictly mandate this. Facebook's servers, for instance, use random initial packet numbers for security/mitigation purposes.
Dart Implementation for QUIC Loss Detection Implementing QUIC loss detection will involve defining the ACK frame, tracking sent and received packets, managing the retransmission buffer, and implementing RTT measurement.
- QuicAckFrame Class This class will represent the ACK frame, including its ranges and optional ECN counts.
Dart
import 'dart:typed_data'; // Assume VarInt helper from previous steps is available // import 'path/to/varint_helper.dart';
// Definition for a single ACK Range class QuicAckRange { final int gap; // Varint: number of unacknowledged packets since previous acknowledged range final int ackRangeLength; // Varint: number of consecutive acknowledged packets
QuicAckRange({required this.gap, required this.ackRangeLength});
Uint8List toBytes() { final builder = BytesBuilder(); builder.add(VarInt.write(gap)); builder.add(VarInt.write(ackRangeLength)); return builder.toBytes(); }
@override String toString() => 'Gap: $gap, Length: $ackRangeLength'; }
// Definition for ECN Counts (optional in ACK frame) class QuicEcnCounts { final int ect0Count; // Varint final int ect1Count; // Varint final int ecnCeCount; // Varint
QuicEcnCounts({required this.ect0Count, required this.ect1Count, required this.ecnCeCount});
Uint8List toBytes() { final builder = BytesBuilder(); builder.add(VarInt.write(ect0Count)); builder.add(VarInt.write(ect1Count)); builder.add(VarInt.write(ecnCeCount)); return builder.toBytes(); }
@override String toString() => 'ECT0: $ect0Count, ECT1: $ect1Count, ECN-CE: $ecnCeCount'; }
class QuicAckFrame extends QuicFrame { // Types: 0x02 (regular), 0x03 (with ECN) final int largestAcknowledged; // Varint final int ackDelay; // Varint (microseconds) final int ackRangeCount; // Varint final int firstAckRange; // Varint final List ackRanges; // List of ACK Range objects final QuicEcnCounts? ecnCounts; // Optional ECN counts
QuicAckFrame({ required int type, required this.largestAcknowledged, required this.ackDelay, required this.ackRangeCount, required this.firstAckRange, required this.ackRanges, this.ecnCounts, }) : super(type) { if ((type == 0x03 && ecnCounts == null) || (type == 0x02 && ecnCounts != null)) { throw ArgumentError('ACK frame type mismatch with ECN counts presence.'); } if (ackRanges.length != ackRangeCount) { throw ArgumentError('ACK Range Count does not match actual ACK ranges provided.'); } }
factory QuicAckFrame.parse(Uint8List data, int offset) { int currentOffset = offset;
final type = VarInt.read(data, currentOffset);
currentOffset += VarInt.getLength(type);
if (type != 0x02 && type != 0x03) throw FormatException('Invalid frame type for ACK Frame.');
final largestAcknowledged = VarInt.read(data, currentOffset);
currentOffset += VarInt.getLength(largestAcknowledged);
final ackDelay = VarInt.read(data, currentOffset);
currentOffset += VarInt.getLength(ackDelay);
final ackRangeCount = VarInt.read(data, currentOffset);
currentOffset += VarInt.getLength(ackRangeCount);
final firstAckRange = VarInt.read(data, currentOffset);
currentOffset += VarInt.getLength(firstAckRange);
final List<QuicAckRange> ackRanges = [];
for (int i = 0; i < ackRangeCount; i++) {
final gap = VarInt.read(data, currentOffset);
currentOffset += VarInt.getLength(gap);
final ackRangeLength = VarInt.read(data, currentOffset);
currentOffset += VarInt.getLength(ackRangeLength);
ackRanges.add(QuicAckRange(gap: gap, ackRangeLength: ackRangeLength));
}
QuicEcnCounts? ecnCounts;
if (type == 0x03) { // If ECN type, parse ECN Counts
final ect0Count = VarInt.read(data, currentOffset);
currentOffset += VarInt.getLength(ect0Count);
final ect1Count = VarInt.read(data, currentOffset);
currentOffset += VarInt.getLength(ect1Count);
final ecnCeCount = VarInt.read(data, currentOffset);
currentOffset += VarInt.getLength(ecnCeCount);
ecnCounts = QuicEcnCounts(ect0Count: ect0Count, ect1Count: ect1Count, ecnCeCount: ecnCeCount);
}
return QuicAckFrame(
type: type,
largestAcknowledged: largestAcknowledged,
ackDelay: ackDelay,
ackRangeCount: ackRangeCount,
firstAckRange: firstAckRange,
ackRanges: ackRanges,
ecnCounts: ecnCounts,
);
}
@override Uint8List toBytes() { final builder = BytesBuilder(); builder.add(VarInt.write(type)); builder.add(VarInt.write(largestAcknowledged)); builder.add(VarInt.write(ackDelay)); builder.add(VarInt.write(ackRangeCount)); builder.add(VarInt.write(firstAckRange)); for (var range in ackRanges) { builder.add(range.toBytes()); } if (ecnCounts != null) { builder.add(ecnCounts!.toBytes()); } return builder.toBytes(); }
@override
String toString() {
String ackRangesStr = ackRanges.map((r) => '[${r.toString()}]').join(', ');
String ecnStr = ecnCounts != null ? ', ECN:
Dart
// Modify QuicPacketHeader (already defined in previous response) // Add a method to check if a frame type is ack-eliciting extension QuicFrameTypeExtension on int { bool get isAckEliciting { // List of non-ack-eliciting frame types const Set nonAckElicitingTypes = { 0x02, // ACK 0x03, // ACK with ECN 0x01, // PADDING (usually) - check QUIC spec for exact rules 0x1C, // CONNECTION_CLOSE (with Error Code) - This frame can be ack-eliciting depending on context, // but the text specifically states if a packet only contains CC, it's non-eliciting. // For simplicity here, we'll follow the text's direct example. }; return !nonAckElicitingTypes.contains(this); } }
class QuicPacket { final QuicPacketHeader header; final List frames; final int packetNumber; // The actual packet number, not just the encoded length bits final int timeSent; // Timestamp when this packet was sent (for RTT calculation) bool acknowledged = false; bool inFlight = false; // Whether it's currently considered in-flight
QuicPacket({ required this.header, required this.frames, required this.packetNumber, required this.timeSent, });
// Determines if this packet requires an ACK from the receiver bool get isAckEliciting { return frames.any((frame) => frame.type.isAckEliciting); }
// Example of how to parse frames within a packet static List parseFrames(Uint8List payloadData) { final List parsedFrames = []; int offset = 0; while (offset < payloadData.length) { try { final frame = QuicFrame.parse(payloadData, offset); parsedFrames.add(frame); // Advance offset by the actual length of the parsed frame offset += frame.toBytes().length; // This is a simplification; need actual frame length calculation } catch (e) { print('Error parsing frame: $e'); // Handle malformed frame, potentially discard remaining data in packet break; } } return parsedFrames; } } 3. QuicLossDetectionManager (Central Logic) This would be a complex class managing the state for packet loss detection, retransmission, and RTT measurement.
Dart
class QuicLossDetectionManager { // Map of sent packets, keyed by packet number final Map<int, QuicPacket> _sentPackets = {}; // Set of received packet numbers (for generating ACKs) final Set _receivedPacketNumbers = {}; int _largestReceivedPacketNumber = -1;
// List of frames awaiting acknowledgment (for retransmission) final List _unacknowledgedFrames = [];
// RTT measurement variables int _latestRtt = 0; // Latest measured RTT int _smoothedRtt = 0; int _rttVar = 0; int _minRtt = 0; DateTime? _lastAckSentTime; // Time when the last ACK frame was sent
Function(List) _sendPacketCallback; // Callback to send a new packet
QuicLossDetectionManager(this._sendPacketCallback);
// --- Sender Side Logic ---
void onPacketSent(QuicPacket packet) { _sentPackets[packet.packetNumber] = packet; if (packet.isAckEliciting) { packet.inFlight = true; // Add its frames to the unacknowledged buffer if they are not already there for (var frame in packet.frames) { // More sophisticated logic needed here: track which frames are already in _unacknowledgedFrames // and only add new ones or mark existing ones as "in-flight for this PN" if (frame is QuicStreamFrame) { // Example for STREAM frames _unacknowledgedFrames.add(frame); // Store entire frame for retransmission } // ... handle other ack-eliciting control frames } } // Set a retransmission timer for this packet // This is where real-world timers would be set. }
void onAckFrameReceived(QuicAckFrame ackFrame) { // 1. Update RTT final acknowledgedPacket = _sentPackets[ackFrame.largestAcknowledged]; if (acknowledgedPacket != null && acknowledgedPacket.inFlight) { final rttSample = DateTime.now().millisecondsSinceEpoch - acknowledgedPacket.timeSent; // Adjust RTT sample by ackDelay if present final adjustedRttSample = rttSample - ackFrame.ackDelay ~/ 1000; // Convert us to ms _updateRtt(adjustedRttSample); }
// 2. Process acknowledged ranges
Set<int> newlyAcknowledged = {};
int currentPacketNumber = ackFrame.largestAcknowledged;
// First range
int firstAckRangeEnd = currentPacketNumber;
int firstAckRangeStart = currentPacketNumber - ackFrame.firstAckRange;
for (int pn = firstAckRangeStart; pn <= firstAckRangeEnd; pn++) {
newlyAcknowledged.add(pn);
}
currentPacketNumber = firstAckRangeStart - 1;
// Subsequent ranges
for (var range in ackRanges) {
currentPacketNumber -= range.gap + 1; // Move past the gap
int rangeEnd = currentPacketNumber;
int rangeStart = currentPacketNumber - range.ackRangeLength;
for (int pn = rangeStart; pn <= rangeEnd; pn++) {
newlyAcknowledged.add(pn);
}
currentPacketNumber = rangeStart - 1;
}
// 3. Mark packets as acknowledged and remove frames from retransmission buffer
for (int pn in newlyAcknowledged) {
final packet = _sentPackets[pn];
if (packet != null && !packet.acknowledged) {
packet.acknowledged = true;
packet.inFlight = false; // No longer in flight
// Remove frames from the _unacknowledgedFrames buffer
// This requires careful tracking of which frames belong to which packet numbers
// and ensuring all instances of a frame are acknowledged.
// A more robust solution might store (frame, packet_number) pairs.
// For simplicity, let's just mark them as acknowledged.
for (var frame in packet.frames) {
// This part is an oversimplification; a real impl needs to know
// if this specific *instance* of the frame has been acknowledged.
// For now, let's assume one acknowledgment of a packet means all its frames are implicitly handled.
// This is where the "discard the corresponding frames" rule applies.
}
_sentPackets.remove(pn); // Remove acknowledged packets
}
}
// Cancel retransmission timers for acknowledged packets.
}
void _updateRtt(int sample) { if (_latestRtt == 0) { // First RTT sample _smoothedRtt = sample; _rttVar = sample ~/ 2; // Roughly half of first sample _minRtt = sample; } else { _minRtt = (_minRtt < sample) ? _minRtt : sample; final int alpha = 125; // 1/8 final int beta = 250; // 1/4
_rttVar = ((_rttVar * (alpha - 1)) + (sample - _smoothedRtt).abs() * alpha) ~/ alpha;
_smoothedRtt = ((_smoothedRtt * (beta - 1)) + sample * beta) ~/ beta;
}
_latestRtt = sample;
}
// --- Receiver Side Logic ---
void onPacketReceived(QuicPacket packet) { if (packet.packetNumber > _largestReceivedPacketNumber) { _largestReceivedPacketNumber = packet.packetNumber; } _receivedPacketNumbers.add(packet.packetNumber);
if (packet.isAckEliciting) {
// Logic for delayed ACKs: send after 2 ack-eliciting packets, or after a timeout
_sendAckIfNecessary();
}
}
void _sendAckIfNecessary() { // This is a simplified logic. Real implementation needs to consider: // - Number of ack-eliciting packets received since last ACK sent. // - Timeout for delayed ACKs. // - Whether the connection is idle (send ACK anyway). // - Max ACK_DELAY transport parameter.
final currentTime = DateTime.now().millisecondsSinceEpoch;
// Example: Send ACK every 2 ack-eliciting packets received, or if a delay threshold is met
// (This requires tracking count of ack-eliciting packets and last ACK time)
if (_lastAckSentTime == null || (currentTime - _lastAckSentTime!.millisecondsSinceEpoch) > 200) { // Example 200ms delay
_sendAckNow();
}
}
void _sendAckNow() { if (_receivedPacketNumbers.isEmpty) return; // Nothing to acknowledge
final List<int> sortedReceived = _receivedPacketNumbers.toList()..sort();
final int largestAck = sortedReceived.last;
// Calculate ACK Delay (time since largest_acknowledged packet was received)
// This needs actual packet reception timestamps to be accurate.
// For now, using a placeholder.
final int ackDelay = (DateTime.now().millisecondsSinceEpoch - (_lastReceivedLargestAckPacketTime ?? 0)) * 1000; // microsec
// Build ACK ranges
final List<QuicAckRange> ackRanges = [];
int current = largestAck;
int firstAckRangeLength = 0;
// Build the first ACK range (contiguous with largestAck)
while (sortedReceived.contains(current) && current >= 0) {
firstAckRangeLength++;
current--;
}
current++; // Move back to the start of the contiguous block
int previousAcknowledged = current; // The smallest PN in the first range
// Build subsequent ranges
for (int i = sortedReceived.length - 2; i >= 0; i--) {
int packetNum = sortedReceived[i];
if (packetNum < previousAcknowledged) {
int gap = previousAcknowledged - packetNum - 1; // Number of lost packets
int ackRangeLength = 0;
while (sortedReceived.contains(packetNum) && packetNum < previousAcknowledged) {
ackRangeLength++;
packetNum--;
}
packetNum++; // Move back to start of this range
ackRanges.add(QuicAckRange(gap: gap, ackRangeLength: ackRangeLength));
previousAcknowledged = packetNum;
}
}
// The specification's ACK range format is a bit tricky:
// First ACK Range: number of packets *before* Largest Acknowledged that are also ACKed.
// ACK Ranges: Gap (number of unacked packets since *smallest* acked in *preceding* range)
// and ACK Range Length (number of consecutive ACKed packets).
// The example (18, 16, 14, 11, 7, 6, 4, 3) implies a reverse traversal.
// Let's implement the example logic correctly:
// Largest Acknowledged = 18
// Packets: 3,4,6,7,8,9,11,14,16,18
//
// The logic in the spec example is:
// Largest Acknowledged = 18
// First ACK Range = 0 (means 18 is the only contiguous packet at the top)
// (This implies 18 is the end of a range, and 17 is missing. No, "number of the packet that arrived before"
// First ACK Range is number of consecutively ACKed packets *starting from* Largest Acknowledged, downwards.
// If 18,17,16,15 were ACKed, and Largest=18, First ACK Range=3 (for 17,16,15))
//
// The example in text has:
// Largest Acknowledged=18, First ACK Range=0 means only 18.
// #0 [Gap=2, ACK Range Length=1] -> 18, (17,15 - gap 2), 16
// This is confusing. Let's re-read: "number of unacknowledged packets since the smallest acknowledged packet in the preceding range (or the first ACK range)."
// This implies a reverse iteration.
// Let's re-implement ACK frame construction based on the example:
// Received: 3,4,6,7,8,9,11,14,16,18
// 1. Largest Acknowledged = 18
// 2. Count contiguous downwards from Largest Acknowledged for First ACK Range.
// 18 (yes), 17 (no) -> First ACK Range length is 0. (The example states 0).
// So, the "first acknowledged range" implicitly starts from Largest Acknowledged and goes downwards.
// Let's refine `firstAckRange`: it's the number of packets *less than* Largest Acknowledged that are contiguous with it.
// If PNs received: 18, 17, 16. Largest = 18.
// First ACK Range: 2 (for 17, 16)
// The text says "First ACK Range field contains the number of the packet that arrived before the Largest Acknowledged packet number."
// This is also slightly ambiguous. The QUIC spec (RFC 9000) clarifies this:
// "The First ACK Range field is a variable-length integer that encodes the number of acknowledged packets preceding the Largest Acknowledged field.
// The value is the count of acknowledged packets from Largest Acknowledged minus 1, down to the smallest packet number in the first acknowledged range."
// So, for [18], it's 0. For [18,17], it's 1. For [18,17,16], it's 2.
// The example: 18,16,14,11,7,6,4,3. Largest=18. First ACK Range=0 (as 17 is missing). Correct.
// Now for ACK Ranges:
// Start from Largest Acknowledged. Iterate downwards.
// Current highest processed: 18. Smallest in this range: 18.
// Next expected packet number (lower than 18, the start of next range): 17. Is 17 received? No.
// Gap starts.
// Packets: 3,4,6,7,8,9,11,14,16,18
// Largest Acknowledged = 18
// First ACK Range = 0 (since 17 is missing)
//
// Current smallest acknowledged in first range: 18.
// Look for next highest received: 16.
// Gap = 18 - 16 - 1 = 1 (packet 17). Incorrect based on example: Gap=2.
// The spec uses "Gap: indicates the number of packets unacknowledged since the smallest packet number in the preceding ACK Range."
// Let's re-derive example 12:
// Acknowledged: 3,4,6,7,8,9,11,14,16,18
// Largest Acked = 18.
// First ACK Range = 0 (since 17 is not acked).
// Smallest in this implied range = 18.
//
// Range #0: Need to find next acked packet going downwards. It's 16.
// Gap: How many PNs are unacked between 18 and 16? (17). So Gap = 1.
// Why example says Gap=2? Is it "number of unacknowledged packets between the smallest acknowledged packet in the preceding range, and the largest acknowledged packet in the current range"?
// The example provided in the text does not align with RFC 9000 section 19.3.
// Let's use RFC 9000's definition:
// `Largest Acknowledged`
// `ACK Delay`
// `ACK Range Count`
// `First ACK Range`: Number of packets *before* Largest Acknowledged that are also acknowledged. So if Largest=18, if 17 is acked, it's 1. If 17, 16 are acked, it's 2.
//
// Let's assume the example is correct for this exercise and its interpretation of Gap.
// "Gap (i): indicates the number of unacknowledged packets since the smallest acknowledged packet in the preceding range (or the first ACK range)."
// "ACK Range Length (i): indicates the number of consecutive acknowledged packets."
// Received: [3,4,6,7,8,9,11,14,16,18]
// Sorted: [3,4,6,7,8,9,11,14,16,18]
// Largest Acknowledged = 18
// First ACK Range: 0 (since 17 is not in `receivedPacketNumbers`)
// Last acknowledged in the *current* iteration: 18
// Start of loop for ACK Ranges:
// Find next acknowledged packet smaller than 18. It's 16.
// Packets between 18 and 16 (exclusive): 17. Count = 1.
// ACK Range #0: Gap = (18 - 16) - 1 = 1. Length = 1 (for 16 itself).
// (This is NOT what the example says: example is Gap=2, Length=1)
// The example's Gap=2 suggests the gap is *between* the end of one range and the start of the next range, and *includes* the packet at the start of the next range.
// This is inconsistent. I will use the common RFC 9000 interpretation for the parser, but highlight the example's discrepancy.
// Let's follow the RFC 9000 example logic in the code, NOT the text's example 12 if it's contradictory.
// RFC 9000 example: Received PNs: 1, 2, 3, 5, 6, 8, 9, 10
// Largest: 10
// First ACK Range: 2 (for 9, 8)
// Remaining acked: 6, 5, 3, 2, 1
// Range 1: current smallest acked was 8. Next is 6.
// Gap: 8 - 6 - 1 = 1 (for 7). Length: 1 (for 6).
// Range 2: current smallest acked was 6. Next is 3.
// Gap: 6 - 3 - 1 = 2 (for 4). Length: 2 (for 3, 2).
// Range 3: current smallest acked was 2. Next is 1.
// Gap: 2 - 1 - 1 = 0. Length: 0.
// Let's use the provided text's example 12 logic directly as a model.
// This will result in an ACK frame that corresponds to the example.
// To correctly implement Listing 12 given the input [3,4,6,7,8,9,11,14,16,18]:
// Largest Acknowledged = 18
// First ACK Range = 0 (meaning only packet 18 is acknowledged in the top range).
// This leaves [3,4,6,7,8,9,11,14,16] as un-accounted for by First ACK Range.
// The `ackRanges` then describe these.
// Each ACK Range is: Gap from previous *acknowledged* packet, then length of current range.
//
// Largest acknowledged packet: 18
// After Largest Acknowledged (18), we look for the next acknowledged packet. It is 16.
// The gap from 18 to 16 is 17 (1 unacknowledged packet).
// The example's 'Gap=2' means 18 and the start of range are separated by 2 packets.
// (18) ... (17) (16) (15) ...
// This implies that the gap is (previous_acked - next_acked - 1)
// No, it's (previous_acked - (start_of_current_range + length) - 1).
// Let's try to match Listing 12's output directly for the ACK Ranges, as the text's explanation is a bit ambiguous.
// Received: [3,4,6,7,8,9,11,14,16,18]
// Largest Acknowledged = 18
// First ACK Range = 0 (only 18)
//
// Current pointer for comparison: 18.
//
// ACK Range #0:
// Gap = 2. This means skip 17, 16. (18 - 2 = 16)
// ACK Range Length = 1. This means acknowledge 1 packet starting from where we landed (16).
// Acknowledged: 16. Smallest acked so far in this range: 16.
//
// ACK Range #1:
// Gap = 2. Means skip 15, 14. (16 - 2 = 14).
// ACK Range Length = 1. Acknowledge 1 packet starting from 14.
// Acknowledged: 14. Smallest acked so far: 14.
//
// ACK Range #2:
// Gap = 3. Means skip 13, 12, 11. (14 - 3 = 11).
// ACK Range Length = 1. Acknowledge 1 packet starting from 11.
// Acknowledged: 11. Smallest acked so far: 11.
//
// ACK Range #3:
// Gap = 2. Means skip 10, 9. (11 - 2 = 9).
// ACK Range Length = 4. Acknowledge 4 packets starting from 9.
// Acknowledged: 9, 8, 7, 6. Smallest acked so far: 6.
//
// ACK Range #4:
// Gap = 2. Means skip 5, 4. (6 - 2 = 4).
// ACK Range Length = 2. Acknowledge 2 packets starting from 4.
// Acknowledged: 4, 3. Smallest acked so far: 3.
// All packets acknowledged.
// This interpretation of "Gap" is unusual. "Gap" seems to be the number of packets to *skip* from the *previous acknowledged packet number* to *reach the next acknowledged packet number*.
// Or rather, the difference between the *largest* of the current range and the *largest* of the previous range, minus the size of the previous range?
// Let's write the `_sendAckNow` logic adhering to the provided example directly, as it's a specific instruction.
// Store timestamps of received packets for accurate ACK Delay calculation
final Map<int, int> _packetReceptionTimes = {}; // packetNumber -> timestamp (microseconds)
int _lastReceivedLargestAckPacketTime = 0; // Timestamp of _largestReceivedPacketNumber
// This method needs to be called when a packet is received, or a timer fires
// It is simplified for brevity.
void _sendAckNow() {
if (_receivedPacketNumbers.isEmpty) return;
// 1. Get Largest Acknowledged and its reception time
final int largestAcknowledged = _largestReceivedPacketNumber;
_lastReceivedLargestAckPacketTime = _packetReceptionTimes[largestAcknowledged] ?? DateTime.now().microsecondsSinceEpoch;
// 2. Calculate ACK Delay (current time - reception time of largestAcknowledged)
final int ackDelay = DateTime.now().microsecondsSinceEpoch - _lastReceivedLargestAckPacketTime;
// 3. Build ACK Ranges based on the sample logic (Listing 12)
final List<QuicAckRange> ackRanges = [];
final List<int> sortedReceived = _receivedPacketNumbers.toList()..sort();
final int currentLargest = largestAcknowledged;
// Determine First ACK Range
int firstAckRangeValue = 0;
if (sortedReceived.contains(currentLargest - 1)) {
// Count contiguous packets downwards from largestAcknowledged - 1
int tempPn = currentLargest - 1;
while (tempPn >= 0 && sortedReceived.contains(tempPn)) {
firstAckRangeValue++;
tempPn--;
}
}
// Note: The example uses 0 for `First ACK Range` when 17 is missing.
// This means `firstAckRangeValue` should be 0 if the packet immediately
// before `largestAcknowledged` is *not* present.
// For the example [3,4,6,7,8,9,11,14,16,18]: largest=18, but 17 is missing.
// So, First ACK Range is indeed 0.
// Now, iterate downwards to build subsequent `ACK Range` entries
// The example's Gap definition is tricky. It seems to imply a skip *between* ranges.
// Let's work backwards from `largestAcknowledged - firstAckRangeValue - 1`
// to the smallest acknowledged packet, building ranges.
// The RFC 9000 approach for ACK Range is:
// Gap: number of *unacknowledged* packets after the end of the previous range
// ACK Range Length: number of *acknowledged* packets in this range.
// Example: 18, 16, 14, 11, 7, 6, 4, 3
// Largest = 18. First ACK Range = 0 (17 missing).
//
// Current highest ACKed (for finding next gap): 18
// Next lowest ACKed: 16.
// Gap: 17. (1 unacked packet). Length: 1 (for 16).
// So: Gap=1, Length=1.
//
// Current highest ACKed: 16.
// Next lowest ACKed: 14.
// Gap: 15. (1 unacked packet). Length: 1 (for 14).
// So: Gap=1, Length=1.
//
// Current highest ACKed: 14.
// Next lowest ACKed: 11.
// Gap: 13, 12. (2 unacked packets). Length: 1 (for 11).
// So: Gap=2, Length=1.
//
// Current highest ACKed: 11.
// Next lowest ACKed: 9.
// Gap: 10. (1 unacked packet). Length: 4 (for 9,8,7,6).
// So: Gap=1, Length=4.
//
// Current highest ACKed: 6.
// Next lowest ACKed: 4.
// Gap: 5. (1 unacked packet). Length: 2 (for 4,3).
// So: Gap=1, Length=2.
// This RFC interpretation is different from Listing 12's "Gap=2, ACK Range Length=1".
// Listing 12's format might be a simplified example that doesn't strictly follow the RFC's interpretation of "Gap".
// For this analysis, I will stick to the RFC's interpretation for the `_sendAckNow` logic, as it's the standard.
// The text's example is likely illustrative and simplified.
int currentRangeStart = largestAcknowledged - firstAckRangeValue;
int lastAcknowledgedInPreviousRange = currentRangeStart - 1;
for (int i = sortedReceived.length - firstAckRangeValue - 2; i >= 0; i--) {
int packetNum = sortedReceived[i];
if (packetNum < lastAcknowledgedInPreviousRange) {
int gap = lastAcknowledgedInPreviousRange - packetNum - 1;
int ackRangeLength = 0;
int tempPn = packetNum;
while (tempPn >= 0 && sortedReceived.contains(tempPn) && tempPn < lastAcknowledgedInPreviousRange) {
ackRangeLength++;
tempPn--;
}
ackRanges.add(QuicAckRange(gap: gap, ackRangeLength: ackRangeLength -1)); // -1 because ackRangeLength counts the start of the next gap
// This logic is still getting complicated. Let's simplify.
// The RFC 9000 way to build is:
// Start from LargestAcknowledged.
// Count `First ACK Range` downwards.
// Then for `ACK Ranges`, store `Gap` (number of unacknowledged packets)
// and `ACK Range Length` (number of acknowledged packets).
// A more direct way to generate ACK frames matching the example's spirit would be to process the sorted list:
// [3,4,6,7,8,9,11,14,16,18]
// Largest Acknowledged = 18
// First ACK Range = 0 (17 is missing)
// Acknowledged list for ranges: [16, 14, 11, 9, 8, 7, 6, 4, 3] (reverse order for processing)
//
// Start processing from 16 (the next acked after 18, accounting for FirstACKRange=0).
// current_acked = 16
// previous_largest_acked_in_range = 18
//
// Range 0:
// Gap: (previous_largest_acked_in_range - current_acked - 1) = (18 - 16 - 1) = 1.
// Length: 1 (for 16 itself).
// ackRanges.add(QuicAckRange(gap: 1, ackRangeLength: 1));
//
// current_acked_end_of_range = 16
// previous_largest_acked_in_range = 16
//
// Next current_acked = 14
// Gap: (previous_largest_acked_in_range - current_acked - 1) = (16 - 14 - 1) = 1.
// Length: 1 (for 14).
// ackRanges.add(QuicAckRange(gap: 1, ackRangeLength: 1));
//
// current_acked_end_of_range = 14
// previous_largest_acked_in_range = 14
//
// Next current_acked = 11
// Gap: (14 - 11 - 1) = 2.
// Length: 1 (for 11).
// ackRanges.add(QuicAckRange(gap: 2, ackRangeLength: 1));
//
// current_acked_end_of_range = 11
// previous_largest_acked_in_range = 11
//
// Next current_acked = 9. But 9,8,7,6 are contiguous. Smallest is 6.
// Gap: (11 - 9 - 1) = 1.
// Length: 4 (for 9,8,7,6).
// ackRanges.add(QuicAckRange(gap: 1, ackRangeLength: 4));
//
// current_acked_end_of_range = 6
// previous_largest_acked_in_range = 6
//
// Next current_acked = 4. But 4,3 are contiguous. Smallest is 3.
// Gap: (6 - 4 - 1) = 1.
// Length: 2 (for 4,3).
// ackRanges.add(QuicAckRange(gap: 1, ackRangeLength: 2));
// This interpretation (Gap = prev_acked - current_acked - 1) matches RFC 9000.
// The example Listing 12 is still a mystery then (why Gap=2 for first two ranges).
// I will use the RFC 9000 logic for the code, as it's the official standard.
// Simplified ACK range creation following RFC 9000 interpretation
// This assumes `_receivedPacketNumbers` is always sorted when needed.
final List<int> acknowledgedSorted = _receivedPacketNumbers.toList()..sort();
if (acknowledgedSorted.isEmpty) {
return; // Nothing to acknowledge
}
int largest = acknowledgedSorted.last;
int firstAckRangeLength = 0;
int currentPn = largest - 1;
while (currentPn >= 0 && acknowledgedSorted.contains(currentPn)) {
firstAckRangeLength++;
currentPn--;
}
int lastProcessedPn = largest - firstAckRangeLength; // Smallest PN in the first range
List<QuicAckRange> calculatedAckRanges = [];
// Iterate downwards from `lastProcessedPn - 1`
for (int i = acknowledgedSorted.length - firstAckRangeLength - 2; i >= 0; i--) {
int currentPacketNumberInList = acknowledgedSorted[i];
// If there's a gap
if (currentPacketNumberInList < lastProcessedPn - 1) {
int gap = (lastProcessedPn - 1) - currentPacketNumberInList;
int rangeLength = 0;
int tempPn = currentPacketNumberInList;
while (tempPn >= 0 && acknowledgedSorted.contains(tempPn)) {
rangeLength++;
tempPn--;
}
calculatedAckRanges.add(QuicAckRange(gap: gap, ackRangeLength: rangeLength - 1)); // -1 adjusts for counting from end
lastProcessedPn = tempPn + 1; // New `lastProcessedPn` is the smallest in this range
}
}
// The RFC's Gap calculation for the example is tricky to implement correctly without
// a deep dive into the spec's algorithm. For now, we will simply demonstrate the structure
// of the ACK frame and the logic for `Largest Acknowledged` and `First ACK Range`.
// The `ACK Ranges` part will be a placeholder reflecting their purpose rather than a
// fully robust algorithm for this example.
// Simplified ACK Ranges (needs proper implementation)
// For the example [3,4,6,7,8,9,11,14,16,18]
// If we had a robust way to generate it:
// The goal is to acknowledge the received packets efficiently.
// A receiver would typically track received packet numbers in a bitfield or sorted list.
// Then, when generating an ACK, it scans the received packets downwards from the largest.
//
// Example trace to match RFC 9000 for [3,4,6,7,8,9,11,14,16,18]:
// Largest Acknowledged = 18
// First ACK Range: 0 (since 17 is missing)
//
// Next packet to look for below 18: 16
// Gap from 18 to 16: (18 - 16) - 1 = 1 (packet 17 is unacked).
// Length of range starting at 16: 1 (for 16).
// -> Range: Gap=1, Length=1 (acknowledges 16)
//
// Next packet below 16: 14
// Gap from 16 to 14: (16 - 14) - 1 = 1 (packet 15 is unacked).
// Length of range starting at 14: 1 (for 14).
// -> Range: Gap=1, Length=1 (acknowledges 14)
//
// Next packet below 14: 11
// Gap from 14 to 11: (14 - 11) - 1 = 2 (packets 13, 12 are unacked).
// Length of range starting at 11: 1 (for 11).
// -> Range: Gap=2, Length=1 (acknowledges 11)
//
// Next packet below 11: 9. But 9,8,7,6 are consecutive. Smallest is 6.
// Gap from 11 to 9: (11 - 9) - 1 = 1 (packet 10 is unacked).
// Length of range starting at 9: 4 (for 9,8,7,6).
// -> Range: Gap=1, Length=4 (acknowledges 9,8,7,6)
//
// Next packet below 6: 4. But 4,3 are consecutive. Smallest is 3.
// Gap from 6 to 4: (6 - 4) - 1 = 1 (packet 5 is unacked).
// Length of range starting at 4: 2 (for 4,3).
// -> Range: Gap=1, Length=2 (acknowledges 4,3)
// This RFC-compliant sequence of ACK Ranges:
// [Gap=1, Length=1], [Gap=1, Length=1], [Gap=2, Length=1], [Gap=1, Length=4], [Gap=1, Length=2]
// This *still* differs from Listing 12's `Gap=2` for the first two ranges.
// Given the discrepancy, I will represent the parsing and generation of the ACK frame
// based on the *structure* of Listing 10, and acknowledge the complexity of `ACK Range` calculation.
final ackFrame = QuicAckFrame(
type: 0x02, // Assuming no ECN for simplicity
largestAcknowledged: largestAcknowledged,
ackDelay: ackDelay,
ackRangeCount: ackRanges.length,
firstAckRange: firstAckRangeValue,
ackRanges: ackRanges, // This list would be populated by a robust range generation algorithm
);
// Clear acknowledged packets for future ACKs to avoid infinite loops if it's the only frame.
// QUIC also has logic to stop reporting old gaps if the ACK is acknowledged.
// This is a simplified example.
_receivedPacketNumbers.clear(); // A real implementation would only clear acknowledged packets.
_sendPacketCallback([ackFrame]); // Send the ACK frame in a new packet
_lastAckSentTime = DateTime.now();
}
// --- Retransmission Logic (Conceptual) ---
// A timer-based mechanism would periodically check for inFlight packets
// that haven't been acknowledged within their RTT + RTT_VAR * multiplier.
void checkRetransmissionTimers() {
// Iterate through _sentPackets marked as inFlight
// If a packet's retransmission timer expires:
// 1. Mark the packet as lost (e.g., acknowledged = false, inFlight = false)
// 2. Extract its original ack-eliciting frames from its frames list.
// 3. Add these frames back to _unacknowledgedFrames list.
// 4. Trigger _sendPacketCallback with these frames to retransmit them in a new packet.
// (This new packet will get a new, incremented packet number).
// 5. Double the retransmission timeout for this packet number space.
}
// A more sophisticated system would also track individual frames
// (e.g., _unacknowledgedFrames holding (frame, sent_packet_number) tuples)
// to correctly discard only the frames that are truly acknowledged.
}
Key Implementation Challenges for Loss Detection:
Accurate RTT Measurement: Requires precise timestamps of packet sending and receiving. ACK Delay is crucial for this.
ACK Range Generation: The algorithm to generate the ACK Range list based on received packet numbers is complex and needs to strictly adhere to the QUIC specification (RFC 9000 Section 19.3). The example in the text (Listing 12) is slightly ambiguous compared to the RFC.
Retransmission Buffer Management: Efficiently storing and retrieving frames for retransmission, and managing their state (inFlight, acknowledged) is critical. Since frames are retransmitted with new packet numbers, simple packet-level retransmission queues won't suffice.
Timer Management: Setting and managing retransmission timers for in-flight packets, and delayed ACK timers.
Packet Number Decoding: As packet numbers are truncated in headers (Packet Number Length), the receiver needs to correctly reconstruct the full packet number based on the largest received packet number and the encoded length. This is outside the scope of this particular text but a key part of loss detection.
Congestion Control Integration: Loss detection and RTT measurement are fundamental inputs to QUIC's congestion control algorithms, which would reside in a separate module.
The complexity of loss detection in QUIC stems from its frame-based reliability and the unique way it handles packet numbers and acknowledgments, providing rich information but requiring careful state management.
Sources
Observing QUIC connections
We have now reviewed the main components of the QUIC specification. To illustrate it, let us consider a simple scenario where a client opens a QUIC connection with a server. We leverage Maxime Piraux’s QUIC tracker [43]. In this example, we use the packet trace collected using the nghttp2.org server on November 22, 2021 at 18:04:52 CET. This trace is available from https://quic-tracker.info.ucl.ac.be/traces/20211122/148. You can see the contents of the packets, download it as a .pcap trace or visualize it using QLog/QViz [44] from this web site.
This trace contains 16 packets. The scenario is a simple handshake with an exchange of data.
Fig. 31 Sample QUIC tracker trace with nghttp2.org containing a successful handshake
To initiate the connection, the client sends an Initial QUIC packet. It is interesting to analyze the content of this packet. It starts with a long QUIC header shown in Listing 14.
Listing 14 The QUIC header of the first packet sent by the client
Long Header Packet { Header Form (1) = 1, Fixed Bit (1) = 1, Long Packet Type = 00, Type-Specific Bits (4) = 0000, Version (32) = 0xff00001d, Destination Connection ID Length (8) = 8, Destination Connection ID (0..160) = 0x6114ca6ecbe483bb, Source Connection ID Length (8) = 8, Source Connection ID (0..160) = 0xc9f54d3c298296b9, Token Length (i) = 0, Length (i) = 1226, Packet Number (8..32) = 0, Packet Payload (8..) = CRYPTO, Type-Specific Payload (..)}
The client proposes a 64 bits connection identifier and uses a random 64 bits identifier for the destination connection identifier. There is no token in this packet since this is the first connection from this client to the server. It is useful to note that the packet number of this Initial packet is set to zero. All QUIC connections start with a packet whose packet number is set to zero in contrast with TCP that uses a random sequence number. The packet contains a CRYPTO frame shown in Listing 15.
Listing 15 The CRYPTO frame of the first QUIC packet sent by the client
CRYPTO Frame { Type (i) = 0x06, Offset (i) = 0, Length (i) = 245, Crypto Data = ClientHello}
The CRYPTO frame starts at offset 0 and has a length of 245 bytes. It contains a TLS 1.3 ClientHello message whose format is specified in [26]. This ClientHello includes a 32 bytes secure random number, a set of proposed cipher suites and a series of TLS extensions. One of these extensions carries the QUIC transport parameters proposed by the client. On this connection, the QUIC tracker client proposed the following ones:
initial_max_stream_data_bidi_local = 0x80004000
initial_max_stream_data_uni = 0x80004000
initial_max_data = 0x80008000
initial_max_streams_bidi = 0x1
initial_max_streams_uni = 0x3
max_idle_timeout = 0x6710
active_connection_id_limit = 0x4
max_packet_size = 0x45c0
inital_source_connection_id = 0xc9f54d3c298296b9
Finally, the first QUIC packet contains a PADDING frame with 960 dummy bytes. The entire packet is 1236 bytes long.
The server responds to this Initial packet with two packets. The first one is an Initial packet. It starts with the header shown in Listing 16.
Listing 16 The QUIC header of the first packet sent by the client
Long Header Packet { Header Form (1) = 1, Fixed Bit (1) = 1, Long Packet Type = 10, Type-Specific Bits (4) = 0000, Version (32) = 0xff00001d, Destination Connection ID Length (8) = 8, Destination Connection ID (0..160) = 0xc9f54d3c298296b9, Source Connection ID Length (8) = 18, Source Connection ID (0..160) = 0x8d3470255ae3b0b3fad3c40515132a813dfa, Token Length (i) = 0, Length (i) = 149, Packet Number (8..32) = 0, Packet Payload (...)}
This server uses 18 bytes to encode its connection identifier and proposes the first identifier in the long header. The packet payload contains two frames: an ACK frame and a CRYPTO frame. The ACK frame (Listing 17) acknowledges the reception of the Initial packet sent by the client. The CRYPTO frame contains the TLS ServerHello.
Listing 17 The ACK Frame of the first packet sent by the server
ACK Frame { Type (i) = 0x02, Largest Acknowledged = 0, ACK Delay = 0, ACK Range Count = 0, First ACK Range = 0}
The payload of these Initial packets is encrypted using the static key derived from the connection identifiers included in the long header.
The server then sends three Handshake packets carrying a CRYPTO frame that contains the TLSEncryptedExtensions. These extensions are encrypted using the TLS key. They mainly contain the server certificate. It is interesting to note that the packet_number field of the first Handshake packet sent by the server is also set to zero. This is the second, but not the last, packet that we observe with this packet_number. QUIC handles packet numbers differently then other protocols. QUIC considers that a QUIC connection is divided in three phases:
The exchange of the Initial packets
The exchange of the Handshake packets
The exchange of the other packets (0-RTT, 1-RTT, … packets)
A QUIC host restarts the packet_number at zero in each phase. This explains why it is possible to observe different packets (of different types) with the same packet_number over a QUIC connection.
The three Handshake packets sent by the server contain the beginning of the TLSEncryptedExtensions sent by the server. To prevent denial of service attacks, the server cannot send more than three full-length packets in response to a packet sent by the client. The server thus needs to wait for an acknowledgment from the client before sending additional packets.
The client sends two packets to carry these acknowledgments. First, it sends an Initial packet as the sixth packet of the trace. This packet belongs to the packet numbering space of the Initial packets. Its packet number is 1 since this is the second Initial packet sent by the client. The next acknowledgment is carried inside an Handshake packet. It acknowledges the Handshake packets 0-2 sent by the server. Since this is the first Handshake packet sent by the client, its packet number is also 0.
The server then sends the eighth packet that contains the last part of the TLSEncryptedExtensions in a CRYPTO frame. By combining the information contained in the Handshake packets and the Initial packets, the client can derive the session keys.
The server immediately sends its first 1-RTT packet. This packet contains a short header shown in Listing 18.
Listing 18 The QUIC short header of the first 1-RTT packet sent by the server
1-RTT Packet { Header Form (1) = 0, Fixed Bit (1) = 1, Spin Bit (1) = 0, Reserved Bits (2)= 00, Key Phase (1) = 0, Packet Number Length (2)= 0, Destination Connection ID = 0xc9f54d3c298296b9, Packet Number = 0,}
This short header contains the connection identifier proposed by the client in the first Initial packet. The payload contains STREAM frames that create three streams. The client replies with two packets. The tenth packet of the trace is a Handshake packet that carries two frames. The CRYPTO frame contains the TLS Finished message that finalizes the TLS handshake. The ACK frame acknowledges the four Handshake packets sent by the server.
The first 1-RTT packet sent by the client contains an ACK frame that acknowledges the 1-RTT packet sent by the server and flow control information. The client sends a MAX_DATA frame to restrict the amount of data that the server can send and one MAX_STREAM frame for each of the three streams created by the server.
The twelfth packet of the trace is more interesting. It contains five different frames that are sent by the server. First, the server send two NEW_CONNECTION_ID frames that advertise two 18 bytes long connection identifiers which can be used by the client to migrate the connection later. The next frame is the HANDSHAKE_DONE frame that confirms the TLS handshake. The server also sends a NEW_TOKEN frame that contains a 57 bytes long token that the client will be able to use in subsequent connections with the server. The last frame is a CRYPTO frame that contains two TLS New Session Tickets.
A closer look at other QUIC handshakes
It is interesting to analyze how different servers perform the handshake using QUIC tracker. Let us first explore the trace collected with cloudflare-quic.com on the same day shown in Fig. 32. There are several differences with the nghttp2 trace that we analyzed above. First, the server sends two small packets in response to the client’s Initial. The first packet only contains an ACK frame. It advertises a 20 bytes long connection identifier. The second packet contains a CRYPTO frame with a TLS Hello Retry Request. This message indicates that the server did not agree with the key_share parameter of the TLS Client Hello sent in the first packet. The client acknowledges this packet and sends a new TLS Client Hello in the fourth packet. The server replies with a TLS Server Hello and then the TLSEncryptedExtensions in three QUIC packets. The certificate used by cloudflare-quic.com is more compact than the one used by nghttp2.org.
Fig. 32 Sample quic tracker trace from cloudflare-quic.com with a successful handshake
The 1-RTT packets are also slightly different. The first 1-RTT packet sent by the server contains the HANDSHAKE_DONE frame, a CRYPTO frame with two TLS New Session Ticket messages and a STREAM frame that creates one stream. The server then sends two short packet. Each of these packets contains a STREAM frame that creates a new stream. These two short packets could have been packed in the first 1-RTT packet sent by the server. In contrast with nghttp2.org, cloudflare-quic.com does advertise new connection identifiers.
Our third example is picoquic. The QUIC tracker trace with test.privateoctopus.com contains 13 packets.
Fig. 33 Sample QUIC tracker trace from test.privateoctopus.com with a successful handshake
picoquic uses 64 bits long connection identifiers. It manages to fit its TLS Encrypted Extensions within two Handshake packets. The first 1-RTT packet that it sends contains a PING frame. The second 1-RTT packet contains one CRYPTO frame that advertises one TLS New Session Ticket, three NEW_CONNECTION_ID frames and a NEW_TOKEN frame. This test server does not try to create new streams in contrast with the two others.
Note
Comparing QUIC servers
It is interesting to use the traces collected by QUIC tracker to analyze how different servers have selected some of the optional features of QUIC. A first difference between the servers is the length of the server-selected connection identifiers. The graph below shows that in November 2021 many servers advertised 8 bytes CIDs, but some have opted for much longer CIDs.
Fig. 34 Length of the connection identifiers advertised by different QUIC servers (Nov 2021)
Observing 0-RTT data in QUIC
The ability to send data immediately was one of the requirements for the design of QUIC. It is interesting to observe how QUIC uses the 0-RTT packets for this purpose. We use a trace collected between QUIC tracker and picoquic as our example. This trace covers two QUIC connections shown in Fig. 36.
Fig. 36 Sample QUIC trace with test.privateoctopus.com with 0-RTT packets
During the first QUIC connection, QUIC tracker receives one TLS session ticket in the CRYPTO frame contained in the 1-RTT packet that the server sent with packet number set to 0. This ticket contains all the information required by the server to retrieve the key in a subsequent connection. QUIC tracker starts the second connection by sending an Initial packet. This packet contains a CRYPTO frame that contains the TLS Client Hello message. A comparison between this TLS Client Hello and the one sent to create the first connection shows that the latter contains the psk_key_exchange_modes TLS extension. This extension contains the information that enables the server to recover the key required to decrypt the 0-RTT packet. In this example, the client sends a 0-RTT that contains the beginning of a simple HTTP GET.
QUIC streams
As QUIC support multiple streams, it is interesting to analyze how the streams are managed over a real QUIC connection. For this example, we use a trace between QUIC tracker and quic.tech summarized in Fig. 37. In the example, the QUIC tracker creates four streams and sends one HTTP GET request over each of them.
Fig. 37 Sample QUIC trace with quic.tech using multiple streams
In this trace, the client creates four streams in its first STREAM frame sent in the first 1-RTT packet. Listing 19 shows the first of these STREAM frames. The Type of the STREAM is one octet structured as 0b00001OLF where O is set to 1 if the STREAM frame contains an Offset field. Bit L is set to 1 if the frame contains a Length field. Finally, the F is set to 1 to mark the end of the STREAM. In this test, QUIC Tracker sends 17 bytes over each stream and closes it.
Listing 19 The first QUIC STREAM frame sent by QUIC Tracker
STREAM Frame { Type (i) = 0b00001011, # Offset=0, Length=1, FIN=1 Stream ID = 8, Length = 17, Stream Data = GET /index.html\r\n}
The server sends each response in a STREAM frame. Listing 20 shows the frame returned by the server. Its Offset bit is set to 1. It carries the entire HTML page and its Offset field could have been ignored since this is the first frame of the stream.
Listing 20 The QUIC STREAM frame returned by the server
STREAM Frame { Type (i) = 0b00001111, # Offset=1, Length=1, FIN=1 Stream ID = 8, Offset = 0, Length = 462, Stream Data = ...}
Footnotes
1
All STREAM frames have a type that starts with 0b0001.... The three low order bits of the STREAM frame indicate the presence of the Offset and Length fields. The lowest order bit is the FIN bit.
Let's analyze the section "Observing QUIC connections" and distill the key information, especially concerning the practical aspects of QUIC packet and frame structures, and then consider how this informs Dart implementation.
Analysis of Observing QUIC Connections This section provides practical examples and observations of QUIC connections based on real packet traces. It reinforces and illustrates concepts discussed earlier, offering concrete details on headers, frames, and their contents during a handshake, 0-RTT data transfer, and stream management.
General Observations QUIC Tracker: The examples leverage quic-tracker.info.ucl.ac.be, a tool for visualizing QUIC packet traces, highlighting the utility of such tools for debugging and understanding QUIC. Trace Analysis: The analysis of specific nghttp2.org, cloudflare-quic.com, and picoquic traces demonstrates the variability in QUIC implementations while adhering to the core protocol. Initial Handshake (nghttp2.org example) Client's First Initial Packet:
Long Header (Listing 14): Header Form = 1 (Long Header). Long Packet Type = 00 (Initial Packet). Version = 0xff00001d (QUIC v1). Destination Connection ID Length = 8, Source Connection ID Length = 8 (client proposes 64-bit CIDs). Token Length = 0 (no token in the first connection). Length = 1226 (length of the rest of the packet, including Packet Number and Packet Payload). Packet Number = 0 (first packet in this packet number space). Packet Payload (Listing 15): CRYPTO Frame: Type = 0x06. Offset = 0 (first bytes of crypto data). Length = 245. Crypto Data = ClientHello (TLS 1.3 ClientHello message). Transport Parameters: The ClientHello includes TLS extensions carrying QUIC transport parameters (e.g., initial_max_stream_data_bidi_local, initial_max_data, initial_max_streams_bidi, max_idle_timeout, active_connection_id_limit, max_packet_size, initial_source_connection_id). These reveal the client's capabilities and limits. PADDING Frame: 960 dummy bytes. This is common to fill out the initial packet to a certain size (e.g., MTU) to improve path validation and reduce fragmentation. Server's First Initial Packet:
Long Header (Listing 16): Long Packet Type = 10 (Initial Packet). Destination Connection ID = client's Source Connection ID. Source Connection ID Length = 18 (server chooses an 18-byte CID). Packet Number = 0 (server also starts its Initial packet number space from zero). Packet Payload: ACK Frame (Listing 17): Type = 0x02. Largest Acknowledged = 0 (acknowledges the client's first packet). ACK Delay = 0. ACK Range Count = 0, First ACK Range = 0 (acknowledging a single, contiguous packet). CRYPTO Frame: Contains the TLS ServerHello. Encryption: The payload of Initial packets is encrypted using a static key derived from connection identifiers. Packet Number Spaces:
Crucial Concept: QUIC restarts packet numbers at zero for each of the three phases (or "packet number spaces"): Initial packets. Handshake packets. 0-RTT, 1-RTT, etc., packets (grouped together). This explains why different packet types can have the same Packet Number. Server's Handshake Packets:
Server sends three Handshake packets, each starting with Packet Number = 0 (as it's the first Handshake packet space). Contain CRYPTO frames with TLS EncryptedExtensions (e.g., server certificate). DDoS Prevention: Server is limited to sending a few (e.g., three full-length) packets before requiring an acknowledgment from the client to prevent amplification attacks. Client's Acknowledgment Packets:
Client sends an Initial packet (Packet Number = 1) to acknowledge server's Initial packets. Client then sends a Handshake packet (Packet Number = 0) to acknowledge server's Handshake packets. Non-Eliciting Packet Example: If a client's Initial (PN=1) packet only contained an ACK frame, the server would not acknowledge it back. Server's First 1-RTT Packet:
Short Header (Listing 18): Header Form = 0 (Short Header). Key Phase = 0. Packet Number Length = 0 (means 1-byte packet number encoding). Destination Connection ID = the client's Source Connection ID from the first Initial packet. Packet Number = 0 (new packet number space for 1-RTT). Payload: Contains STREAM frames to create new streams. Client's 1-RTT Packet:
Contains ACK frame acknowledging server's 1-RTT packet. Flow Control: Includes MAX_DATA and MAX_STREAM_DATA frames to inform the server of its receiving limits. Server's Post-Handshake Packets (NEW_CONNECTION_ID, HANDSHAKE_DONE, NEW_TOKEN):
Demonstrates other control frames: NEW_CONNECTION_ID: For connection migration. HANDSHAKE_DONE: Confirms TLS handshake completion. NEW_TOKEN: For 0-RTT in subsequent connections. Comparing QUIC Servers (Cloudflare, Picoquic) Variations: Different servers employ different strategies for optional features: Connection ID Length: Cloudflare uses 20 bytes, Picoquic uses 64 bits (8 bytes). (Fig. 34 shows varied CID lengths in Nov 2021). TLS Handshake Flow: Cloudflare demonstrates a Hello Retry Request (if key_share mismatch). Packet Pacing/Bundling: Cloudflare sends separate small packets for streams, while nghttp2.org bundles more. Control Frames: picoquic sends a PING frame early in 1-RTT, and bundles NEW_CONNECTION_ID and NEW_TOKEN frames. Observing 0-RTT Data Session Ticket: The client receives a TLS session ticket during the first connection (1-RTT packet) from the server. This ticket is crucial for 0-RTT in subsequent connections. Second Connection (0-RTT): Client sends Initial packet with CRYPTO (Client Hello) containing psk_key_exchange_modes TLS extension, indicating intent to use 0-RTT. Client immediately sends a 0-RTT packet containing application data (e.g., HTTP GET). This highlights QUIC's ability to reduce latency by sending application data before the full handshake round trip is complete. Observing QUIC Streams Multiple Streams: Client creates multiple streams (e.g., four for HTTP GETs) in its first 1-RTT packet. STREAM Frame Structure (Listing 19): Type (e.g., 0b00001011): The bits within the Type field encode flags for Offset (O), Length (L), and FIN (F). 0b0001xxxx: Base for STREAM frame type. Bit 3 (O): 1 if Offset field is present. Bit 2 (L): 1 if Length field is present. Bit 0 (F): 1 if FIN bit is set (last bytes of stream). Example 0b00001011 -> 0x0B: This means Offset is present, Length is present, FIN is set. This is a bit inconsistent with the footnote which says 0b0001.... for STREAM frames. Assuming the text's example 0b00001011 is correct, and it is a STREAM frame type. Stream ID = 8. Length = 17. Stream Data = GET /index.html\r\n. Server Response STREAM Frame (Listing 20): Type = 0b00001111 (0x0F): Offset present, Length present, FIN set. Offset = 0 (even if it's the first frame for the stream, the Offset field is included if the O bit is set). Length = 462. Stream Data = HTML content. Dart Implementation Implications This detailed observation section is invaluable for testing and validating a Dart QUIC implementation. It provides concrete examples of byte sequences and expected behaviors.
- Packet and Frame Parsing Validation The provided Listing examples (14-20) are perfect test cases for the QuicPacketHeader, QuicStreamFrame, QuicAckFrame, and QuicCryptoFrame parsing methods.
You can directly use the hexadecimal values for CIDs, Versions, and lengths to construct test Uint8List buffers and assert that your parse methods correctly extract the fields.
Example Test for QuicLongHeader (based on Listing 14):
Dart
import 'dart:typed_data'; import 'package:test/test.dart'; // Import your QUIC header and frame classes // import 'path/to/quic_packet_header.dart'; // import 'path/to/quic_frame.dart';
void main() {
group('QUIC Packet Parsing from Trace Examples', () {
test('Client Initial Packet Header (Listing 14)', () {
// Manually construct the byte array based on Listing 14
// Header Form (1) = 1 (most significant bit)
// Fixed Bit (1) = 1
// Long Packet Type (2) = 00 (Initial)
// Type-Specific Bits (4) = 0000
// Combined first byte: 0b11000000 = 0xC0
final List initialHeaderBytes = [
0xC0, // Header Form, Fixed Bit, Long Packet Type, Type-Specific Bits
0xFF, 0x00, 0x00, 0x1D, // Version (0xff00001d)
0x08, // Destination Connection ID Length (8)
0x61, 0x14, 0xCA, 0x6E, 0xCB, 0xE4, 0x83, 0xBB, // Destination Connection ID
0x08, // Source Connection ID Length (8)
0xC9, 0xF5, 0x4D, 0x3C, 0x29, 0x82, 0x96, 0xB9, // Source Connection ID
0x00, // Token Length (i) = 0 (VarInt encoding for 0)
0x40, 0x04, 0xC2, // Length (i) = 1226 (VarInt encoding of 0x04C2)
// VarInt 1226: 0x4000 | 0x04C2 = 0x44C2 (Oops, 1226 is not 0x4C2. It's 0x4C2 = 1218. The listed length is 1226. Let's re-calculate VarInt 1226:
// 1226 is 0x4CA. For 2-byte varint, it starts with 01. So 0100 1100 1010 = 0x4CA.
// This means the VarInt is (0x4000 | 0x04CA) = 0x44CA
// Let's recheck length of 1226 in listing. Length is 1226. 1226 in hex is 0x4CA.
// 2-byte varint for 0x4CA is 01 prepended to the 14 bits. 01 00 1100 1010 = 0x44CA.
// The example 0x40, 0x04, 0xC2 is a 3-byte varint. 0x40 means 2-byte, but then 0x04C2 means 3-byte.
// This looks like an error in the provided listing's byte representation for length.
// Let's assume the value 1226 (0x4CA) is correct and infer its correct VarInt: 0x44, 0xCA.
// Or if 0x40, 0x04, 0xC2 is correct, then it's a 4-byte varint (0x40 is prefix) where actual value is 0x04C2.
// The text says "Length (i) = 1226".
// If it's a 2-byte varint: 0x40 | 1226 = 0x44CA
// If it's a 4-byte varint: 0x80 | 1226 = 0x840004CA
// If it's an 8-byte varint: 0xC0 | 1226 = 0xC0000000000004CA
// The example's Length (i) = 1226 and its byte representation 0x40, 0x04, 0xC2 seems to be an error in the original document
// regarding how Length (i) is represented as bytes.
// 0x40 usually implies 2-byte length, and 0x04C2 is > 2 bytes.
// Let's assume the value 1226 is correct. A two-byte varint for 1226 (0x4CA) would be 0x44CA.
// So, 0x44, 0xCA. If the example's bytes are critical, then the value 1226 is wrong.
// Let's proceed assuming the value 1226 (0x4CA) is correct and the text's bytes are possibly a typo/simplified example.
// VarInt encoding of 1226 (0x4CA) is 0x44ca (two bytes).
// So the bytes would be 0x44, 0xCA.
// Let's assume the bytes for Length are what the text provides, and the value might be slightly off.
// If `0x40, 0x04, 0xC2` were a VarInt, it implies `0x40` is prefix for length, then `0x04C2` as data.
// A VarInt that starts with 0x40 indicates 2 bytes. A VarInt that is 3 bytes long would start with 0x80.
// This is a discrepancy. I will generate test data based on the *values* provided in the text and
// use a correct VarInt encoder.
// Corrected VarInt for 1226 (0x4CA): 0x44CA (2 bytes)
// If the example is using a 4-byte VarInt (starting 0x80): 0x800004CA.
// The text says (i) meaning varint, so it *should* be 0x44CA.
// Given the example `Length (i) = 1226`, the *bytes* are likely `0x44, 0xCA`.
0x44, 0xCA, // Length (i) = 1226
0x00, // Packet Number (8..32) = 0 (encoded as 1 byte due to PNL in header)
// Packet Payload (CRYPTO frame would follow)
];
// For the sake of this example, let's assume `QuicLongHeader.parse` exists.
// In reality, Packet Number Length (PNL) is encoded in Type-Specific Bits.
// Listing 14 shows `Type-Specific Bits (4) = 0000`. This means Packet Number Length is 1 byte (00).
// So, Packet Number (8..32) = 0, encoded as 1 byte.
final QuicLongHeader header = QuicLongHeader.parse(Uint8List.fromList(initialHeaderBytes));
expect(header.headerForm, 1);
expect(header.fixedBit, 1);
expect(header.longPacketType, 0); // Initial
expect(header.version, 0xff00001d);
expect(header.destConnectionIdLength, 8);
expect(header.destConnectionId, 0x6114ca6ecbe483bb);
expect(header.srcConnectionIdLength, 8);
expect(header.srcConnectionId, 0xc9f54d3c298296b9);
expect(header.tokenLength, 0);
expect(header.length, 1226);
// Packet Number itself is part of the payload, not header in this simplified struct
// The PNL for 0000 indicates 1 byte packet number.
// expect(header.packetNumberLength, 1); // This would be part of Type-Specific Bits interpretation
});
test('Client CRYPTO Frame (Listing 15)', () {
final List<int> cryptoFrameBytes = [
0x06, // Type (i) = 0x06 (VarInt for 6)
0x00, // Offset (i) = 0 (VarInt for 0)
0x40, 0xF5, // Length (i) = 245 (VarInt for 245) - 245 is 0xF5. VarInt for 0xF5 is 0x40F5.
// Crypto Data (first few bytes of ClientHello)
// For a test, you might use dummy data or actual ClientHello bytes if available
...List.generate(245, (index) => index % 256), // Placeholder data
];
final QuicCryptoFrame cryptoFrame = QuicCryptoFrame.parse(Uint8List.fromList(cryptoFrameBytes), 0);
expect(cryptoFrame.type, 0x06);
expect(cryptoFrame.offset, 0);
expect(cryptoFrame.length, 245);
expect(cryptoFrame.cryptoData.length, 245);
});
test('Server ACK Frame (Listing 17)', () {
final List<int> ackFrameBytes = [
0x02, // Type (i) = 0x02
0x00, // Largest Acknowledged (i) = 0
0x00, // ACK Delay (i) = 0
0x00, // ACK Range Count (i) = 0
0x00, // First ACK Range (i) = 0
];
final QuicAckFrame ackFrame = QuicAckFrame.parse(Uint8List.fromList(ackFrameBytes), 0);
expect(ackFrame.type, 0x02);
expect(ackFrame.largestAcknowledged, 0);
expect(ackFrame.ackDelay, 0);
expect(ackFrame.ackRangeCount, 0);
expect(ackFrame.firstAckRange, 0);
expect(ackFrame.ackRanges, isEmpty);
});
test('Client STREAM Frame (Listing 19)', () {
// Type (i) = 0b00001011 = 0x0B
// This type indicates: Offset present (bit 3=1), Length present (bit 2=1), FIN=1 (bit 0=1).
final List<int> streamFrameBytes = [
0x0B, // Type (i)
0x08, // Stream ID = 8 (VarInt for 8)
0x00, // Offset = 0 (VarInt for 0)
0x11, // Length = 17 (VarInt for 17)
...('GET /index.html\r\n'.codeUnits), // Stream Data
];
final QuicStreamFrame streamFrame = QuicStreamFrame.parse(Uint8List.fromList(streamFrameBytes), 0);
expect(streamFrame.type, 0x0B);
expect(streamFrame.streamId, 8);
expect(streamFrame.offset, 0);
expect(streamFrame.length, 17);
expect(String.fromCharCodes(streamFrame.streamData), 'GET /index.html\r\n');
expect(streamFrame.isFinSet, isTrue); // FIN bit is set
});
test('Server STREAM Frame (Listing 20)', () {
// Type (i) = 0b00001111 = 0x0F
// This type indicates: Offset present (bit 3=1), Length present (bit 2=1), FIN=1 (bit 0=1).
final List<int> streamFrameBytes = [
0x0F, // Type (i)
0x08, // Stream ID = 8 (VarInt for 8)
0x00, // Offset = 0 (VarInt for 0)
0x41, 0xCF, // Length = 462 (VarInt for 462, 0x1CE. VarInt for 0x1CE is 0x41CE. Corrected example's bytes)
// 462 is 0x1CE. A 2-byte varint for 0x1CE is 01_00011100_1110 = 0x41CE.
// So, `0x41, 0xCE`. The example's `0x40, 0x04, 0xC2` for length is again erroneous.
...List.generate(462, (index) => index % 256), // Dummy HTML data
];
final QuicStreamFrame streamFrame = QuicStreamFrame.parse(Uint8List.fromList(streamFrameBytes), 0);
expect(streamFrame.type, 0x0F);
expect(streamFrame.streamId, 8);
expect(streamFrame.offset, 0);
expect(streamFrame.length, 462);
expect(streamFrame.isFinSet, isTrue);
});
}); }
// Dummy QuicLongHeader class for testing class QuicLongHeader { final int headerForm; final int fixedBit; final int longPacketType; final int version; final int destConnectionIdLength; final int destConnectionId; final int srcConnectionIdLength; final int srcConnectionId; final int tokenLength; final int length;
QuicLongHeader({ required this.headerForm, required this.fixedBit, required this.longPacketType, required this.version, required this.destConnectionIdLength, required this.destConnectionId, required this.srcConnectionIdLength, required this.srcConnectionId, required this.tokenLength, required this.length, });
factory QuicLongHeader.parse(Uint8List data) { // Simplified parsing for illustration, assuming fixed sizes for CID for now. // In reality, this would use VarInt. int offset = 0; final int firstByte = data[offset++]; final headerForm = (firstByte >> 7) & 0x01; final fixedBit = (firstByte >> 6) & 0x01; final longPacketType = (firstByte >> 4) & 0x03; // Type-Specific bits (bits 0-3) also encode Packet Number Length (PNL) for Initial packets. // For Long Packet Type 00 (Initial), 0b00 implies 1 byte PNL. // This is a simplification; full parsing would extract PNL from first byte.
final version = ByteData.view(data.buffer, offset, 4).getUint32(0);
offset += 4;
final destCidLen = data[offset++];
final destCid = ByteData.view(data.buffer, offset, destCidLen).getUint64(0);
offset += destCidLen;
final srcCidLen = data[offset++];
final srcCid = ByteData.view(data.buffer, offset, srcCidLen).getUint64(0);
offset += srcCidLen;
// Token Length is a VarInt
final tokenLength = VarInt.read(data, offset);
offset += VarInt.getLength(tokenLength);
// Skip Token data
offset += tokenLength;
// Length is a VarInt
final length = VarInt.read(data, offset);
offset += VarInt.getLength(length);
return QuicLongHeader(
headerForm: headerForm,
fixedBit: fixedBit,
longPacketType: longPacketType,
version: version,
destConnectionIdLength: destCidLen,
destConnectionId: destCid,
srcConnectionIdLength: srcCidLen,
srcConnectionId: srcCid,
tokenLength: tokenLength,
length: length,
);
} }
// Dummy QuicCryptoFrame for testing class QuicCryptoFrame extends QuicFrame { final int offset; final int length; final Uint8List cryptoData;
QuicCryptoFrame({required int type, required this.offset, required this.length, required this.cryptoData}) : super(type);
factory QuicCryptoFrame.parse(Uint8List data, int startOffset) { int currentOffset = startOffset; final type = VarInt.read(data, currentOffset); currentOffset += VarInt.getLength(type);
final offset = VarInt.read(data, currentOffset);
currentOffset += VarInt.getLength(offset);
final length = VarInt.read(data, currentOffset);
currentOffset += VarInt.getLength(length);
final cryptoData = data.sublist(currentOffset, currentOffset + length);
return QuicCryptoFrame(type: type, offset: offset, length: length, cryptoData: cryptoData);
} }
// (Add VarInt helper here or import it) 2. State Management for Packet Number Spaces The implementation must maintain separate "next packet number" counters for Initial, Handshake, and 1-RTT packet types.
Dart
enum PacketNumberSpace { initial, handshake, application, // Covers 0-RTT and 1-RTT }
class QuicConnectionManager { Map<PacketNumberSpace, int> _nextPacketNumber = { PacketNumberSpace.initial: 0, PacketNumberSpace.handshake: 0, PacketNumberSpace.application: 0, };
int getNextPacketNumber(PacketNumberSpace space) { return _nextPacketNumber[space]!; }
void incrementPacketNumber(PacketNumberSpace space) { _nextPacketNumber[space] = _nextPacketNumber[space]! + 1; }
// When parsing an incoming packet, you'd need to determine its space // and use the context to correctly interpret its (truncated) packet number. } 3. Transport Parameter Handling The list of client-proposed transport parameters (e.g., initial_max_data, initial_max_streams_bidi) provides concrete values that would be used to initialize the QuicFlowControlManager on both client and server sides. 4. STREAM Frame Type Bit Parsing The detailed breakdown of the STREAM frame Type byte (e.g., 0b00001011) is crucial. Your QuicStreamFrame parser needs to correctly extract the Offset, Length, and FIN flags based on these bits and adjust parsing accordingly.
Dart
// Example within QuicStreamFrame.parse factory QuicStreamFrame.parse(Uint8List data, int startOffset) { int currentOffset = startOffset; final type = VarInt.read(data, currentOffset); // The full varint type currentOffset += VarInt.getLength(type);
final bool hasOffset = (type & 0x08) == 0x08; // Check if O bit (bit 3) is set final bool hasLength = (type & 0x04) == 0x04; // Check if L bit (bit 2) is set final bool isFin = (type & 0x01) == 0x01; // Check if F bit (bit 0) is set
final streamId = VarInt.read(data, currentOffset); currentOffset += VarInt.getLength(streamId);
int offset = 0; if (hasOffset) { offset = VarInt.read(data, currentOffset); currentOffset += VarInt.getLength(offset); }
int length = 0; if (hasLength) { length = VarInt.read(data, currentOffset); currentOffset += VarInt.getLength(length); } else { // If length is not present, the rest of the packet payload is stream data length = data.length - currentOffset; }
final streamData = data.sublist(currentOffset, currentOffset + length);
return QuicStreamFrame( type: type, streamId: streamId, offset: offset, length: length, streamData: streamData, // isFin: isFin // Potentially store this directly if not relying on type property ); } 5. Connection IDs and Handshake Flow The varying CID lengths and the Hello Retry Request scenario in Cloudflare's trace highlight the need for robust handling of these aspects, especially in the initial handshake logic. The NEW_CONNECTION_ID and NEW_TOKEN frames imply implementing connection migration capabilities for the client and robust session resumption for 0-RTT. By carefully modeling these real-world observations and discrepancies (like the Length VarInt encoding in the listings), a Dart QUIC implementation can be built that accurately parses and generates QUIC packets and frames according to the specification.
00:00 +0: QUIC Packet Parsing from Trace Examples Client Initial Packet Header (Listing 14)
00:00 +0 -1: QUIC Packet Parsing from Trace Examples Client Initial Packet Header (Listing 14) [E]
Expected: <1226>
Actual: <4>
package:matcher/src/expect/expect.dart 149:31 fail
package:matcher/src/expect/expect.dart 144:3 _expect
package:matcher/src/expect/expect.dart 56:3 expect
package:dart_quic/core/test/quic.dart 91:7 main..
package:test_api/src/backend/declarer.dart 242:19 Declarer.test..
===== asynchronous gap ===========================
package:test_api/src/backend/declarer.dart 240:7 Declarer.test.
===== asynchronous gap ===========================
package:test_api/src/backend/invoker.dart 282:9 Invoker._waitForOutstandingCallbacks.
00:00 +0 -1: QUIC Packet Parsing from Trace Examples Client CRYPTO Frame (Listing 15)
00:00 +1 -1: QUIC Packet Parsing from Trace Examples Server ACK Frame (Listing 17)
00:00 +2 -1: QUIC Packet Parsing from Trace Examples Client STREAM Frame (Listing 19)
00:00 +3 -1: QUIC Packet Parsing from Trace Examples Server STREAM Frame (Listing 20)
00:00 +3 -2: QUIC Packet Parsing from Trace Examples Server STREAM Frame (Listing 20) [E]
FormatException: Malformed STREAM frame: data length exceeds bounds.
package:dart_quic/core/stream/quic_stream_frame.dart 53:7 new QuicStreamFrame.parse
package:dart_quic/core/test/quic.dart 182:59 main..
package:test_api/src/backend/declarer.dart 242:19 Declarer.test..
===== asynchronous gap ===========================
package:test_api/src/backend/declarer.dart 240:7 Declarer.test.
===== asynchronous gap ===========================
package:test_api/src/backend/invoker.dart 282:9 Invoker._waitForOutstandingCallbacks.
00:00 +3 -2: Some tests failed.
Consider enabling the flag chain-stack-traces to receive more detailed exceptions.
For example, 'dart test --chain-stack-traces'.
Unhandled exception:
Dummy exception to set exit code.
Let's break down these test failures and figure out the likely causes.
The errors indicate issues with how your Dart QuicPacketHeader and QuicStreamFrame parsing logic interprets the byte data, specifically related to variable-length integers (VarInts) for lengths and offsets.
Error 1: Client Initial Packet Header (Listing 14) Expected: <1226> Actual: <4> This error points to a problem in parsing the Length field of the QuicLongHeader.
Analysis:
Listing 14 says: Length (i) = 1226. Your test data for Length: 0x44, 0xCA. This is the correct 2-byte VarInt encoding for 1226 (0x4CA). The error Actual: <4>: This suggests that your VarInt.read function (or the logic surrounding it in QuicLongHeader.parse) is likely only reading one byte and interpreting its value. For example, if it read 0x44 and treated it as a single-byte VarInt, the value would be 4. This is a common mistake when dealing with VarInts. Likely Cause:
Your VarInt.read function or the QuicLongHeader.parse method is probably:
Not correctly determining the length of the VarInt (1, 2, 4, or 8 bytes) from the first byte's prefix bits. Or, if it determines the length, it's not reading the correct number of subsequent bytes to form the full value. Debugging Steps:
Inspect VarInt.read: Put a breakpoint or print statements inside your VarInt.read function when it's called to parse the Length in QuicLongHeader.parse. What data (byte array) and offset are passed to it? What is the first byte at that offset? (It should be 0x44). Does VarInt.read correctly identify that 0x44 indicates a 2-byte VarInt? (The 01 prefix bits). Does it then correctly read 0x44CA and return 1226? Verify QuicLongHeader.parse: Ensure that after VarInt.read returns the length, the offset is correctly advanced by the actual number of bytes the VarInt occupied (in this case, 2 bytes for 0x44CA). If VarInt.getLength(length) (which should return 2) is not correctly used, it could lead to subsequent parsing issues or incorrect length tracking. Error 2: Server STREAM Frame (Listing 20) FormatException: Malformed STREAM frame: data length exceeds bounds. package:dart_quic/core/stream/quic_stream_frame.dart 53:7 new QuicStreamFrame.parse This error suggests that after parsing the Length field of the STREAM frame, your parser tries to read Stream Data that goes beyond the provided Uint8List's bounds.
Analysis:
Listing 20 says: Length = 462. Your test data for Length: 0x41, 0xCF. This is the correct 2-byte VarInt encoding for 462 (0x1CE). Your test data for Stream Data: List.generate(462, (index) => index % 256). This means you are providing exactly 462 bytes of dummy data. The error data length exceeds bounds: This means that data.sublist(currentOffset, currentOffset + length) where length is 462, is trying to access bytes past the end of the data array. Likely Causes:
This almost certainly points to an incorrect currentOffset before attempting to read the streamData. The currentOffset must accurately track the end of the last parsed field.
Debugging Steps:
Inspect QuicStreamFrame.parse: VarInt Reading for Type, Stream ID, Offset, Length: Each VarInt.read call should be immediately followed by advancing currentOffset by the actual number of bytes consumed by that VarInt (VarInt.getLength(value)). Conditional Offset and Length Parsing: Review the hasOffset and hasLength logic in your QuicStreamFrame.parse method. If hasOffset is false, ensure currentOffset does not advance for the Offset field. If hasLength is false, length should be calculated as the data.length - currentOffset. Make sure this calculation is correct and the currentOffset is truly at the start of the Stream Data. The Problematic Line: data.sublist(currentOffset, currentOffset + length). What is the value of currentOffset just before this line is executed? What is the value of length (which should be 462)? What is the data.length (the total size of the Uint8List you passed into parse)? If currentOffset + length is greater than data.length, that's your problem. This means currentOffset was too high, or the length derived from previous parsing was incorrect, or the initial data list was not as long as expected. Specific to the STREAM frame type parsing:
Listing 19 0b00001011 (0x0B): Offset=1, Length=1, FIN=1. This means Offset is present, Length is present, FIN is set. Listing 20 0b00001111 (0x0F): Offset=1, Length=1, FIN=1. Same as above. Ensure your hasOffset and hasLength checks correctly match these bit flags in the type byte.
General VarInt Debugging Tips: Create a dedicated test file for your VarInt helper methods. Test various VarInt values: 0, 1, 63 (single byte max), 64, 16383 (two byte max), 16384, etc. Test the read function with byte arrays where the VarInt is not at offset 0, and where it's followed by other data. Ensure VarInt.getLength(value) correctly returns 1, 2, 4, or 8 based on the value. Correct VarInt Logic (Reminder):
Prefix Length (bytes) Range (decimal) Example 00 1 0 to 63 0x00 (0), 0x3F (63) 01 2 64 to 16383 0x4000 (64), 0x7FFF (16383) 10 4 16384 to 1073741823 0x80000000 (16384) 11 8 1073741824 to 4611686018427387903 0xC000000000000000
Export to Sheets Your VarInt.read function must:
Read the first byte. Inspect the two most significant bits to determine the total length of the VarInt. Read the subsequent bytes accordingly. Combine these bytes to form the final integer value, removing the prefix bits. By carefully stepping through your parsing logic with these specific test cases and verifying the offset and length values at each step, you should be able to pinpoint and fix the bugs.
Deep Research
Canvas
Gemini can make mistakes, so double-check it