Skip to content

Don't drop a datagram when it can't be sent because it won't fit - #161

Open
holleman wants to merge 4 commits into
apple:mainfrom
holleman:datagram-requeue
Open

holleman wants to merge 4 commits into
apple:mainfrom
holleman:datagram-requeue

Conversation

@holleman

Copy link
Copy Markdown
Contributor

Make sure to requeue a datagram and let the caller know that it could not be sent. An example of this is when a pending ack has consumed space before the datagram.

Make sure to requeue a datagram and let the caller know that it
could not be sent.  An example of this is when a pending ack has
consumed space before the datagram.
@holleman holleman added the 🔨 semver/patch No public API change. label Sep 18, 2026
connection.log.datapath(
"datagram for flow \(firstFlowID.debugDescription) does not fit \(roomBeforeWriting) free bytes; requeueing"
)
datagrams.prepend(frame: datagramFrame)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, so here we just add the frame back in and frame array and that way pending items knows to build a packet here for that flow again, correct?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, redo it. If it was a pending ack it would have gone out and then when we return we can pick up an empty packet and then send the datagram.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general, this seems like a good idea to not drop locally -- but does this cause a case where not being able to send a particular datagram to be at the front of the list forever and if it is too big to fit, no other datagrams can send?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Head of line blocking is now solved by dropping anything that will never fit (and an error message).

@agnosticdev agnosticdev left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

connection.log.error("Unable to write datagram for flow \(firstFlowID)")
// Requeue rather than finalize and pass up the error so a retry is possible
connection.log.datapath(
"datagram for flow \(firstFlowID.debugDescription) does not fit \(roomBeforeWriting) free bytes; requeueing"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"datagram for flow \(firstFlowID.debugDescription) does not fit \(roomBeforeWriting) free bytes; requeueing"
"Datagram for flow \(firstFlowID.debugDescription) does not fit \(roomBeforeWriting) free bytes; requeueing"

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done!

A datagram too big for the flow should be dropped and prevent any
head-of-line blocking.  That works, but add a test to ensure that
as well.
)
} catch {
connection.log.error("Unable to write datagram for flow \(firstFlowID)")
// Requeue rather than finalize and pass up the error so a retry is possible

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only reason why this fails is because we couldn't write, but since we already requested a big enough frame how can this fail?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check my math here, assume we used QUIC options to set the initial packet size to 1350. When an upper asks for datagrams what is returned is 1305:

usableDatagramSize = 1350 - 16 tag - 1 base - 20 dcid - 4 pnReserve - 4 dgramFrameHdr = 1305

So the 1305 is the maximum data that could go into a datagram frame if it was the only thing in the packet. But if there is another frame present (here a higher-priority 5 byte ACK frame), there is not room for both the ACK and datagram.

ackFrameLength = 1 type + 1 vle(largestAcked) + 1 vle(ackDelay) + 1 vle(rangeCount) + 1 vle(firstRange) = 5

So the builder needs to realize this, let the ACK go, and then send the datagram. We can go about it in a couple of different ways, I may improve that logic a bit. But I think the math explains why we need some extra logic here unless I'm missing something.

@josephnoir

josephnoir commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

Does this fix the same thing as #206? This looks similar to the second commit I had there. In that case, I'll close my PR. Sorry for missing this one.

@holleman

holleman commented Oct 9, 2026

Copy link
Copy Markdown
Contributor Author

Does this fix the same thing as #206? This looks similar to the second commit I had there. In that case, I'll close my PR. Sorry for missing this one.

Yes, same issue - sorry I missed the overlap as well even though @agnosticdev highlighted it. I started heading towards a different solution that looked a little like yours. I think we need to drop things that will never fit and error. I'll make sure to pick up your unit test for sure.

that doesn't fit the remaining packet stays queued.  Add batch datagram
test - both from apple#206
@holleman
holleman requested a review from ekinnear as a code owner October 9, 2026 21:53
@holleman holleman changed the title Requeue a datagram when it can't be sent Don't drop a datagram when it can't be sent because it won't fit Oct 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

🔨 semver/patch No public API change.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants