Refund HTTP/2 connection credit for buffered DATA - #1
Conversation
rgarcia
left a comment
There was a problem hiding this comment.
approving — the fix is right and the mechanism checks out. verified the regression test fails on base without the fix, race detector is clean, and the full-suite failure set is byte-identical to master, so the pre-existing breakage claim holds. batching the stream-0 WINDOW_UPDATEs to a half-window cadence instead of per-frame is the right call for the fingerprint too.
one question before we tag a release and bump consumers: the connection window was incidentally the only cap on unread buffered body data per connection (~15.7MB). after this it's 6MB × concurrent streams — we've observed 100+ open unread bodies on a busy host, so worst case is potentially hundreds of MB per connection with nothing bounding the dataBuffer growth. that's what chrome does too, but chrome has its own memory accounting on top and we don't.
can we either:
- size it — what's the p99 concurrent unread bodies per pooled connection under real load? if it's <20 this is a non-issue, or
- add telemetry — a gauge for buffered-unread bytes per ClientConn (sum of bufPipe lengths) exposed so callers can export it, and we watch process RSS during rollout
either answers it. (2) is probably worth having regardless since this class of bug was invisible until we added body-level tracking.
summary
unsentConnRefundand are announced once they reach half the connection window, instead of one frame + flush per DATA frame — avoids hot-path write amplification and a per-frame stream-0 WINDOW_UPDATE cadence that no real browser produceswhy this diverges from upstream x/net
upstream returns connection credit on body reads and never hits this starvation because its default connection window is 1 GB against 4 MB stream windows. fhttp advertises browser-realistic windows (
ConnectionFlow15663105 with 6 MB streams for Chrome), so three unread bodies can pin the whole connection window. refunding at buffer time is required here; the batching mirrors the accumulate-and-threshold approach upstream adopted in CL 448155 (inflow).testing
go test -vet=off -run 'TestTransportPausedBodiesDoNotExhaustConnectionWindow' -count=10 ./http2go test -race -vet=off -run 'TestTransportPausedBodiesDoNotExhaustConnectionWindow|TestTransportReturnsUnusedFlowControl|TestTransportReturnsDataPaddingFlowControl' -count=3 ./http2go test -vet=off -run 'TestTransportReturnsUnusedFlowControl|TestTransportReturnsDataPaddingFlowControl|TestTransportAdjustsFlowControl' ./http2TestTransportFlowControland a nil-pointer panic in legacy server tests). verified the failure set is byte-identical between this branch and its base.