transport: Avoid buffer copies when reading Data frames #8657

arjan-bal · 2025-10-16T15:16:36Z

This change incorporates changes from golang/go#73560 to split reading HTTP/2 frame headers and payloads. If the frame is not a Data frame, it's read through the standard library framer as before. For Data frames, the payload is read directly into a buffer from the buffer pool to avoid copying it from the framer's buffer.

Testing

For 1 MB payloads, this results in ~4% improvement in throughput.

# test command
go run benchmark/benchmain/main.go -benchtime=60s -workloads=streaming \
   -compression=off -maxConcurrentCalls=120 -trace=off \
   -reqSizeBytes=1000000 -respSizeBytes=1000000 -networkMode=Local -resultFile="${RUN_NAME}"

# comparison
go run benchmark/benchresult/main.go streaming-before streaming-after  
               Title       Before        After Percentage
            TotalOps        87536        91120     4.09%
             SendOps            0            0      NaN%
             RecvOps            0            0      NaN%
            Bytes/op   4074102.92   4070489.30    -0.09%
           Allocs/op        83.60        76.55    -8.37%
             ReqT/op 11671466666.67 12149333333.33     4.09%
            RespT/op 11671466666.67 12149333333.33     4.09%
            50th-Lat  78.209875ms  75.159943ms    -3.90%
            90th-Lat 117.764228ms   107.8697ms    -8.40%
            99th-Lat 146.935704ms 139.069685ms    -5.35%
             Avg-Lat  82.310691ms  79.073282ms    -3.93%
           GoVersion     go1.24.7     go1.24.7
         GrpcVersion   1.77.0-dev   1.77.0-dev

For smaller payloads, the difference in minor.

go run benchmark/benchmain/main.go -benchtime=60s -workloads=streaming \
   -compression=off -maxConcurrentCalls=120 -trace=off \
   -reqSizeBytes=100 -respSizeBytes=100 -networkMode=Local -resultFile="${RUN_NAME}"

go run benchmark/benchresult/main.go streaming-before streaming-after 
               Title       Before        After Percentage
            TotalOps     21490752     21477822    -0.06%
             SendOps            0            0      NaN%
             RecvOps            0            0      NaN%
            Bytes/op      1902.92      1902.94     0.00%
           Allocs/op        29.21        29.21     0.00%
             ReqT/op 286543360.00 286370960.00    -0.06%
            RespT/op 286543360.00 286370960.00    -0.06%
            50th-Lat    352.505µs    352.247µs    -0.07%
            90th-Lat    433.446µs    434.907µs     0.34%
            99th-Lat    536.445µs    539.759µs     0.62%
             Avg-Lat    333.403µs    333.457µs     0.02%
           GoVersion     go1.24.7     go1.24.7
         GrpcVersion   1.77.0-dev   1.77.0-dev

RELEASE NOTES:

transport: Avoid a buffer copy when reading data.

codecov · 2025-10-16T15:20:46Z

Codecov Report

❌ Patch coverage is 85.88235% with 12 lines in your changes missing coverage. Please review.
✅ Project coverage is 83.29%. Comparing base (64cba2d) to head (f0e9b13).
⚠️ Report is 1 commits behind head on master.

Files with missing lines	Patch %	Lines
internal/transport/http_util.go	84.37%	3 Missing and 7 partials ⚠️
internal/transport/http2_client.go	90.90%	1 Missing ⚠️
internal/transport/http2_server.go	90.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #8657      +/-   ##
==========================================
- Coverage   83.29%   83.29%   -0.01%     
==========================================
  Files         415      415              
  Lines       32134    32196      +62     
==========================================
+ Hits        26767    26817      +50     
- Misses       4001     4005       +4     
- Partials     1366     1374       +8

Files with missing lines	Coverage Δ
mem/buffer_pool.go	`100.00% <ø> (ø)`
internal/transport/http2_client.go	`92.90% <90.90%> (+0.19%)`	⬆️
internal/transport/http2_server.go	`91.07% <90.00%> (ø)`
internal/transport/http_util.go	`90.77% <84.37%> (-4.45%)`	⬇️

... and 24 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

internal/transport/http_util.go

internal/transport/http_util_test.go

arjan-bal

I realised that the changes in the std lib framer are not imported in g3 os this PR can't be merged in gRPC yet. I've added a Blocked label to indicate this.

internal/transport/http_util_test.go

internal/transport/http_util.go

easwars · 2025-10-27T22:26:13Z

I realised that the changes in the std lib framer are not imported in g3 os this PR can't be merged in gRPC yet. I've added a Blocked label to indicate this.

When is it expected to be imported?

internal/transport/http_util.go

arjan-bal · 2025-10-28T15:57:24Z

I realised that the changes in the std lib framer are not imported in g3 os this PR can't be merged in gRPC yet. I've added a Blocked label to indicate this.

When is it expected to be imported?

I think importing the code is self-service. I'm working on it: cl/824893868

easwars · 2025-10-28T21:06:12Z

@dfawley: Moving to your plate in case you want to take a look. Thanks.

mem/buffer_pool.go

dfawley · 2025-10-30T17:52:36Z

internal/transport/http2_client.go

 				t.controlBuf.put(&outgoingWindowUpdate{s.id, w})
 			}
 		}
-		// TODO(bradfitz, zhaoq): A copy is required here because there is no


dfawley · 2025-10-30T17:54:19Z

internal/transport/http_util.go

+	if memPool == nil {
+		// Note that this is only supposed to be nil in tests. Otherwise, stream
+		// is always initialized with a BufferPool.
+		memPool = mem.DefaultBufferPool()
+	}


Is it hard to fix the tests? Maybe we can do that as a follow-up.

Acknowledged.

Raised #8688 with the cleanup.

Co-authored-by: Doug Fawley <[email protected]>

…-read

arjan-bal added this to the 1.77 Release milestone Oct 16, 2025

arjan-bal added Type: Performance Performance improvements (CPU, network, memory, etc) Area: Transport Includes HTTP/2 client/server and HTTP server handler transports and advanced transport features. labels Oct 16, 2025

arjan-bal force-pushed the copyless-data-frame-read branch from b5f777e to ae8c8df Compare October 16, 2025 16:54

arjan-bal self-assigned this Oct 16, 2025

arjan-bal force-pushed the copyless-data-frame-read branch 2 times, most recently from efe661b to 778af0a Compare October 16, 2025 18:53

arjan-bal added 4 commits October 17, 2025 00:40

changes no tests

9875111

tests

a6f1b72

cleanup

3f98471

eliminate extra heap alloc

5fa01cd

arjan-bal force-pushed the copyless-data-frame-read branch from 778af0a to 45e856f Compare October 16, 2025 19:12

bump x/net to golang.org/x/net@63d1a51

5ec4a4e

arjan-bal force-pushed the copyless-data-frame-read branch from 45e856f to 5ec4a4e Compare October 16, 2025 19:15

arjan-bal removed their assignment Oct 16, 2025

arjan-bal requested review from dfawley and easwars and removed request for easwars October 16, 2025 19:21

arjan-bal assigned easwars and dfawley Oct 16, 2025

easwars reviewed Oct 16, 2025

View reviewed changes

internal/transport/http_util.go Outdated Show resolved Hide resolved

internal/transport/http_util.go Show resolved Hide resolved

internal/transport/http_util.go Show resolved Hide resolved

internal/transport/http_util_test.go Outdated Show resolved Hide resolved

easwars assigned arjan-bal and unassigned easwars Oct 16, 2025

arjan-bal added 2 commits October 21, 2025 14:14

fix error codes and interface types

6820fee

Improve testing of error cases

00586ba

arjan-bal commented Oct 21, 2025

View reviewed changes

internal/transport/http_util_test.go Outdated Show resolved Hide resolved

internal/transport/http_util.go Show resolved Hide resolved

internal/transport/http_util.go Show resolved Hide resolved

internal/transport/http_util.go Outdated Show resolved Hide resolved

arjan-bal added the Status: Blocked label Oct 21, 2025

arjan-bal assigned easwars Oct 21, 2025

arjan-bal removed their assignment Oct 21, 2025

easwars reviewed Oct 27, 2025

View reviewed changes

internal/transport/http_util.go Outdated Show resolved Hide resolved

internal/transport/http_util.go Show resolved Hide resolved

internal/transport/http_util.go Show resolved Hide resolved

easwars assigned arjan-bal and unassigned dfawley Oct 27, 2025

doc comments

8ef2d03

easwars approved these changes Oct 28, 2025

View reviewed changes

easwars assigned dfawley and unassigned easwars and arjan-bal Oct 28, 2025

arjan-bal modified the milestones: 1.77 Release, 1.78 Release Oct 30, 2025

arjan-bal removed the Status: Blocked label Oct 30, 2025

dfawley approved these changes Oct 30, 2025

View reviewed changes

dfawley assigned arjan-bal and unassigned dfawley Oct 30, 2025

arjan-bal and others added 2 commits October 31, 2025 11:13

Apply suggestions from code review

a121b15

Co-authored-by: Doug Fawley <[email protected]>

Fix comment

e7b6d35

arjan-bal force-pushed the copyless-data-frame-read branch from 8537f97 to e7b6d35 Compare October 31, 2025 06:08

arjan-bal added 2 commits October 31, 2025 11:47

Merge remote-tracking branch 'source/master' into copyless-data-frame…

5585538

…-read

Fix merge conflicts

f0e9b13

arjan-bal merged commit 363018c into grpc:master Oct 31, 2025
14 checks passed

arjan-bal deleted the copyless-data-frame-read branch October 31, 2025 06:27

transport: Avoid buffer copies when reading Data frames #8657

transport: Avoid buffer copies when reading Data frames #8657

Uh oh!

Conversation

arjan-bal commented Oct 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Testing

Uh oh!

codecov bot commented Oct 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

arjan-bal left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

easwars commented Oct 27, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

arjan-bal commented Oct 28, 2025

Uh oh!

easwars commented Oct 28, 2025

Uh oh!

Uh oh!

dfawley Oct 30, 2025

Choose a reason for hiding this comment

Uh oh!

dfawley Oct 30, 2025

Choose a reason for hiding this comment

Uh oh!

arjan-bal Oct 31, 2025

Choose a reason for hiding this comment

Uh oh!

arjan-bal Oct 31, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

arjan-bal commented Oct 16, 2025 •

edited

Loading

codecov bot commented Oct 16, 2025 •

edited

Loading