SetRequestInterceptionAsync(true) causing Cannot read incomplete UTF-16 JSON text as string with missing low surrogate

[Tiggerito]

Description

Since 20.0.4 my code that uses SetRequestInterceptionAsync(true) can no longer load pages.

Complete minimal example reproducing the issue

Replace (dot) in the URL...

var options = new LaunchOptions { /*  */ };
var chromiumRevision = BrowserFetcher.DefaultRevision;
var browser = await Puppeteer.LaunchAsync(options, chromiumRevision);
var page = browser.NewPageAsync();

await page.SetRequestInterceptionAsync(true);

page.Request += async (sender, e) => await e.Request.ContinueAsync();

await page.GoToAsync("https://trimmingshop(dot)co(dot)uk/", new NavigationOptions
{
    WaitUntil = new[] { WaitUntilNavigation.Networkidle0 }
});

Expected behavior:

Page loads

Actual behavior:

GoToAsync causes an exception.

PuppeteerSharp.NavigationException: 'Navigating frame was detached: NetworkManager failed to process Fetch.requestPaused. The JSON value could not be converted to System.String. Path: $.request.postData | LineNumber: 0 | BytePositionInLine: 85183..    at System.Text.Json.ThrowHelper.ReThrowWithPath(ReadStack& state, Utf8JsonReader& reader, Exception ex)
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, T& value, JsonSerializerOptions options, ReadStack& state)
   at PuppeteerSharp.Cdp.NetworkManager.Client_MessageReceived(Object sender, MessageEventArgs e)'

Versions

PuppeteerSharp 20.0.4
.NET Core 3.1, .Net 9.0

Additional Information

20.0.3 works fine, but the e.MessageData in page.Client.MessageReceived sometimes throws the same exception when I try to convert it into a JSON string. I'm guessing it is related, but it's not a critical error as I can capture it. It seems that now the code in Client_MessageReceived is also trying to get at that data, and failing.

There was a change to PostData in 20.0.4, which is mentioned in the exception.

#2810

My other issue also seems related:

#2775

[kblok]

It seems trimmingshop blocked my IP. I can't test this on that website :(

[Tiggerito]

Try https://www(dot)skinelite(dot)com/

I have a lot of examples.

[kallin-messerer-SHG]

I am intermittently running into this issue with SetRequestInterceptionAsync true and false with the URL https://www(dot)mycnhistore(dot)com/us/en/newhollandag/serialNumberSearch/serialNumber/HHKH.

[Tiggerito]

It's still happening in 20.1.1

[kblok]

Let's give it another look 👍🏼

[sinand99]

Same here. SetRequestInterceptionAsync() is not working correctly. I get JSON Serialization errors when I enabled it.
Target page: https://www.zara.com/tr/tr/-usa--spor-sweatshirt-p03854792.html?v1=407659833

Exception thrown: 'System.NullReferenceException' in PuppeteerSharp.dll
Object reference not set to an instance of an object.

and

NetworkManager failed to process Fetch.requestPaused. The JSON value could not be converted to System.String. Path: $.request.postData | LineNumber: 0 | BytePositionInLine: 1830.. at System.Text.Json.Serialization.JsonConverter1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state) at System.Text.Json.JsonSerializer.ReadFromSpan[TValue](ReadOnlySpan1 utf8Json, JsonTypeInfo1 jsonTypeInfo, Nullable1 actualByteCount)
at System.Text.Json.JsonSerializer.Deserialize[TValue](JsonElement element, JsonSerializerOptions options)
at PuppeteerSharp.Cdp.NetworkManager.<Client_MessageReceived>d__44.MoveNext() in /home/runner/work/puppeteer-sharp/puppeteer-sharp/lib/PuppeteerSharp/Cdp/NetworkManager.cs:line 170

[kblok]

@Tiggerito @sinand99 can you test this PR #2887?

[Tiggerito]

That fixed it for me 👍