Safer stackallocs #110864

EgorBo · 2024-12-20T15:35:48Z

First batch of changes around various stackalloc expressions in BCL.

The rules are:

Make sure unbound stackallocs have upper (and lower!) bound checks. Add (uint) casts to handle negative/overflows as well (I've checked all places that it's only applied to int, no long, etc.). In many places it's redundant, but it shouldn't affect the performance and reduce number of false-positives reported by automated tooling.
Also, add Debug.Assert to make it more clear.
2) For patterns like len > CONST : stackalloc T[CONST] : new T[len] we change stackalloc T[CONST] to stackalloc T[len] if that CONST is >= 1024 bytes in order to consume less stack. It shouldn't hurt performance since our libs are compiled with SkipLocalsInit (although, a small overhead still there, so we leave it as is for small const sizes).
UPD: although, maybe we should do it only when it's saved to Span to make sure nobody relies on some minimal size..
Flag suspiciously large allocations
Change unmanaged pointers to Spans on the left side of the stackalloc expression (this PR doesn't do it much).

…k-struct # Conflicts: # src/libraries/System.Private.CoreLib/src/System/Globalization/CalendarData.Browser.cs

vcsjones · 2024-12-20T15:44:04Z

change stackalloc T[CONST] to stackalloc T[len] if that CONST is >= 512 bytes in order to consume less stack.

I don't know how much of a win it is to use less stack,but in general I think it is preferable to keep a const in for the stack size. It makes it much easier to reason about "how much is going to be stack allocated" during code review and static analysis.

cc @GrabYourPitchforks Since we were just chatting about this.

EgorBo · 2024-12-20T15:46:51Z

change stackalloc T[CONST] to stackalloc T[len] if that CONST is >= 512 bytes in order to consume less stack.

I don't know how much of a win it is to use less stack,but in general I think it is preferable to keep a const in for the stack size. It makes it much easier to reason about "how much is going to be stack allocated" during code review and static analysis.

cc @GrabYourPitchforks Since we were just chatting about this.

I don't have a strong preference here, but in some cases it's possible that e.g. len is 4, but we still unconditionally allocate 1024 bytes for it

EgorBo · 2024-12-20T16:18:32Z

Related discussion: #97895

vcsjones · 2024-12-20T16:34:18Z

#110843 is still not fully fixed. It changes a potential stack overflow in to an access violation. This computation is done unchecked:

runtime/src/libraries/System.Diagnostics.PerformanceCounter/src/System/Diagnostics/PerformanceData/CounterSet.cs

Lines 208 to 209 in b6038a4

    
           uint CounterSetInfoSize = (uint)sizeof(Interop.PerfCounter.PerfCounterSetInfoStruct) 
        
                           + (uint)_idToCounter.Count * (uint)sizeof(Interop.PerfCounter.PerfCounterInfoStruct);

Since it is unsigned and unchecked, can overflow to a small positive value

When we try to access memory here:

runtime/src/libraries/System.Diagnostics.PerformanceCounter/src/System/Diagnostics/PerformanceData/CounterSet.cs

Line 234 in b6038a4

    
           CounterInfo = (Interop.PerfCounter.PerfCounterInfoStruct*)(CounterSetBuffer + CounterSetInfoUsed);

The address will be invalid.

Can be reproduced on this PR with

[Fact]
public static void BigNumberOfCounters()
{
    CounterSet counterSet = new(Guid.NewGuid(), Guid.NewGuid(), CounterSetInstanceType.Single);

    for (int i = 0; i <= 134_217_726; i++)
    {
        counterSet.AddCounter(i, CounterType.ElapsedTime);
    }

    counterSet.CreateCounterSetInstance("potato");
}

EgorBo · 2024-12-20T16:43:06Z

#110843 is still not fully fixed. It changes a potential stack overflow in to an access violation. This computation is done unchecked:

Thanks. Just wrapping the compution into checked context should be enough? There are many patterns where stackalloc size is computed like that, presumably, we should checked them all?

vcsjones · 2024-12-20T16:53:07Z

Just wrapping the computation into checked context should be enough?

Seems reasonable.

There are many patterns where stackalloc size is computed like that, presumably, we should checked them all?

I don't see a problem with it. Many will probably be unnecessary, but I don't think it introduces any overhead that is problematic.

jkotas · 2024-12-20T16:55:15Z

For patterns like len > CONST : stackalloc T[CONST] : new T[len] we change stackalloc T[CONST] to stackalloc T[len] if that CONST is >= 1024 bytes in order to consume less stack.

Hardcoded ad-hoc policies like this are never going be "right". The proper fix would be an API like #52065 so that one can just ask for a scratch buffer and leave the choice of strategy to the system.

tannergooding · 2024-12-20T16:59:04Z

I don't have a strong preference here, but in some cases it's possible that e.g. len is 4, but we still unconditionally allocate 1024 bytes for it

I think this is a case where we should be ensuring our code is correctly tuned.

If we know the vast majority of cases will be <= n bytes then that's a much better threshold to use than the default max threshold (which is generally 1024, which comes from the typical threshold used by native malloca implementations). If we have effectively two thresholds to handle (most are small, some are slightly larger), then I think that doing 2 explicit stackalloc thresholds are better (if (size <= 64) { stackalloc byte[64]; } else if (size <= 1024) { stackalloc byte[1024]; } else { RentOrAllocateNewArray; }).

Dynamic stackalloc is also "expensive", it introduces a loop that has to probe pages. All our usages will be a single page probe, which means that will be mispredicted by the CPU in the default scenario, incurring an approx 20 cycle penalty. Since stackalloc should not be getting used in known recursive functions or directly in loops, it is unlikely the predictor will ever get that trained correctly and this will be a "permanent" cost. Dynamic stackalloc also means the JIT cannot easily optimize around it, like it currently does with some small fixed allocation sizes.

I also don't think the JIT implicitly doing things to the primitive stackalloc pattern is goodness, it makes code that is likely being used in lowlevel/tuned scenarios less predictable, meaning it can't be used for it's intended scenario. I'd rather instead see us invest in actually exposing our own StackallocOrRent helper that can then do such optimizations itself, giving users the option of convenience vs control.

Notably dynamic length stackallocs are often considered dangerous as well. The pattern we're using here is safe (relatively speaking), but as the pattern gets copied around and used its also easy for people to misunderstand and do the wrong thing. Since we're trying to reduce unsafe code, I think being explicit and adding documentation around these areas and why certain thresholds are used/correct is a better investment; as is pushing for the helper intrinsic that generally simplifies the pattern and abstracts away the "right" sizes to use so that it can be correctly tuned, potentially participate in PGO, etc.

EgorBo · 2024-12-20T18:25:20Z

Reverted that part. Added checked everywhere where length is calculated near the stackalloc.
Happy to replace all places with a helper call, but seems like it won't be there soon.

jkotas · 2024-12-20T19:25:12Z

...ies/Common/src/Interop/Android/System.Security.Cryptography.Native.Android/Interop.Cipher.cs

            }

-            fixed (byte* pOutput = &MemoryMarshal.GetReference(notNullOutput))
+            fixed (byte* pOutput = notNullOutput)


I think this change is introducing a bug. It looks like the code used MemoryMarshal.GetReference to avoid normalization of empty spans to null. You would have to also delete .Slice(1) above to make this change work.

If the only requirement is that the pointer is non-null, wouldn't it make sense to use MemoryMarshal.GetNonNullPinnableReference? It's used in Encoding for the same reason.

Thanks! Replaced with GetNonNullPinnableReference

src/libraries/Common/src/System/Net/SocketAddress.cs

Co-authored-by: Miha Zupan <[email protected]>

EgorBo · 2024-12-20T23:19:32Z

This is ready for review. Here is the code analyzer I used to find all unbound stackallocs: https://github.com/EgorBo/UnsafeCodeAnalyzer/blob/main/src/Analyzer/UnboundStackallocAnalyzer.cs (reports 263 places in Libraries and Corelib excluding tests)

ChrML · 2024-12-25T00:40:12Z

For patterns like len > CONST : stackalloc T[CONST] : new T[len] we change stackalloc T[CONST] to stackalloc T[len] if that CONST is >= 1024 bytes in order to consume less stack.

Hardcoded ad-hoc policies like this are never going be "right". The proper fix would be an API like #52065 so that one can just ask for a scratch buffer and leave the choice of strategy to the system.

I think if you already use "stackalloc", you should already have a good reason, you work low-level, and should have knowledge about when it's reasonable to do so. Could maybe make an analyzer that warns on stackallocing unverified dynamic lengths.

Imo it's better longterm to improve cases where normal new operations are recognized by JIT to not escape, do the length check there and automatically stackalloc them. And leave stackalloc as-is, a programmer controlled thing. Benefits existing code without changing heuristics. If this gets good enough in the future, at some point ImmutableArray.Builder with simple valuetypes populated by a loop, could occur only on the stack up to a certain number of items even putting the builder class on the stack.

I don't think simplifying ops like this is needed, except maybe being able to create scratch buffers inside methods that get allocated at callsite (such as ArrayBuilder.Create(int initialCapacity), that could return a ref struct and decide internally if put on stack or heap). Instead of passing in scratch area.

EgorBo · 2024-12-25T01:59:02Z

Could maybe make an analyzer that warns on stackallocing unverified dynamic lengths.

Presumably, that requires quite complex range check analysis like nullability. Our general strategy is to avoid spending efforts on low-level/unsafe things, if someone decides to go the unsafe route, they're on their own

Imo it's better longterm to improve cases where normal new operations are recognized by JIT to not escape, do the length check there and automatically stackalloc them.

Nobody argues with that, but it might take a while till we get there, the most complicated part is the Inter-procedure analysis on top of functions which might change their behavior any time (arguments suddenly start escaping) e.g. due to profilers with ReJIT attached in the middle of the flight.

I don't think simplifying ops like this is needed

I think till we get to the proper escape analysis it could be a reasonable API to simplify these patterns, I personally don't have a strong opinion on it. Another "cons" of the escape analysis is that it might be fragile and you will never know whether it kicks in or not

ChrML · 2024-12-25T02:12:10Z

Could maybe make an analyzer that warns on stackallocing unverified dynamic lengths.

Presumably, that requires quite complex range check analysis like nullability. Our general strategy is to avoid spending efforts on low-level/unsafe things, if someone decides to go the unsafe route, they're on their own

Imo it's better longterm to improve cases where normal new operations are recognized by JIT to not escape, do the length check there and automatically stackalloc them.

Nobody argues with that, but it might take a while till we get there, the most complicated part is the Inter-procedure analysis on top of functions which might change their behavior any time (arguments suddenly start escaping) e.g. due to profilers with ReJIT attached in the middle of the flight.

I don't think simplifying ops like this is needed

I think till we get to the proper escape analysis it could be a reasonable API to simplify these patterns, I personally don't have a strong opinion on it. Another "cons" of the escape analysis is that it might be fragile and you will never know whether it kicks in or not

Your reasoning makes very much sense, I agree.

For me personally, I think example below would increase my productivity/safety most writing low-level / high-performant code. It would help more than any language word or implicit API. Being able to write methods like this:

static MyRefStructArrayBuilder MyRefStructArrayBuilder.CreateOnStackOrHeap();
static MyRefStructArrayBuilder MyRefStructArrayBuilder.CreateOnStackOrArrayPool();

Basically moving responsability for making the stack scratch area, and where to allocate if not fitting the stack, into a common reusable place. Within the helper / builder that's used elsewhere. Rather than having this allocation at each callsite and pass it in.

No idea how it would be solved though, as you can't stack alloc inside those and return that area without some weird caller/callee contract. It'd have to be like a macro that's always inlined to the callsite or something.

Just some thoughts, maybe someone else can spin off on it, or confirm if they have the same use case as me.

stephentoub · 2025-01-06T15:31:34Z

src/coreclr/System.Private.CoreLib/src/System/RuntimeType.CoreCLR.cs

-                            else
-                            {
-                                byte* pUtf8Name = stackalloc byte[cUtf8Name];
                                list = GetListByName(pName, cNameLen, pUtf8Name, cUtf8Name, listType, cacheType);


Does GetListByName still need to accept pointers?

Addressed! Although, Filter and its MdUtf8String still expect pointers, I'll look into cleaning it up separately

stephentoub · 2025-01-06T15:32:36Z

src/coreclr/System.Private.CoreLib/src/System/RuntimeType.CoreCLR.cs


-                        bool* overrides = stackalloc bool[numVirtuals];
-                        new Span<bool>(overrides, numVirtuals).Clear();
+                        Span<bool> overrides = (uint)numVirtuals > 512 ? new bool[numVirtuals] : stackalloc bool[numVirtuals];


Is 512 here just a heuristic?

Yes, like everywhere else? Do you mean it should be a named constant?

I've tended to leave a short comment about it being an arbitrary cutoff. I wasn't sure in this particular case if there was meaning to 512 with regards to how many virtuals could exist.

Added a comment

stephentoub · 2025-01-06T15:35:11Z

src/libraries/Common/src/Interop/Unix/System.Security.Cryptography.Native/Interop.Ssl.cs

        {
            int length = GetAlpnProtocolListSerializedLength(applicationProtocols);
-            Span<byte> buffer = length <= 256 ? stackalloc byte[256].Slice(0, length) : new byte[length];
+            Span<byte> buffer = (uint)length <= 256 ? stackalloc byte[256].Slice(0, length) : new byte[length];


When are we deciding to do stackalloc T[const].Slice(0, length) vs stackalloc T[length]? Both show up in this PR.

I decided to preserve the current behavior in this PR. Although, we agreed that stackalloc T[const].Slice(0, length) pattern is generally better.

Although, we agreed that stackalloc T[const].Slice(0, length) pattern is generally better.

Which is unfortunate as it's not intuitive or canonical: I'd hope we could make "stackalloc T[length]" do the "right thing".

Although, we agreed that stackalloc T[const].Slice(0, length) pattern is generally better.

Which is unfortunate as it's not intuitive or canonical: I'd hope we could make "stackalloc T[length]" do the "right thing".

@stephentoub we had a discussion in this PR about it (starting from #110864 (comment) comment and below).

TL;DR:
stackalloc T[length]:
Pros:

less stack consumption (if the length is less than the threshold)

simpler code

Cons:

Performance impact (unknown length leads to additional stack probes, a lot slower initialization if SkipLocalsInit is not set)

Constant-sized stackalloc reads better/safer

Thanks. I understand the current tradeoffs. I'm suggesting we should think through how to avoid the perf cost of the simpler approach.

Thanks. I understand the current tradeoffs. I'm suggesting we should think through how to avoid the perf cost of the simpler approach.

I guess performance was not even the main concern in that debate? I agree with Jan that we either want an API that does what is better under the hood, or we invest into escape analysis and just rely on JIT

If we can avoid having to write stackalloc altogether, all the better.

But until then, I don't think we should be proliferating changing stackalloc T[length] to stackalloc T[const].Slice(0, length). That's just creating less readable / maintainable debt that will also proliferate through the ecosystem as a "best practice" when it's really just a workaround for a perf limitation, and a small one at that. I also disagree with the latter being much safer, in particular in all of the relevant cases here, where it's immediately guarded by a length check such that it's obvious just from looking at that one line that the length is in-bounds.

stephentoub · 2025-01-06T15:40:34Z

src/libraries/Common/src/Interop/Windows/SspiCli/SSPIWrapper.cs



-            Span<IntPtr> buffer = stackalloc IntPtr[1];
+            Span<byte> buffer = stackalloc byte[IntPtr.Size];


The codegen for:

IntPtr p = default; Span<byte> buffer = MemoryMarshal.AsBytes(new Span<IntPtr>(ref p));

ends up being better:
https://sharplab.io/#v2:EYLgxg9gTgpgtADwGwBYA0AXEBDAzgWwB8ABAJgEYBYAKGIAYACY8gOgCUBXAOwwEt8YLAMIR8AB14AbGFADKMgG68wMXAG4a9Jq049+ggJI8ZEMfKhKV6mpoDMTUgyEMaAbxoNPTe8RQMAsuQAFACULtReDO4RkV6yYthcADzAAJ4YMAB8DMAcAGZ5MgwAvAy4GNhgANbYkpIQYDnpMADaRhgAChhQLLK8AF4wALoaMbEMAHKmQbkFMiGjkQC+Hl6rnsQ+fv6koeGR0eOe8YlJ7V1Q2bOFUCVlFdW19Y3n3S3kI+uxU2JB/jD4aCpfzYKC4AAWtRYAEFcAAhZq4Gb5G4hBZfFZjfZeTZMba2PZfQ5HV63a5FUoAExgeWwHEkGEWRx+fwBQJBYMhkhh8MRQS4MAA7gwTslSZkgrA8jkUfM0UzPJjIl8Wv8MOCIJSDOJJKz1ZrtWJJAB5MR8CBcXAsKZGSS8Lj2gDmISGXzEUF4CmwGW0SDxk2mopSzSuiJCrkxSyAA==

Is there any way to write the equivalent without using MemoryMarshal.AsBytes? Maybe with some JIT tweaks?

From my understanding, the codegen is better because your snippet doesn't require a GS cookie while we always emit it for stackallocs (whether we need it or not for such small stackallocs has been discussed here: #52979)

I'll see if we can safely remove the GS cookie when JIT sees that the stackalloc is consumed into Span

stephentoub · 2025-01-06T15:42:15Z

src/libraries/Common/src/System/Net/IPv4AddressHelper.Common.cs

            int numberBase = IPv4AddressHelper.Decimal;
            int ch = 0;
-            long* parts = stackalloc long[3]; // One part per octet. Final octet doesn't have a terminator, so is stored in currentValue.
+            Span<long> parts = stackalloc long[3]; // One part per octet. Final octet doesn't have a terminator, so is stored in currentValue.


I seem to remember this resulted in some small regressions the last time it was attempted. Can we confirm that's no longer an issue?

I've just checked - no extra bounds checks generated

stephentoub · 2025-01-06T15:46:29Z

src/libraries/Common/src/System/Security/Cryptography/ECDiffieHellmanDerivation.cs

                        ret);

-                    Span<byte> part2 = stackalloc byte[ret.Length];
+                    Debug.Assert(ret.Length == 48);


Where does "48" come from?

Added a constant

stephentoub · 2025-01-06T15:50:36Z

src/libraries/System.Formats.Tar/src/System/Formats/Tar/TarHeader.Read.cs

-                    (buffer = ArrayPool<byte>.Shared.Rent((int)_size));
-                span = span.Slice(0, (int)_size);
+                    (buffer = ArrayPool<byte>.Shared.Rent((int)size));
+                span = span.Slice(0, (int)size);


If the size could actually be greater than Array.MaxValue, is this logic not busted?

To be fair, no clue - I presume my changes don't change the behavior here?

…k-struct

...ies/Common/src/Interop/Android/System.Security.Cryptography.Native.Android/Interop.Cipher.cs

jkotas · 2025-01-15T16:04:14Z

...ies/Common/src/Interop/Android/System.Security.Cryptography.Native.Android/Interop.Cipher.cs

-            }
-
-            fixed (byte* pOutput = &MemoryMarshal.GetReference(notNullOutput))
+            // We can't pass null down to the native shim, so create a valid pointer if we have an empty span,


We own the code for the native shim, so this looks like a self-inflicted wound. Would it be better to relax the argument checking in the native shim instead?

Good point! Addressed

EgorBo · 2025-01-16T14:40:04Z

/azp run runtime-androidemulator

azure-pipelines · 2025-01-16T14:40:22Z

Azure Pipelines successfully started running 1 pipeline(s).

EgorBo · 2025-01-16T18:22:37Z

/ba-g "timeouts on Build android-x86 Release AllSubsets_Mono and Build Libraries Test Run release coreclr windows x64 Debug"

EgorBo added 2 commits December 20, 2024 15:46

Check stackalloc bounds

46180a0

more

d56d562

ghost added the needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners label Dec 20, 2024

dotnet-policy-service bot assigned EgorBo Dec 20, 2024

EgorBo added 2 commits December 20, 2024 16:38

Merge branch 'main' of github.com:dotnet/runtime into remove-nullchec…

eaaedc8

…k-struct # Conflicts: # src/libraries/System.Private.CoreLib/src/System/Globalization/CalendarData.Browser.cs

fix bug

b6a534e

Clean up

2bde1b7

EgorBo added 2 commits December 20, 2024 18:51

revert to const

6f48506

add more 'checked' contexts

ba65b54

jkotas reviewed Dec 20, 2024

View reviewed changes

teo-tsirpanis added area-Meta and removed needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners labels Dec 20, 2024

address feedback

12e1615

EgorBo marked this pull request as ready for review December 20, 2024 21:13

MihaZupan reviewed Dec 20, 2024

View reviewed changes

src/libraries/Common/src/System/Net/SocketAddress.cs Outdated Show resolved Hide resolved

Update src/libraries/Common/src/System/Net/SocketAddress.cs

52bf611

Co-authored-by: Miha Zupan <[email protected]>

stephentoub reviewed Jan 6, 2025

View reviewed changes

stephentoub approved these changes Jan 6, 2025

View reviewed changes

EgorBo mentioned this pull request Jul 28, 2024

.NET 9 Unsafe Study #94941

Open

15 tasks

EgorBo added 3 commits January 14, 2025 21:34

Merge branch 'main' of github.com:dotnet/runtime into remove-nullchec…

9cfa540

…k-struct

Merge branch 'main' of github.com:dotnet/runtime into remove-nullchec…

037518f

…k-struct

Address feedback

122323c

jkotas reviewed Jan 15, 2025

View reviewed changes

...ies/Common/src/Interop/Android/System.Security.Cryptography.Native.Android/Interop.Cipher.cs Outdated Show resolved Hide resolved

jkotas reviewed Jan 15, 2025

View reviewed changes

Address feedback

2099fa7

EgorBo force-pushed the remove-nullcheck-struct branch from fa5175f to 2099fa7 Compare January 15, 2025 16:13

EgorBo mentioned this pull request Jan 15, 2025

Codegen for a constant-sized stackalloc into a Span<T> should be on par with explicit-sized no-init local declaration #52979

Open

This was referenced Jan 15, 2025

slow macOS - "##[error]The job running on agent Azure Pipelines 9 ran longer than the maximum time of 60 minutes." dotnet/dnceng#1883

Open

The Operation will be canceled. The next steps may not contain expected logs. dotnet/dnceng#3008

Open

EgorBo merged commit 7790117 into dotnet:main Jan 16, 2025
148 of 152 checks passed

EgorBo deleted the remove-nullcheck-struct branch January 16, 2025 18:34

github-actions bot locked and limited conversation to collaborators Feb 16, 2025

EgorBo added the reduce-unsafe label Mar 1, 2025



		Span<IntPtr> buffer = stackalloc IntPtr[1];
		Span<byte> buffer = stackalloc byte[IntPtr.Size];

Uh oh!

Safer stackallocs #110864

Safer stackallocs #110864

Uh oh!

Conversation

EgorBo commented Dec 20, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vcsjones commented Dec 20, 2024

Uh oh!

EgorBo commented Dec 20, 2024

Uh oh!

EgorBo commented Dec 20, 2024

Uh oh!

vcsjones commented Dec 20, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

EgorBo commented Dec 20, 2024

Uh oh!

vcsjones commented Dec 20, 2024

Uh oh!

jkotas commented Dec 20, 2024

Uh oh!

tannergooding commented Dec 20, 2024

Uh oh!

EgorBo commented Dec 20, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

EgorBo commented Dec 20, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ChrML commented Dec 25, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

EgorBo commented Dec 25, 2024

Uh oh!

ChrML commented Dec 25, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

stephentoub Jan 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

stephentoub Jan 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

stephentoub Jan 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

EgorBo commented Dec 20, 2024 •

edited

Loading

vcsjones commented Dec 20, 2024 •

edited

Loading

EgorBo commented Dec 20, 2024 •

edited

Loading

EgorBo commented Dec 20, 2024 •

edited

Loading

ChrML commented Dec 25, 2024 •

edited

Loading

ChrML commented Dec 25, 2024 •

edited

Loading

stephentoub Jan 6, 2025 •

edited

Loading

stephentoub Jan 6, 2025 •

edited

Loading

stephentoub Jan 6, 2025 •

edited

Loading