Skip to content

Conversation

@ds5678
Copy link
Contributor

@ds5678 ds5678 commented Oct 28, 2025

Merge duplicate code under a new class CXXMethodDeclarationCSharpTest

Merge duplicate code under a new class CXXMethodDeclarationCSharpTest
@ds5678 ds5678 changed the title Update testing framework for C# Update testing framework Oct 29, 2025
@ds5678
Copy link
Contributor Author

ds5678 commented Oct 29, 2025

Although I think this is an improvement over what was there (and I would appreciate a merge), I don't think it's a good long-term solution to the problem. I occasionally work on ILSpy, which I think has a great system for unit testing. Every test method has a pair of files (corresponding to input and output) and a set of multiple options on which to test.

https://github.com/icsharpcode/ILSpy/tree/2f792f6fb663edbdd6517565daf0ea8e0e11296b/ICSharpCode.Decompiler.Tests/TestCases/ILPretty

https://github.com/icsharpcode/ILSpy/blob/2f792f6fb663edbdd6517565daf0ea8e0e11296b/ICSharpCode.Decompiler.Tests/ILPrettyTestRunner.cs

https://github.com/icsharpcode/ILSpy/blob/2f792f6fb663edbdd6517565daf0ea8e0e11296b/ICSharpCode.Decompiler.Tests/PrettyTestRunner.cs#L284

The output files are whitespace flexible and support macros for altering the expected output based on testing options. In our case, the set of macros would likely include:

  • WINDOWS
  • UNIX
  • COMPATIBLE
  • DEFAULT
  • LATEST
  • PREVIEW
  • BIT32
  • BIT64

They use a custom class for comparing program output with expected output.

https://github.com/icsharpcode/ILSpy/blob/2f792f6fb663edbdd6517565daf0ea8e0e11296b/ICSharpCode.Decompiler.Tests/Helpers/CodeAssert.cs

@tannergooding
Copy link
Member

tannergooding commented Nov 1, 2025

Every test method has a pair of files (corresponding to input and output) and a set of multiple options on which to test.

I am not a fan of checking in what are effectively "baseline" files or building up complex infrastructure (which effectively needs its own testing) to run these tests.

The current approach had been done with because it was simple. What was largely missing was the sharing of the test between versions when it was identical (what this PR largely fixes) to avoid unnecessary duplication.

There is then another factor where the inputContents are always identical between tests and its only the expectedOutputContents that should differ in some cases. Sharing the input is fairly easy while tweaking the output between the tests is harder without more error prone infrastructure.

Part of it could potentially be done by just having clear hierarchy to the tests. CXXMethodDeclarationTest and then a Compatible which inherits from that and a Preview : Latest : Compatible setup. Windows vs Unix differences would probably have to be changed to be based off a condition since you can't have multiple inheritance. This would maximize sharing and only require overriding in the case where the newer deviates from the older.

@tannergooding tannergooding merged commit 6759bb0 into dotnet:main Nov 1, 2025
14 checks passed
@ds5678 ds5678 deleted the update-testing-framework-for-csharp branch November 2, 2025 04:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants