Skip to content

Conversation

@dotnet-maestro
Copy link
Contributor

@dotnet-maestro dotnet-maestro bot commented Oct 8, 2024

This pull request updates the following dependencies

From https://github.com/dotnet/sdk

  • Subscription: 736b3d22-d45c-4c36-de26-08db63374d9b
  • Build: 20241010.13
  • Date Produced: October 11, 2024 1:07:59 AM UTC
  • Commit: 1f220e48e5bc666e992b24f118a17e01ca316819
  • Branch: refs/heads/main

From https://github.com/dotnet/source-build-externals

  • Subscription: b4338da2-58ef-486c-5124-08dbb92a7ffc
  • Build: 20241021.1
  • Date Produced: October 21, 2024 5:15:38 PM UTC
  • Commit: 152bd2a5bc0d0f19033b42f48a026339d50101ad
  • Branch: refs/heads/main

From https://github.com/dotnet/icu

  • Subscription: 4247a230-8931-4538-5b64-08d8d8feb47e
  • Build: 20241021.2
  • Date Produced: October 21, 2024 3:18:09 PM UTC
  • Commit: c72b1a1b16e0c2afea821970cb793299f015d46e
  • Branch: refs/heads/dotnet/main

From https://github.com/dotnet/runtime-assets

  • Subscription: 0c5a34f5-504e-413b-9376-08d8d8ff2d75
  • Build: 20241021.1
  • Date Produced: October 21, 2024 4:29:14 PM UTC
  • Commit: a301793441179ac06d7e0e1d84cd5ebb12dd8009
  • Branch: refs/heads/main

From https://github.com/dotnet/hotreload-utils

  • Subscription: bfe6dacf-8231-4ea1-e2fe-08d962847885
  • Build: 20241014.1
  • Date Produced: October 14, 2024 3:05:49 PM UTC
  • Commit: 3f3ab836c338d37ecdda238f630e81f1bbde7126
  • Branch: refs/heads/main

From https://github.com/dotnet/cecil

  • Subscription: bb5d2106-9fd3-425f-0abc-08daad65778c
  • Build: 20241021.1
  • Date Produced: October 21, 2024 1:16:12 PM UTC
  • Commit: b897087e8b76481a9213ae422f5dc16f64a124b5
  • Branch: refs/heads/main

From https://github.com/dotnet/runtime

  • Subscription: 3db4b8c8-0fae-4f82-086c-08dad31ae87d
  • Build: 20241022.5
  • Date Produced: October 22, 2024 5:06:50 PM UTC
  • Commit: b780069
  • Branch: refs/heads/main

Coherency Updates

The following updates ensure that dependencies with a CoherentParentDependency
attribute were produced in a build used as input to the parent dependency's build.
See Dependency Description Format

  • Coherency Updates:
    • runtime.linux-arm64.Microsoft.NETCore.Runtime.JIT.Tools: from 19.1.0-alpha.1.24501.1 to 19.1.0-alpha.1.24503.1 (parent: Microsoft.NET.Workload.Emscripten.Current.Manifest-10.0.100.Transport)
    • runtime.linux-x64.Microsoft.NETCore.Runtime.JIT.Tools: from 19.1.0-alpha.1.24501.1 to 19.1.0-alpha.1.24503.1 (parent: Microsoft.NET.Workload.Emscripten.Current.Manifest-10.0.100.Transport)
    • runtime.linux-musl-arm64.Microsoft.NETCore.Runtime.JIT.Tools: from 19.1.0-alpha.1.24501.1 to 19.1.0-alpha.1.24503.1 (parent: Microsoft.NET.Workload.Emscripten.Current.Manifest-10.0.100.Transport)
    • runtime.linux-musl-x64.Microsoft.NETCore.Runtime.JIT.Tools: from 19.1.0-alpha.1.24501.1 to 19.1.0-alpha.1.24503.1 (parent: Microsoft.NET.Workload.Emscripten.Current.Manifest-10.0.100.Transport)
    • runtime.win-arm64.Microsoft.NETCore.Runtime.JIT.Tools: from 19.1.0-alpha.1.24501.1 to 19.1.0-alpha.1.24503.1 (parent: Microsoft.NET.Workload.Emscripten.Current.Manifest-10.0.100.Transport)
    • runtime.win-x64.Microsoft.NETCore.Runtime.JIT.Tools: from 19.1.0-alpha.1.24501.1 to 19.1.0-alpha.1.24503.1 (parent: Microsoft.NET.Workload.Emscripten.Current.Manifest-10.0.100.Transport)
    • runtime.osx-arm64.Microsoft.NETCore.Runtime.JIT.Tools: from 19.1.0-alpha.1.24501.1 to 19.1.0-alpha.1.24503.1 (parent: Microsoft.NET.Workload.Emscripten.Current.Manifest-10.0.100.Transport)
    • runtime.osx-x64.Microsoft.NETCore.Runtime.JIT.Tools: from 19.1.0-alpha.1.24501.1 to 19.1.0-alpha.1.24503.1 (parent: Microsoft.NET.Workload.Emscripten.Current.Manifest-10.0.100.Transport)
    • runtime.linux-arm64.Microsoft.NETCore.Runtime.Mono.LLVM.Sdk: from 19.1.0-alpha.1.24501.1 to 19.1.0-alpha.1.24503.1 (parent: Microsoft.NET.Workload.Emscripten.Current.Manifest-10.0.100.Transport)
    • runtime.linux-arm64.Microsoft.NETCore.Runtime.Mono.LLVM.Tools: from 19.1.0-alpha.1.24501.1 to 19.1.0-alpha.1.24503.1 (parent: Microsoft.NET.Workload.Emscripten.Current.Manifest-10.0.100.Transport)
    • runtime.linux-musl-arm64.Microsoft.NETCore.Runtime.Mono.LLVM.Sdk: from 19.1.0-alpha.1.24501.1 to 19.1.0-alpha.1.24503.1 (parent: Microsoft.NET.Workload.Emscripten.Current.Manifest-10.0.100.Transport)
    • runtime.linux-musl-arm64.Microsoft.NETCore.Runtime.Mono.LLVM.Tools: from 19.1.0-alpha.1.24501.1 to 19.1.0-alpha.1.24503.1 (parent: Microsoft.NET.Workload.Emscripten.Current.Manifest-10.0.100.Transport)
    • runtime.linux-x64.Microsoft.NETCore.Runtime.Mono.LLVM.Sdk: from 19.1.0-alpha.1.24501.1 to 19.1.0-alpha.1.24503.1 (parent: Microsoft.NET.Workload.Emscripten.Current.Manifest-10.0.100.Transport)
    • runtime.linux-x64.Microsoft.NETCore.Runtime.Mono.LLVM.Tools: from 19.1.0-alpha.1.24501.1 to 19.1.0-alpha.1.24503.1 (parent: Microsoft.NET.Workload.Emscripten.Current.Manifest-10.0.100.Transport)
    • runtime.linux-musl-x64.Microsoft.NETCore.Runtime.Mono.LLVM.Sdk: from 19.1.0-alpha.1.24501.1 to 19.1.0-alpha.1.24503.1 (parent: Microsoft.NET.Workload.Emscripten.Current.Manifest-10.0.100.Transport)
    • runtime.linux-musl-x64.Microsoft.NETCore.Runtime.Mono.LLVM.Tools: from 19.1.0-alpha.1.24501.1 to 19.1.0-alpha.1.24503.1 (parent: Microsoft.NET.Workload.Emscripten.Current.Manifest-10.0.100.Transport)
    • runtime.win-x64.Microsoft.NETCore.Runtime.Mono.LLVM.Sdk: from 19.1.0-alpha.1.24501.1 to 19.1.0-alpha.1.24503.1 (parent: Microsoft.NET.Workload.Emscripten.Current.Manifest-10.0.100.Transport)
    • runtime.win-x64.Microsoft.NETCore.Runtime.Mono.LLVM.Tools: from 19.1.0-alpha.1.24501.1 to 19.1.0-alpha.1.24503.1 (parent: Microsoft.NET.Workload.Emscripten.Current.Manifest-10.0.100.Transport)
    • runtime.osx-arm64.Microsoft.NETCore.Runtime.Mono.LLVM.Sdk: from 19.1.0-alpha.1.24501.1 to 19.1.0-alpha.1.24503.1 (parent: Microsoft.NET.Workload.Emscripten.Current.Manifest-10.0.100.Transport)
    • runtime.osx-arm64.Microsoft.NETCore.Runtime.Mono.LLVM.Tools: from 19.1.0-alpha.1.24501.1 to 19.1.0-alpha.1.24503.1 (parent: Microsoft.NET.Workload.Emscripten.Current.Manifest-10.0.100.Transport)
    • runtime.osx-x64.Microsoft.NETCore.Runtime.Mono.LLVM.Sdk: from 19.1.0-alpha.1.24501.1 to 19.1.0-alpha.1.24503.1 (parent: Microsoft.NET.Workload.Emscripten.Current.Manifest-10.0.100.Transport)
    • runtime.osx-x64.Microsoft.NETCore.Runtime.Mono.LLVM.Tools: from 19.1.0-alpha.1.24501.1 to 19.1.0-alpha.1.24503.1 (parent: Microsoft.NET.Workload.Emscripten.Current.Manifest-10.0.100.Transport)

Microsoft.DotNet.ApiCompat.Task
 From Version 10.0.100-alpha.1.24480.5 -> To Version 10.0.100-alpha.1.24507.16
…nals build 20241007.2

Microsoft.SourceBuild.Intermediate.source-build-externals
 From Version 10.0.0-alpha.1.24502.3 -> To Version 10.0.0-alpha.1.24507.2
@dotnet-maestro dotnet-maestro bot changed the title [main] Update dependencies from dotnet/sdk [main] Update dependencies from dotnet/sdk, dotnet/source-build-externals Oct 8, 2024
Microsoft.NETCore.Runtime.ICU.Transport
 From Version 10.0.0-alpha.1.24503.1 -> To Version 10.0.0-alpha.1.24507.2
@dotnet-maestro dotnet-maestro bot changed the title [main] Update dependencies from dotnet/sdk, dotnet/source-build-externals [main] Update dependencies from dotnet/icu, dotnet/sdk, dotnet/source-build-externals Oct 9, 2024
…ld 20241007.1

Microsoft.DotNet.CilStrip.Sources , System.ComponentModel.TypeConverter.TestData , System.Data.Common.TestData , System.Drawing.Common.TestData , System.Formats.Tar.TestData , System.IO.Compression.TestData , System.IO.Packaging.TestData , System.Net.TestData , System.Private.Runtime.UnicodeData , System.Runtime.Numerics.TestData , System.Runtime.TimeZoneData , System.Security.Cryptography.X509Certificates.TestData , System.Text.RegularExpressions.TestData , System.Windows.Extensions.TestData
 From Version 10.0.0-beta.24480.2 -> To Version 10.0.0-beta.24507.1
@dotnet-maestro dotnet-maestro bot changed the title [main] Update dependencies from dotnet/icu, dotnet/sdk, dotnet/source-build-externals [main] Update dependencies from dotnet/icu, dotnet/runtime-assets, dotnet/sdk, dotnet/source-build-externals Oct 10, 2024
…ild 20241007.1

Microsoft.DotNet.HotReload.Utils.Generator.BuildTool
 From Version 10.0.0-alpha.0.24480.1 -> To Version 10.0.0-alpha.0.24507.1
Microsoft.NETCore.Runtime.ICU.Transport
 From Version 10.0.0-alpha.1.24507.2 -> To Version 10.0.0-alpha.1.24510.1
…ld 20241010.1

Microsoft.DotNet.CilStrip.Sources , System.ComponentModel.TypeConverter.TestData , System.Data.Common.TestData , System.Drawing.Common.TestData , System.Formats.Tar.TestData , System.IO.Compression.TestData , System.IO.Packaging.TestData , System.Net.TestData , System.Private.Runtime.UnicodeData , System.Runtime.Numerics.TestData , System.Runtime.TimeZoneData , System.Security.Cryptography.X509Certificates.TestData , System.Text.RegularExpressions.TestData , System.Windows.Extensions.TestData
 From Version 10.0.0-beta.24507.1 -> To Version 10.0.0-beta.24510.1
…ild 20241010.1

Microsoft.DotNet.HotReload.Utils.Generator.BuildTool
 From Version 10.0.0-alpha.0.24507.1 -> To Version 10.0.0-alpha.0.24510.1
…10.1

Microsoft.SourceBuild.Intermediate.cecil , Microsoft.DotNet.Cecil
 From Version 0.11.5-alpha.24480.1 -> To Version 0.11.5-alpha.24510.1
@dotnet-maestro dotnet-maestro bot changed the title [main] Update dependencies from dotnet/hotreload-utils, dotnet/icu, dotnet/runtime-assets, dotnet/sdk, dotnet/source-build-externals [main] Update dependencies from dotnet/cecil, dotnet/hotreload-utils, dotnet/icu, dotnet/runtime-assets, dotnet/sdk, dotnet/source-build-externals Oct 11, 2024
…nals build 20241011.4

Microsoft.SourceBuild.Intermediate.source-build-externals
 From Version 10.0.0-alpha.1.24507.2 -> To Version 10.0.0-alpha.1.24511.4
@akoeplinger
Copy link
Member

@kg This could be due to changes to ilasm as these are IL-based tests, maybe related to 2b4e823?

@kg
Copy link
Member

kg commented Oct 19, 2024

@kg This could be due to changes to ilasm as these are IL-based tests, maybe related to 2b4e823?

It appears to happen when ilasm flows (I reproduced it locally), but the "corrupt" DLL files are not corrupt according to peverify, ilverify, ilspy, or ildasm. It seems like it might be a latent bug in the IL verifier in coreclr (that is only enabled in checked builds). I'm not sure why ilasm's behavior changed though, I've verified manually that none of the new behavior I added should be engaged (it's off by default).

@kg
Copy link
Member

kg commented Oct 19, 2024

Should I undo the flowing of IL SDK and ilasm for now? It's beyond me how it's broken, but it would unblock flow for now. I think the verifier is what needs to be fixed, since nothing seems to be wrong with the generated DLLs.

…1021.1

Microsoft.NET.Sdk.IL , Microsoft.NETCore.App.Runtime.win-x64 , Microsoft.NETCore.ILAsm , runtime.native.System.IO.Ports , System.Reflection.Metadata , System.Reflection.MetadataLoadContext , System.Text.Json , Microsoft.SourceBuild.Intermediate.runtime.linux-x64
 From Version 10.0.0-alpha.1.24517.1 -> To Version 10.0.0-alpha.1.24521.1
Microsoft.DotNet.ApiCompat.Task
 From Version 10.0.100-alpha.1.24480.5 -> To Version 10.0.100-alpha.1.24510.13
@kg
Copy link
Member

kg commented Oct 21, 2024

The check failure causing the mcc_i tests to fail in checked lanes is in CheckILOnlyImportByNameTable:

+ ./artifacts/tests/coreclr/linux.x64.Checked/Tests/Core_Root/corerun /home/kate/mcc_i06.dll
CHECK failure: (s_Images != __null) '' @ /home/kate/Projects/dotnet-runtime-wasm/src/coreclr/vm/peimage.cpp:59
CHECK failure: ((*(UINT32 *)(&pImportArray[1])) == 0) '' @ /home/kate/Projects/dotnet-runtime-wasm/src/coreclr/utilcode/pedecoder.cpp:1521
CHECK failure: (CheckILOnlyImportByNameTable(((pID[0]).u.OriginalFirstThunk))) '' @ /home/kate/Projects/dotnet-runtime-wasm/src/coreclr/utilcode/pedecoder.cpp:1496
CHECK failure: (CheckILOnlyImportDlls()) '' @ /home/kate/Projects/dotnet-runtime-wasm/src/coreclr/utilcode/pedecoder.cpp:1395
CHECK failure: (CheckILOnly()) '' @ /home/kate/Projects/dotnet-runtime-wasm/src/coreclr/utilcode/pedecoder.cpp:37
CHECK failure: (CheckFormat()) '' @ /home/kate/Projects/dotnet-runtime-wasm/src/coreclr/utilcode/pedecoder.cpp:91
CHECK failure: (GetOrCreateLayout(PEImageLayout::LAYOUT_ANY)->CheckILFormat()) '' @ /home/kate/Projects/dotnet-runtime-wasm/src/coreclr/vm/peimage.cpp:66
Unhandled exception. System.BadImageFormatException: Bad IL format. The format of the file '/home/kate/mcc_i06.dll' is invalid.

I'm still not certain these DLLs are actually corrupt but this at least narrows down what coreclr (in checked mode only) dislikes about them.

@kg
Copy link
Member

kg commented Oct 21, 2024

The corrupt DLL appears to have a malformed imports section somehow. Comparing a malformed ilasm output with a BCL dll using dumpbin:

PS E:\desktop> dumpbin /imports .\mcc_i06.dll
Microsoft (R) COFF/PE Dumper Version 14.41.34120.0
Copyright (C) Microsoft Corporation.  All rights reserved.
Dump of file .\mcc_i06.dll

File Type: EXECUTABLE IMAGE

  Section contains the following imports:

    mscoree.dll
                402000 Import Address Table
                40F150 Import Name Table
                     0 time date stamp
                     0 Index of first forwarder reference
(36 lines of whitespace omitted)

  Summary

        2000 .reloc
        E000 .text

PS E:\desktop> dumpbin /imports .\Microsoft.CSharp.dll
Microsoft (R) COFF/PE Dumper Version 14.41.34120.0
Copyright (C) Microsoft Corporation.  All rights reserved.
Dump of file .\Microsoft.CSharp.dll

File Type: DLL

  Section contains the following imports:

    mscoree.dll
                402000 Import Address Table
                452CA7 Import Name Table
                     0 time date stamp
                     0 Index of first forwarder reference

                    0 _CorDllMain

  Summary

        2000 .reloc
        2000 .rsrc
       52000 .text

@kg
Copy link
Member

kg commented Oct 21, 2024

ilasm is broken because when it does:

if (FAILED(hr=m_pEmitter->GetSaveSize(cssAccurate, &metaDataSize))) goto exit;

it gets a metaDataSize that is 1 byte too small, at least for mcc_i06. Then later it does this and attempts to overrun the buffer by one byte:

if (FAILED(hr=m_pCeeFileGen->EmitMetaDataAt(m_pCeeFile, m_pEmitter, m_pILSection, metaDataOffset, metaData, metaDataSize))) goto exit;

EmitMetaDataAt calls down into a function that only bounds checks with an ASSERTE, so outside of checked/debug builds, we will either silently truncate the metadata by one byte when writing it, or we will overrun a buffer. It's not obvious to me which of the two is happening, but the result is a dll with a misaligned import section.

This points to CLiteWeightStgdbRW::GetSaveSize being broken. I haven't been able to spot the problem yet.

We can unblock flow temporarily by not flowing ilasm, but ilasm (well, its dependency) is just actually busted and the next time it flows it will break again.

kg added a commit that referenced this pull request Oct 22, 2024
This should fix the System.BadImageFormatException failures in #108636 that were being caused by ilasm producing malformed DLLs in some cases.
Microsoft.NETCore.Runtime.ICU.Transport
 From Version 10.0.0-alpha.1.24514.3 -> To Version 10.0.0-alpha.1.24521.2
…21.1

Microsoft.SourceBuild.Intermediate.cecil , Microsoft.DotNet.Cecil
 From Version 0.11.5-alpha.24515.1 -> To Version 0.11.5-alpha.24521.1
…ld 20241021.1

Microsoft.DotNet.CilStrip.Sources , System.ComponentModel.TypeConverter.TestData , System.Data.Common.TestData , System.Drawing.Common.TestData , System.Formats.Tar.TestData , System.IO.Compression.TestData , System.IO.Packaging.TestData , System.Net.TestData , System.Private.Runtime.UnicodeData , System.Runtime.Numerics.TestData , System.Runtime.TimeZoneData , System.Security.Cryptography.X509Certificates.TestData , System.Text.RegularExpressions.TestData , System.Windows.Extensions.TestData
 From Version 10.0.0-beta.24514.1 -> To Version 10.0.0-beta.24521.1
…nals build 20241021.1

Microsoft.SourceBuild.Intermediate.source-build-externals
 From Version 10.0.0-alpha.1.24516.1 -> To Version 10.0.0-alpha.1.24521.1
…1021.2

Microsoft.NET.Sdk.IL , Microsoft.NETCore.App.Runtime.win-x64 , Microsoft.NETCore.ILAsm , runtime.native.System.IO.Ports , System.Reflection.Metadata , System.Reflection.MetadataLoadContext , System.Text.Json , Microsoft.SourceBuild.Intermediate.runtime.linux-x64
 From Version 10.0.0-alpha.1.24521.1 -> To Version 10.0.0-alpha.1.24521.2
dotnet-maestro bot and others added 2 commits October 22, 2024 17:48
…1022.5

Microsoft.NET.Sdk.IL , Microsoft.NETCore.App.Runtime.win-x64 , Microsoft.NETCore.ILAsm , runtime.native.System.IO.Ports , System.Reflection.Metadata , System.Reflection.MetadataLoadContext , System.Text.Json , Microsoft.SourceBuild.Intermediate.runtime.linux-x64
 From Version 10.0.0-alpha.1.24521.2 -> To Version 10.0.0-alpha.1.24522.5
@akoeplinger
Copy link
Member

@kg looks like your fix in #109086 worked, we don't see the issue anymore with the new flow :)

@akoeplinger akoeplinger merged commit e16be41 into main Oct 23, 2024
157 of 164 checks passed
@akoeplinger akoeplinger deleted the darc-main-1ab3acfe-5f2c-4ca0-9919-7d14fa548ee9 branch October 23, 2024 10:16
@github-actions github-actions bot locked and limited conversation to collaborators Nov 23, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

area-codeflow for labeling automated codeflow

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants