Test failure: System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.RevokeEndEntity_PolicyErrors_NotTimeValid(policyErrors: True, notTimeValid: True)

[v-haren]

failed in job: runtime-libraries outerloop 20200702.2

Error message

Assert.Equal() Failure
Expected: X509ChainStatusFlags[] [NotValidForUsage, NotValidForUsage, NotTimeValid | Revoked | NotValidForUsage]
Actual: X509ChainStatusFlags[] [NoError, NoError, NotTimeValid | Revoked]


Stack trace
   at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.AssertChainStatus(X509Chain chain, X509ChainStatusFlags rootStatus, X509ChainStatusFlags issrStatus, X509ChainStatusFlags leafStatus) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 1247
   at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.<>c__DisplayClass17_0.<RevokeEndEntity_PolicyErrors_NotTimeValid>b__0(CertificateAuthority root, CertificateAuthority intermediate, X509Certificate2 endEntity, ChainHolder holder) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 602
   at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.SimpleTest(PkiOptions pkiOptions, RunSimpleTest callback, String callerName, Boolean pkiOptionsInTestName) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 1218
   at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.RevokeEndEntity_PolicyErrors_NotTimeValid(Boolean policyErrors, Boolean notTimeValid) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 564

[BruceForstall]

net5.0-Linux-Release-x64-CoreCLR_release-(Fedora.32.Amd64.Open)ubuntu.1604.amd64.open@mcr.microsoft.com/dotnet-buildtools/prereqs:fedora-32-helix-20200512010618-efb9f14

Tagging subscribers to this area: @bartonjs, @vcsjones, @krwq
Notify danmosemsft if you want to be subscribed.

[ManickaP]

Fails since 2.7.2020.
These 4 tests are failing:

Type	Method	Arguments
System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests	RevokeEndEntity_PolicyErrors_NotTimeValid	policyErrors: True, notTimeValid: False
System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests	RevokeEndEntity_PolicyErrors_NotTimeValid	policyErrors: True, notTimeValid: True
System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests	RevokeIntermediate_PolicyErrors_NotTimeValid	policyErrors: True, notTimeValid: False
System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests	RevokeIntermediate_PolicyErrors_NotTimeValid	policyErrors: True, notTimeValid: True

Reproduces locally.
AzDO fail, e.g.: https://dev.azure.com/dnceng/public/_build/results?buildId=715844&view=ms.vss-test-web.build-test-results-tab&runId=22182572&resultId=104464&paneView=debug

When reverted #38182, the tests pass locally.
cc: @wfurt

[bartonjs]

Ah, I had forgotten these tests were marked OuterLoop. I had a feeling that something was ensuring no TLS-Server EKU was present, but since the tests were passing I shrugged it off. We can just change this test to look for something else, like codesign.