Skip to content

Commit d6ca550

Browse files
lambdageeklambdageek
authored
[cdac] CodeVersions contract and tests (#109021)
* [cdac] Add a CodeVersions contract The CodeVersions contract implements the IL and native code versioning as described in [code-versioning.md](docs/design/features/code-versioning.md) Contributes to #108553 Contributes to #99302 * rename contract NativeCodePointers => CodeVersions * FindActiveILCodeVersion * implement GetModuleLookupMapElement * FindActiveILCodeVersion/FindActiveNativeCodeVersion * il code version lookup table * remove AppDomain.CodeVersionManager from cdac * CodeVersionManager is basically a static class in the C++ side * NativeCodeVersionContract.GetSpecificNativeCodeVersion * checkpoint: start adding NativeCodeVersion operations * WIP: native code version * tighter cdac_data friends * remove unused DataType values they'll come in from a future PR * move ecma metadata helpers to their own class * WIP: CodeVersionsTests * WIP: TestGetNativeCodeVersionOneVersionVersionable * implement GetNativeCode for a NativeCodeVersionNode handle * checkpoint: TestGetNativeCodeVersionOneVersionVersionable passes * Add TestGetActiveNativeCodeVersionDefaultCase * don't add ILCodeVersioningState::Next field yet * describe Loader::GetModuleLookupMapElement contract * remove unused member * update contract markdown * simplify flags handling a bit * add TODOs for RuntimeTypeSystem additions
1 parent be5bbca commit d6ca550

File tree

22 files changed

+1338
-4
lines changed

22 files changed

+1338
-4
lines changed

‎docs/design/datacontracts/CodeVersions.md‎

Lines changed: 256 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,256 @@
1+
# Contract CodeVersions
2+
3+
This contract encapsulates support for [code versioning](../features/code-versioning.md) in the runtime.
4+
5+
## APIs of contract
6+
7+
```csharp
8+
internal struct NativeCodeVersionHandle
9+
{
10+
// no public constructors
11+
internal readonly TargetPointer MethodDescAddress;
12+
internal readonly TargetPointer CodeVersionNodeAddress;
13+
internal NativeCodeVersionHandle(TargetPointer methodDescAddress, TargetPointer codeVersionNodeAddress)
14+
{
15+
if (methodDescAddress != TargetPointer.Null && codeVersionNodeAddress != TargetPointer.Null)
16+
{
17+
throw new ArgumentException("Only one of methodDescAddress and codeVersionNodeAddress can be non-null");
18+
}
19+
MethodDescAddress = methodDescAddress;
20+
CodeVersionNodeAddress = codeVersionNodeAddress;
21+
}
22+
23+
internal static NativeCodeVersionHandle Invalid => new(TargetPointer.Null, TargetPointer.Null);
24+
public bool Valid => MethodDescAddress != TargetPointer.Null || CodeVersionNodeAddress != TargetPointer.Null;
25+
}
26+
```
27+
28+
```csharp
29+
// Return a handle to the version of the native code that includes the given instruction pointer
30+
public virtual NativeCodeVersionHandle GetNativeCodeVersionForIP(TargetCodePointer ip);
31+
// Return a handle to the active version of the native code for a given method descriptor
32+
public virtual NativeCodeVersionHandle GetActiveNativeCodeVersion(TargetPointer methodDesc);
33+
34+
// returns true if the given method descriptor supports multiple code versions
35+
public virtual bool CodeVersionManagerSupportsMethod(TargetPointer methodDesc);
36+
37+
// Return the instruction pointer corresponding to the start of the given native code version
38+
public virtual TargetCodePointer GetNativeCode(NativeCodeVersionHandle codeVersionHandle);
39+
```
40+
41+
## Version 1
42+
43+
See [code versioning](../features/code-versioning.md) for a general overview and the definitions of *synthetic* and *explicit* nodes.
44+
45+
Data descriptors used:
46+
| Data Descriptor Name | Field | Meaning |
47+
| --- | --- | --- |
48+
| MethodDescVersioningState | Flags | `MethodDescVersioningStateFlags` flags, see below |
49+
| MethodDescVersioningState | NativeCodeVersionNode | code version node of this method desc, if active |
50+
| NativeCodeVersionNode | Next | pointer to the next native code version |
51+
| NativeCodeVersionNode | MethodDesc | indicates a synthetic native code version node |
52+
| NativeCodeVersionNode | NativeCode | indicates an explicit native code version node |
53+
| ILCodeVersioningState | ActiveVersionKind | an `ILCodeVersionKind` value indicating which fields of the active version are value |
54+
| ILCodeVersioningState | ActiveVersionNode | if the active version is explicit, the NativeCodeVersionNode for the active version |
55+
| ILCodeVersioningState | ActiveVersionModule | if the active version is synthetic or unknown, the pointer to the Module that defines the method |
56+
| ILCodeVersioningState | ActiveVersionMethodDef | if the active version is synthetic or unknown, the MethodDef token for the method |
57+
58+
The flag indicates that the default version of the code for a method desc is active:
59+
```csharp
60+
internal enum MethodDescVersioningStateFlags : byte
61+
{
62+
IsDefaultVersionActiveChildFlag = 0x4
63+
};
64+
```
65+
66+
The value of the `ILCodeVersioningState::ActiveVersionKind` field is one of:
67+
```csharp
68+
private enum ILCodeVersionKind
69+
{
70+
Unknown = 0,
71+
Explicit = 1, // means Node is set
72+
Synthetic = 2, // means Module and Token are set
73+
}
74+
```
75+
76+
Global variables used: *none*
77+
78+
Contracts used:
79+
| Contract Name |
80+
| --- |
81+
| ExecutionManager |
82+
| Loader |
83+
| RuntimeTypeSystem |
84+
85+
### Finding the start of a specific native code version
86+
87+
```csharp
88+
NativeCodeVersionHandle ICodeVersions.GetNativeCodeVersionForIP(TargetCodePointer ip)
89+
{
90+
Contracts.IExecutionManager executionManager = _target.Contracts.ExecutionManager;
91+
EECodeInfoHandle? info = executionManager.GetEECodeInfoHandle(ip);
92+
if (!info.HasValue)
93+
{
94+
return NativeCodeVersionHandle.Invalid;
95+
}
96+
TargetPointer methodDescAddress = executionManager.GetMethodDesc(info.Value);
97+
if (methodDescAddress == TargetPointer.Null)
98+
{
99+
return NativeCodeVersionHandle.Invalid;
100+
}
101+
IRuntimeTypeSystem rts = _target.Contracts.RuntimeTypeSystem;
102+
MethodDescHandle md = rts.GetMethodDescHandle(methodDescAddress);
103+
if (!rts.IsVersionable(md))
104+
{
105+
return new NativeCodeVersionHandle(methodDescAddress, codeVersionNodeAddress: TargetPointer.Null);
106+
}
107+
else
108+
{
109+
TargetCodePointer startAddress = executionManager.GetStartAddress(info.Value);
110+
return GetSpecificNativeCodeVersion(md, startAddress);
111+
}
112+
}
113+
114+
NativeCodeVersionHandle GetSpecificNativeCodeVersion(MethodDescHandle md, TargetCodePointer startAddress)
115+
{
116+
TargetPointer methodDescVersioningStateAddress = target.Contracts.RuntimeTypeSystem.GetMethodDescVersioningState(md);
117+
if (methodDescVersioningStateAddress == TargetPointer.Null)
118+
{
119+
return NativeCodeVersionHandle.Invalid;
120+
}
121+
Data.MethodDescVersioningState methodDescVersioningStateData = _target.ProcessedData.GetOrAdd<Data.MethodDescVersioningState>(methodDescVersioningStateAddress);
122+
return FindFirstCodeVersion(methodDescVersioningStateData, (codeVersion) =>
123+
{
124+
return codeVersion.MethodDesc == md.Address && codeVersion.NativeCode == startAddress;
125+
});
126+
}
127+
128+
NativeCodeVersionHandle FindFirstCodeVersion(Data.MethodDescVersioningState versioningState, Func<Data.NativeCodeVersionNode, bool> predicate)
129+
{
130+
TargetPointer currentAddress = versioningState.NativeCodeVersionNode;
131+
while (currentAddress != TargetPointer.Null)
132+
{
133+
Data.NativeCodeVersionNode current = _target.ProcessedData.GetOrAdd<Data.NativeCodeVersionNode>(currentAddress);
134+
if (predicate(current))
135+
{
136+
return new NativeCodeVersionHandle(methodDescAddress: TargetPointer.Null, currentAddress);
137+
}
138+
currentAddress = current.Next;
139+
}
140+
return NativeCodeVersionHandle.Invalid;
141+
}
142+
```
143+
144+
### Finding the active native code version of a method descriptor
145+
146+
```csharp
147+
NativeCodeVersionHandle ICodeVersions.GetActiveNativeCodeVersion(TargetPointer methodDesc)
148+
{
149+
IRuntimeTypeSystem rts = _target.Contracts.RuntimeTypeSystem;
150+
MethodDescHandle md = rts.GetMethodDescHandle(methodDesc);
151+
TargetPointer mtAddr = rts.GetMethodTable(md);
152+
TypeHandle typeHandle = rts.GetTypeHandle(mtAddr);
153+
TargetPointer module = rts.GetModule(typeHandle);
154+
uint methodDefToken = rts.GetMethodToken(md);
155+
ILCodeVersionHandle methodDefActiveVersion = FindActiveILCodeVersion(module, methodDefToken);
156+
if (!methodDefActiveVersion.IsValid)
157+
{
158+
return NativeCodeVersionHandle.Invalid;
159+
}
160+
return FindActiveNativeCodeVersion(methodDefActiveVersion, methodDesc);
161+
}
162+
163+
ILCodeVersionHandle ILCodeVersionHandleFromState(Data.ILCodeVersioningState ilState)
164+
{
165+
switch ((ILCodeVersionKind)ilState.ActiveVersionKind)
166+
{
167+
case ILCodeVersionKind.Explicit:
168+
return new ILCodeVersionHandle(module: TargetPointer.Null, methodDef: 0, ilState.ActiveVersionNode);
169+
case ILCodeVersionKind.Synthetic:
170+
case ILCodeVersionKind.Unknown:
171+
return new ILCodeVersionHandle(ilState.ActiveVersionModule, ilState.ActiveVersionMethodDef, TargetPointer.Null);
172+
default:
173+
throw new InvalidOperationException($"Unknown ILCodeVersionKind {ilState.ActiveVersionKind}");
174+
}
175+
}
176+
177+
ILCodeVersionHandle FindActiveILCodeVersion(TargetPointer module, uint methodDefinition)
178+
{
179+
ModuleHandle moduleHandle = _target.Contracts.Loader.GetModuleHandle(module);
180+
TargetPointer ilCodeVersionTable = _target.Contracts.Loader.GetLookupTables(moduleHandle).MethodDefToILCodeVersioningState;
181+
TargetPointer ilVersionStateAddress = _target.Contracts.Loader.GetModuleLookupMapElement(ilCodeVersionTable, methodDefinition, out var _);
182+
if (ilVersionStateAddress == TargetPointer.Null)
183+
{
184+
return new ILCodeVersionHandle(module, methodDefinition, TargetPointer.Null);
185+
}
186+
Data.ILCodeVersioningState ilState = _target.ProcessedData.GetOrAdd<Data.ILCodeVersioningState>(ilVersionStateAddress);
187+
return ILCodeVersionHandleFromState(ilState);
188+
}
189+
190+
bool IsActiveNativeCodeVersion(NativeCodeVersionHandle nativeCodeVersion)
191+
{
192+
if (nativeCodeVersion.MethodDescAddress != TargetPointer.Null)
193+
{
194+
MethodDescHandle md = _target.Contracts.RuntimeTypeSystem.GetMethodDescHandle(nativeCodeVersion.MethodDescAddress);
195+
TargetPointer versioningStateAddress = _target.Contracts.RuntimeTypeSystem.GetMethodDescVersioningState(md);
196+
if (versioningStateAddress == TargetPointer.Null)
197+
{
198+
return true;
199+
}
200+
Data.MethodDescVersioningState versioningState = _target.ProcessedData.GetOrAdd<Data.MethodDescVersioningState>(versioningStateAddress);
201+
MethodDescVersioningStateFlags flags = (MethodDescVersioningStateFlags)versioningState.Flags;
202+
return flags.HasFlag(MethodDescVersioningStateFlags.IsDefaultVersionActiveChildFlag);
203+
}
204+
else if (nativeCodeVersion.CodeVersionNodeAddress != TargetPointer.Null)
205+
{
206+
throw new NotImplementedException(); // TODO[cdac]: IsActiveNativeCodeVersion - explicit
207+
}
208+
else
209+
{
210+
throw new ArgumentException("Invalid NativeCodeVersionHandle");
211+
}
212+
}
213+
214+
NativeCodeVersionHandle FindActiveNativeCodeVersion(ILCodeVersionHandle methodDefActiveVersion, TargetPointer methodDescAddress)
215+
{
216+
if (methodDefActiveVersion.Module != TargetPointer.Null)
217+
{
218+
NativeCodeVersionHandle provisionalHandle = new NativeCodeVersionHandle(methodDescAddress: methodDescAddress, codeVersionNodeAddress: TargetPointer.Null);
219+
if (IsActiveNativeCodeVersion(provisionalHandle))
220+
{
221+
return provisionalHandle;
222+
}
223+
else
224+
{
225+
throw new NotImplementedException(); // TODO[cdac]: iterate through versioning state nodes
226+
}
227+
}
228+
else
229+
{
230+
throw new NotImplementedException(); // TODO: [cdac] find explicit il code version
231+
}
232+
}
233+
```
234+
235+
### Determining whether a method descriptor supports code versioning
236+
237+
```csharp
238+
bool ICodeVersions.CodeVersionManagerSupportsMethod(TargetPointer methodDescAddress)
239+
{
240+
IRuntimeTypeSystem rts = _target.Contracts.RuntimeTypeSystem;
241+
MethodDescHandle md = rts.GetMethodDescHandle(methodDescAddress);
242+
if (rts.IsDynamicMethod(md))
243+
return false;
244+
if (rts.IsCollectibleMethod(md))
245+
return false;
246+
TargetPointer mtAddr = rts.GetMethodTable(md);
247+
TypeHandle mt = rts.GetTypeHandle(mtAddr);
248+
TargetPointer modAddr = rts.GetModule(mt);
249+
ILoader loader = _target.Contracts.Loader;
250+
ModuleHandle mod = loader.GetModuleHandle(modAddr);
251+
ModuleFlags modFlags = loader.GetFlags(mod);
252+
if (modFlags.HasFlag(ModuleFlags.EditAndContinue))
253+
return false;
254+
return true;
255+
}
256+
```

‎docs/design/datacontracts/Loader.md‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,7 @@ TargetPointer GetLoaderAllocator(ModuleHandle handle);
3838
TargetPointer GetThunkHeap(ModuleHandle handle);
3939
TargetPointer GetILBase(ModuleHandle handle);
4040
ModuleLookupTables GetLookupTables(ModuleHandle handle);
41+
TargetPointer GetModuleLookupMapElement(TargetPointer table, uint token, out TargetNUInt flags);
4142
```
4243

4344
## Version 1
@@ -58,6 +59,9 @@ Data descriptors used:
5859
| `Module` | `TypeDefToMethodTableMap` | Mapping table |
5960
| `Module` | `TypeRefToMethodTableMap` | Mapping table |
6061
| `ModuleLookupMap` | `TableData` | Start of the mapping table's data |
62+
| `ModuleLookupMap` | `SupportedFlagsMask` | Mask for flag bits on lookup map entries |
63+
| `ModuleLookupMap` | `Count` | Number of TargetPointer sized entries in this section of the map |
64+
| `ModuleLookupMap` | `Next` | Pointer to next ModuleLookupMap segment for this map
6165

6266
``` csharp
6367
ModuleHandle GetModuleHandle(TargetPointer modulePointer)
@@ -109,4 +113,32 @@ ModuleLookupTables GetLookupTables(ModuleHandle handle)
109113
MethodDefToILCodeVersioningState: target.ReadPointer(handle.Address + /*
110114
Module::MethodDefToILCodeVersioningState */));
111115
}
116+
117+
TargetPointer GetModuleLookupMapElement(TargetPointer table, uint token, out TargetNUInt flags);
118+
{
119+
uint rid = /* get row id from token*/ (token);
120+
flags = new TargetNUInt(0);
121+
if (table == TargetPointer.Null)
122+
return TargetPointer.Null;
123+
uint index = rid;
124+
// have to read lookupMap an extra time upfront because only the first map
125+
// has valid supportedFlagsMask
126+
TargetNUInt supportedFlagsMask = _target.ReadNUInt(table + /* ModuleLookupMap::SupportedFlagsMask */);
127+
do
128+
{
129+
if (index < _target.Read<uint>(table + /*ModuleLookupMap::Count*/))
130+
{
131+
TargetPointer entryAddress = _target.ReadPointer(lookupMap + /*ModuleLookupMap::TableData*/) + (ulong)(index * _target.PointerSize);
132+
TargetPointer rawValue = _target.ReadPointer(entryAddress);
133+
flags = rawValue & supportedFlagsMask;
134+
return rawValue & ~(supportedFlagsMask.Value);
135+
}
136+
else
137+
{
138+
table = _target.ReadPointer(lookupMap + /*ModuleLookupMap::Next*/);
139+
index -= _target.Read<uint>(lookupMap + /*ModuleLookupMap::Count*/);
140+
}
141+
} while (table != TargetPointer.Null);
142+
return TargetPointer.Null;
143+
}
112144
```

‎docs/design/datacontracts/RuntimeTypeSystem.md‎

Lines changed: 45 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -138,6 +138,19 @@ partial interface IRuntimeTypeSystem : IContract
138138
// Return true if a MethodDesc represents an IL Stub dynamically generated by the runtime
139139
// A IL Stub method is also a StoredSigMethodDesc, and a NoMetadataMethod
140140
public virtual bool IsILStub(MethodDescHandle methodDesc);
141+
142+
// Return true if a MethodDesc is in a collectible module
143+
public virtual bool IsCollectibleMethod(MethodDescHandle methodDesc);
144+
145+
// Return true if a MethodDesc supports mulitiple code versions
146+
public virtual bool IsVersionable(MethodDescHandle methodDesc);
147+
148+
// Return a pointer to the IL versioning state of the MethodDesc
149+
public virtual TargetPointer GetMethodDescVersioningState(MethodDescHandle methodDesc);
150+
151+
// Get an instruction pointer that can be called to cause the MethodDesc to be executed
152+
public virtual TargetCodePointer GetNativeCode(MethodDescHandle methodDesc);
153+
141154
}
142155
```
143156

@@ -607,6 +620,7 @@ The version 1 `MethodDesc` APIs depend on the `MethodDescAlignment` global and t
607620
| `MethodDescAlignment` | `MethodDescChunk` trailing data is allocated in multiples of this constant. The size (in bytes) of each `MethodDesc` (or subclass) instance is a multiple of this constant. |
608621
| `MethodDescTokenRemainderBitCount` | Number of bits in the token remainder in `MethodDesc` |
609622

623+
**TODO** MethodDesc code pointers additions
610624

611625
In the runtime a `MethodDesc` implicitly belongs to a single `MethodDescChunk` and some common data is shared between method descriptors that belong to the same chunk. A single method table
612626
will typically have multiple chunks. There are subkinds of MethodDescs at runtime of varying sizes (but the sizes must be mutliples of `MethodDescAlignment`) and each chunk contains method descriptors of the same size.
@@ -631,6 +645,15 @@ We depend on the following data descriptors:
631645
| `StoredSigMethodDesc` | `ExtendedFlags` | Flags field for the `StoredSigMethodDesc` |
632646
| `DynamicMethodDesc` | `MethodName` | Pointer to Null-terminated UTF8 string describing the Method desc |
633647

648+
**TODO** MethodDesc code pointers additions
649+
650+
The contract depends on the following other contracts
651+
652+
| Contract |
653+
| --- |
654+
| Loader |
655+
| ReJIT |
656+
| CodeVersions |
634657

635658
And the following enumeration definitions
636659

@@ -821,4 +844,25 @@ And the various apis are implemented with the following algorithms
821844
return ((DynamicMethodDescExtendedFlags)ExtendedFlags).HasFlag(DynamicMethodDescExtendedFlags.IsILStub);
822845
}
823846
```
824-
**TODO(cdac)**
847+
848+
Determining if a method is in a collectible module:
849+
850+
```csharp
851+
bool IRuntimeTypeSystem.IsCollectibleMethod(MethodDescHandle methodDesc) => // TODO[cdac]: finish this
852+
```
853+
854+
Determining if a method supports multiple code versions:
855+
856+
```csharp
857+
bool IRuntimeTypeSystem.IsVersionable(MethodDescHandle methodDesc) => // TODO[cdac]: finish this
858+
```
859+
860+
Extracting a pointer to the `MethodDescVersioningState` data for a given method
861+
```csharp
862+
TargetPointer IRuntimeTypeSystem.GetMethodDescVersioningState(MethodDescHandle methodDesc) => // TODO[cdac]: finish this
863+
```
864+
865+
Getting the native code pointer for methods with a NativeCodeSlot or a stable entry point
866+
```csharp
867+
public virtual TargetCodePointer GetNativeCode(MethodDescHandle methodDesc) => // TODO[cdac]: finish this
868+
```

‎src/coreclr/debug/runtimeinfo/contracts.jsonc‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@
99
// cdac-build-tool can take multiple "-c contract_file" arguments
1010
// so to conditionally include contracts, put additional contracts in a separate file
1111
{
12+
"CodeVersions": 1,
1213
"DacStreams": 1,
1314
"EcmaMetadata" : 1,
1415
"Exception": 1,

0 commit comments

Comments
 (0)