Proposal: Add new GetValue method to System.Reflection

[Esidar]

This new method would accept an "object" to which a Value could be copied instead of being unnecessarily boxed. Example of use would look like this:

static object boxedFloat = null;
....
boxedFloat = fieldInfo.GetValue( myObject, boxedFloat );

Boxing of the value would occur only once. This would eliminate unnecessarily memory allocations in situations when GetValue is called many times. When method is used on a class types, copying would not occur and stored object would be returned instead.

One of the uses of GetValue method is User Interface based on MVVM (Model View Viewmodel) where values used to control an UI are pulled directly from the data. In realtime applications (like games) this boxing makes hundreds if not thousands of unnecessary memory allocations every second.

Another use of GetValue is serialization, and having ability to serialize hundreds of values without the need of boxing/unboxing, would greatly improve memory usage and performace.

There was also an idea https://github.com/dotnet/coreclr/issues/8277 that this new GetValue method would take a form of generic method, but I think this would limit the use as one of the advantages of working with System.Object is this abstraction that makes some parts of the code simpler.

So, ideally this new method should remove the unnecessary boxing while keeping convenience of System.Object. Additionally please bear in mind this should work with both FieldInfo and PropertyInfo, as well as give option to get values from nested Value as in following example:

struct Vector3
{
   float x,y,z;
}
struct Transform
{
    Vector3 position { get; set }
}

transform = positionProperty.GetValue( transformObject, transform );
boxedX = xField.GetValue( transform, boxedX );
DisplayValue( boxedX );

(note intermixing fields with property)

[svick]

Interesting, I didn't realize you can reassign values inside boxes.

If this is going to work with properties, I think it should be made completely general and work with any method that returns a value.

And a couple questions:

1. What should the behavior be if the passed-in boxed value is of the wrong type? The same as null, i.e. allocate a new box?
2. Wouldn't it be simpler if the input and output were specified at the same time using a ref parameter, i.e.: fieldInfo.GetValue(myObject, ref boxedFloat);? No need to specify boxedFloat twice.

[jkotas]

FieldInfo.GetValue/SetValue are part of reflection invoke APIs.

All current reflection invoke APIs have a general problem that they expected everything boxed. It is not just a performance problem mentioned here, but also a functional deficiency problem.

Among other things, it means that it is not possible to write high fidelity IL interpreter using reflection. Let's say that you would like to interpret static void m1(ref int x) { m2(ref x); }. There is no way to do it using reflection today because of there is no reflection invoke method that can pass x along as byref.

If we were to add a new reflection invoke APIs, we should aim to solve both of these problems.

CoreRT has internal Internal.Runtime.CallInterceptor.MakeDynamicCall that is basically a box-free version of the MemberInfo.Invoke. It would need to be cleaned up and argument validation added for it to become good public API.

[MichalStrehovsky]

There is no way to do it using reflection today because of there is no reflection invoke method that can pass x along as byref

MethodBase.Invoke will actually modify the elements of the parameters array if one of the parameters is a byref (a ref or out parameter). A better example would be a method that returns a byref (this will be exposed as a new language feature in C# soon). There's no way to reflection-invoke those.

Another thing to consider would be finding a replacement for FieldInfo.GetValueDirect and FieldInfo.SetValueDirect. These APIs let you access fields on a value type without boxing the struct that owns the field. Since TypedReference never was a first class citizen (and won't be part of .NET Core) a replacement API would be a nice addition.

[bartdesmet]

MethodBase.Invoke will actually modify the elements of the parameters array if one of the parameters is a byref (a ref or out parameter).

While that's true, it's rarely the case that the storage locations of the objects or values being passed by reference are truly in the object[] being passed. Typically one copies the value into the object[] and performs a write-back after the Invoke method returns. This causes issues such as https://github.com/dotnet/corefx/issues/4411 and https://github.com/dotnet/corefx/issues/13458.

The expression interpreter runs into a number of other issues as well. For example, we can't pass Type.Missing in a verbatim manner, see https://github.com/dotnet/corefx/issues/4112. Also, the object[] allocation and the copy made by the reflection API is expensive, see https://github.com/dotnet/corefx/issues/4125. An issue to discuss addressing just these two limitations is opened at https://github.com/dotnet/coreclr/issues/7253.

Having boxes passed to reflection APIs has also led to behavioral changes in the expression interpreter compared to the compiler, such as https://github.com/dotnet/corefx/issues/13007 (though one can argue the interpreter is doing the right thing in this case, merely by accident).

[Esidar]

Looking through available features (to avoid major changes in FX/CLR) I've stumbled upon arglist. C# have this undocumented keyword that is supported by CLR which allow for passing varying number of arguments and the use looks like this:

void Foo( __arglist )
{
   var it = new ArgIterator( __arglist );
}
void Test()
{
    float myFloat = 1;
    Foo( __arglist( 10, ref myFloat ) );
}

As you can see, you can even pass float as a ref.

From what I can see by disassembling a code (x86-64), values are passed clean, without any boxing.

New ReflectionAPI could utilize this to have both: box free GetValue as in fieldInfo.GetValue( obj, ref myFloat ) and a pretty neat, box free way of calling methods fooMethodInfo.Invoke( 10, ref myFloat )

Of course there would be a need to make a syntax nicer than a keyword __arglist, also, I'm not sure how this API would work with other languages. There is also a limit on how __arglist can be used. Currently ArgIterator, to read a value/argument, returns TypedReference and the only way to retrieve value is via TypedReference.ToObject() which does boxing. So that limits the use in managed environment. It is not a problem in native environment because GetValue and Invoke (both are implemented in native code), can read values and pointers directly from ArgIterator without boxing.

[Esidar]

Having boxes passed to reflection APIs

Problem shown in #19094 cannot be fixed by removing boxing. Boxing ensures that value types are valid during delegate invocation. In given example, variable o is valid only when delegate is created. Actual call to function f can occur much later when variable o is no longer available, thus copy must be made.

[MichalStrehovsky]

TypedReference, ArgIterator and RuntimeArgumentHandle are a no-go. None of those types are included in .NET Core or .NET Standard - they only exist in the legacy runtimes and AFAIK there are no plans to bring them back. I don't think we really need them. MakeDynamicCall mentioned above can get by without them, for example.

[jkotas]

MakeDynamicCall mentioned above can get by without them

The LocalVariableSet type used by MakeDynamicCall is very close to TypedReference. It is basically array of TypedReferences. It may be interesting to bring TypedReference back and making it useful, instead of creating a new same thing under a different name.

[Esidar]

Is there a specific reason why TypedReference is being avoided? I guess there can be stability issue (reference to a local that is no longer available), but is there anything else?

Another solution would be use of new Tuples feature. methodInfo.Invoke could accept tuple struct instead of object[] array. CoreCR/VM could then use the same tuple if invoked method has any ref/out arguments

var args = ( name: "Mike", value: 10.0f );
methodInfo.Invoke( obj, args );
Console.Write( args.value );

I don't know the implementation details of the tuples, but afaik tuple is a struct that still needs to be boxed but we're boxing one struct instead of N-arguments and N-out-results. This wouldn't solve most of the problems, but GetValue could also use tuple to return a value (again, tuple is boxed only once and can be reused), could solve problems with properly resolving Missing arguments or unboxing values in Closures and so on.

[jkotas]

Is there a specific reason why TypedReference is being avoided?

TypedReference has not been very useful: C# does not have a first class support for it, it requires non-trivial custom logic in the codegenerator and runtime, not all .NET runtimes support it. It is missing some methods that would make it useful and easy to work with, e.g. it should have a method to cast the value to strong typed value, like ref T As<T>() - this method is equivalent to the undocumented __refvalue C# keyword.

The non-boxing reflection method to get the value can then be: TypedReference GetValueReference(). To get the field value without boxing, one would write: fieldInfo.GetValueReference(myObject).As<float>().

new Tuples feature

I do not think Tuples are a good fit for this.

[jkotas]

I have opened dotnet/corefx#14088 to bring back TypedReference that should be the first on this.

cc @davidwrighton

[Thaina]

I think it should support return ref on FieldInfo directly, without TypedReference

I think TypedReference add more complexity to implementation. It make reference that would need to be fixed can be passed around. While return ref are compatible with dotnet behaviour

GameObject obj;

FieldInfo fi = typeof(obj).GetField("Transform");
ref Matrix4x4 transform = fi.GetRefValue<Matrix4x4>(ref obj);
// ref transform live in this stack
transform.M43 = 1; // obj.Transform.M43 = 1

Also, GetMethod should be able to use by cast it into generic delegate that can be used with ref object

Matrix4x4 matrix;

MethodInfo mi = typeof(matrix).GetMethod("RotateY");
RefAction<Matrix4x4,float> rotate = mi.GetRefAction<Matrix4x4,float>();
rotate(ref matrix,90); // Internally call matrix.RotateY(90);

////

namespace System
{
    public delegate void RefAction<T>(ref T obj);
    public delegate void RefAction<T,P1>(ref T obj,P1 param1);
    public delegate void RefAction<T,P1...>(ref T obj,P1 param1...);

    public delegate R RefFunc<T,R>(ref T obj);
    public delegate R RefFunc<T,P1>(ref T obj,P1 param1);
    public delegate R RefFunc<T,P1...>(ref T obj,P1 param1...);

    public delegate ref R RefFuncRef<T,R>(ref T obj);
    public delegate ref R RefFuncRef<T,P1>(ref T obj,P1 param1);
    public delegate ref R RefFuncRef<T,P1...>(ref T obj,P1 param1...);
}

class System.Reflection.MethodInfo
{
    public RefAction<T> GetRefAction<T,P>(P param) where T : class
    {
        // return void action(ref T t,param) that wrap void t.action(param);
    }
}

[jkotas]

The implementation that returns TypedReference is more efficient and more powerful.

It is more efficient because of it does not suffer from generic code bloat.

It is more powerful because of you can easily add a strong typed generic convenience wrapper extension method around it that just returns ref (this wrapper can be part of the API proposal too); but the other direction is not possible.

[jkotas]

Also, GetMethod should be able to use by cast it into generic delegate that can be used with ref object

There are existing methods that do that like Delegate.CreateDelegate(Type type, Object firstArgument, MethodInfo method). If you wish to have more convenient wrappers around them, it would be best to split it into a separate API proposal issue.

[Thaina]

@jkotas You are right that it more powerful, which, at the same time, overkill. It not safe to pass TypedReference around. And TypedReference is hard to have it implement everywhere while return ref is more promising to be standard

as you say

it requires non-trivial custom logic in the codegenerator and runtime, not all .NET runtimes support it.

So I want to trade off here. We can live with generic in the most case