[Java.Interop] Java native method registration

The Java Native Interface Functions [0] mostly covers calling Java
code *from* non-Java code, e.g. JNIEnv::CallVoidMethod() can be used
to call all Java instance methods which have a `void` return type.

What about the calling non-Java code from Java? That can't be *fully*
done through the JNIEnv type; it also requires cooperation with Java
bytecode, though the use of `native` method declarations [1]:

	// Java
	class Example {
		public native void m ();
	}

Before Java code can invoke the Example.m() instance method, that
method must first be *registered* via JNIEnv::RegisterNatives() [2].

What we have, then, is a two step process:

 1. Write/generate/otherwise obtain Java bytecode with `native` method
    declarations.

 2. At runtime, "somehow" ensure that the native methods declared in
    (1) are *registered* before they are first used from Java.

To date, JNIEnv::RegisterNatives() has been used in numerous places in
Java.Interop via JniType.RegisterNativeMethods() -- e.g. the
JavaProxyObject static constructor -- but the current architecture
doesn't support (2), as there is no way to *ensure* that e.g.
JavaProxyObject registers its native methods before Java invokes them.
(Normal use would preclude that, but if someone uses Reflection to
create a JavaProxyObject instance...)

How do we ensure that "something" -- native method registration --
happens before anything important is done with the Java type? By using
Java static initalizers, and introducing the new
com.xamarin.java_interop.ManagedPeer.registerNativeMembers() method:

	class Example {
		static final String assemblyQualifiedName = "Example, ...";
		static {
			com.xamarin.java_interop.ManagedPeer.registerNativeMembers (
					Example.class,
					assemblyQualifiedName,
					"method-blob"
			);
		}
	}

ManagedPeer.registerNativeMembers() will call into managed code and
(eventually) call JNIEnv::RegisterNatives() for the Java type.

Next important question: How does ManagedPeer.registerNativeMembers()
*actually* register the native methods?

Additionally, this mechanism needs to help support the future desired
Ahead-Of-Time (AOT) compilation mechanism of jnimarshalmethod-gen
(176240d).

The answer is the new
JniRuntime.JniTypeManager.RegisterNativeMembers() virtual method.
The default implementation will do the following:

 1. Lookup the assembly to use.
 1.a: Check to see if there is a corresponding -JniMarshalMethods
      assembly, as generated by jnimarshalmethod-gen.
      For example, if the type to register is in Example.dll, then the
      assembly Example-JniMarshalMethods.dll will be loaded.
 1.b: If (1.a) fails, then use the assembly that contains the
      Type being registered -- the assemblyQualifiedName parameter
      provided to ManagedPeer.registerNativeMembers().

 2. Lookup the type to use, which is the type's FullName resolved
    from the assembly found in (1).

 3. Look for the __RegisterNativeMembers() method on the type from (2)
    and invoke it, passing the JniType which corresponds to the Java
    class to register, and the "method-blob" value.

For example, when registering methods for the Example type in
Example.dll, the following methods will be searched for, and the first
method found will be invoked:

  Example.__RegisterNativeMembers() from Example-JniMarshalMethods.dll
  Example.__RegisterNativeMembers() from Example.dll

If no such __RegisterNativeMembers() method is found, we bail.

JniRuntime.JniTypeManager subclasses may do something else. For
example, Xamairn.Android could (will?) process the "method-blob"
string as it normally does within JNIEnv.RegisterJniNatives().

The benefit of this architecture is that it allows "normal user"
assemblies to be post-processed to emit the -JniMarshalMethods
assembly, and use *that* for method registration at runtime.
This in turn avoids the need to emit marshaling code at runtime, as
marshaling code was previously emitted.

[0]: http://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/functions.html
[1]: http://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/design.html#compiling_loading_and_linking_native_methods
[2]: http://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/functions.html#RegisterNatives

Author: jonpryor

Makefile

@@ -119,3 +119,4 @@ run-android: $(ATESTS)
 run-test-jnimarshal: bin/$(CONFIGURATION)/Java.Interop.Export-Tests.dll
 	MONO_TRACE_LISTENER=Console.Out \
 	mono --debug bin/$(CONFIGURATION)/jnimarshalmethod-gen.exe bin/$(CONFIGURATION)/Java.Interop.Export-Tests.dll
+	$(call RUN_TEST,$<)

src/Java.Interop.Export/Tests/Export-Tests.projitems

@@ -17,7 +17,7 @@
   </ItemGroup>
   <Target Name="BuildExportTestJar" Inputs="@(JavaExportTestJar)" Outputs="$(OutputPath)export-test.jar">
     <MakeDir Directories="$(OutputPath)et-classes" />
-    <Exec Command="javac -source 1.5 -target 1.6 -d &quot;$(OutputPath)et-classes&quot; @(JavaExportTestJar -&gt; '%(Identity)', ' ')" />
+    <Exec Command="javac -classpath &quot;$(OutputPath)java-interop.jar&quot; -source 1.5 -target 1.6 -d &quot;$(OutputPath)et-classes&quot; @(JavaExportTestJar -&gt; '%(Identity)', ' ')" />
     <Exec Command="jar cf &quot;$(OutputPath)export-test.jar&quot; -C &quot;$(OutputPath)et-classes&quot; ." />
   </Target>
 </Project>

src/Java.Interop.Export/Tests/Java.Interop/ExportTest.cs

@@ -1,4 +1,5 @@
 using System;
+using System.Linq;
 using System.Linq.Expressions;
 using System.Reflection;
 
@@ -10,6 +11,13 @@ namespace Java.InteropTests
 	[JniTypeSignature ("com/xamarin/interop/export/ExportType")]
 	public class ExportTest : JavaObject
 	{
+		static void __RegisterNativeMembers (JniType type, string members)
+		{
+			var methods = JniEnvironment.Runtime.ExportedMemberBuilder
+				.GetExportedMemberRegistrations (typeof (ExportTest));
+			type.RegisterNativeMethods (methods.ToArray ());
+		}
+
 		public ExportTest (ref JniObjectReference reference, JniObjectReferenceOptions transfer)
 			: base (ref reference, transfer)
 		{

src/Java.Interop.Export/Tests/Java.Interop/ExportedMemberBuilderTest.cs

@@ -33,8 +33,6 @@ public void AddExportMethods ()
 				Assert.AreEqual ("()V",             methods [1].Signature);
 				Assert.IsTrue (methods [1].Marshaler is Action<IntPtr, IntPtr>);
 
-				t.RegisterNativeMethods (methods.ToArray ());
-
 				var m = t.GetStaticMethod ("testStaticMethods", "()V");
 				JniEnvironment.StaticMethods.CallStaticVoidMethod (t.PeerReference, m);
 				Assert.IsTrue (ExportTest.StaticHelloCalled);

src/Java.Interop.Export/Tests/java/com/xamarin/interop/export/ExportType.java

@@ -1,6 +1,20 @@
 package com.xamarin.interop.export;
 
-public class ExportType {
+import java.util.ArrayList;
+
+import com.xamarin.java_interop.GCUserPeerable;
+
+public class ExportType
+		implements GCUserPeerable
+{
+
+	static  final   String  assemblyQualifiedName   = "Java.InteropTests.ExportTest, Java.Interop.Export-Tests, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null";
+	static {
+		com.xamarin.java_interop.ManagedPeer.registerNativeMembers (
+				ExportType.class,
+				assemblyQualifiedName,
+				"");
+	}
 
 	public static void testStaticMethods () {
 		staticAction ();
@@ -30,4 +44,16 @@ public void testMethods () {
 	public native void action ();
 	public native long funcInt64 ();
 	public native Object funcIJavaObject ();
+
+	ArrayList<Object>       managedReferences     = new ArrayList<Object>();
+
+	public void jiAddManagedReference (java.lang.Object obj)
+	{
+		managedReferences.add (obj);
+	}
+
+	public void jiClearManagedReferences ()
+	{
+		managedReferences.clear ();
+	}
 }

src/Java.Interop/Java.Interop/JavaProxyObject.cs

@@ -11,7 +11,7 @@ sealed class JavaProxyObject : JavaObject
 		static  readonly    JniPeerMembers                                  _members        = new JniPeerMembers (JniTypeName, typeof (JavaProxyObject));
 		static  readonly    ConditionalWeakTable<object, JavaProxyObject>   CachedValues    = new ConditionalWeakTable<object, JavaProxyObject> ();
 
-		static JavaProxyObject ()
+		static void __RegisterNativeMembers (JniType type, string members)
 		{
 			_members.JniPeerType.RegisterNativeMethods (
 					new JniNativeMethodRegistration ("equals",      "(Ljava/lang/Object;)Z",    (Func<IntPtr, IntPtr, IntPtr, bool>)    _Equals),

src/Java.Interop/Java.Interop/JavaProxyThrowable.cs

@@ -7,6 +7,10 @@ class JavaProxyThrowable : JavaException
 	{
 		new internal    const   string  JniTypeName = "com/xamarin/java_interop/internal/JavaProxyThrowable";
 
+		static void __RegisterNativeMembers (JniType type, string members)
+		{
+		}
+
 		public  Exception   Exception {get; private set;}
 
 		public JavaProxyThrowable (Exception exception)

src/Java.Interop/Java.Interop/JniRuntime.JniTypeManager.cs

@@ -1,5 +1,6 @@
 using System;
 using System.Collections.Generic;
+using System.Diagnostics;
 using System.Linq;
 using System.Reflection;
 
@@ -174,6 +175,50 @@ IEnumerable<Type> CreateGetTypesForSimpleReferenceEnumerator (string jniSimpleRe
 #endif  // !XA_INTEGRATION
 				yield break;
 			}
+
+			public virtual void RegisterNativeMembers (JniType nativeClass, Type type, string methods)
+			{
+				if (!TryLoadExternalJniMarshalMethods (nativeClass, type, methods) &&
+						!TryRegisterNativeMembers (nativeClass, type, methods)) {
+					throw new NotSupportedException ($"Could not register Java.class={nativeClass.Name} Managed.type={type.FullName}");
+				}
+			}
+
+			static bool TryLoadExternalJniMarshalMethods (JniType nativeClass, Type type, string methods)
+			{
+				var marshalMethodAssemblyName   = new AssemblyName (type.GetTypeInfo ().Assembly.GetName ().Name + "-JniMarshalMethods");
+				var marshalMethodsAssembly      = TryLoadAssembly (marshalMethodAssemblyName);
+				if (marshalMethodsAssembly == null)
+					return false;
+
+				var marshalType = marshalMethodsAssembly.GetType (type.FullName);
+				if (marshalType == null)
+					return false;
+				return TryRegisterNativeMembers (nativeClass, marshalType, methods);
+			}
+
+			static Assembly TryLoadAssembly (AssemblyName name)
+			{
+				try {
+					return Assembly.Load (name);
+				}
+				catch (Exception e) {
+					Debug.WriteLine ("Warning: Could not load JNI Marshal Method assembly '{0}': {1}", name, e);
+					return null;
+				}
+			}
+
+			static bool TryRegisterNativeMembers (JniType nativeClass, Type marshalType, string methods)
+			{
+				var registerMethod  = marshalType.GetTypeInfo ().GetDeclaredMethod ("__RegisterNativeMembers");
+				if (registerMethod == null) {
+					return false;
+				}
+
+				var register    = (Action<JniType, string>) registerMethod.CreateDelegate (typeof(Action<JniType, string>));
+				register (nativeClass, methods);
+				return true;
+			}
 		}
 	}
 }

src/Java.Interop/Java.Interop/ManagedPeer.cs

@@ -20,7 +20,14 @@ namespace Java.Interop {
 		static ManagedPeer ()
 		{
 			_members.JniPeerType.RegisterNativeMethods (
-					new JniNativeMethodRegistration ("construct",   ConstructSignature,     (Action<IntPtr, IntPtr, IntPtr, IntPtr, IntPtr, IntPtr>) Construct)
+					new JniNativeMethodRegistration (
+						"construct",
+						ConstructSignature,
+						(Action<IntPtr, IntPtr, IntPtr, IntPtr, IntPtr, IntPtr>) Construct),
+					new JniNativeMethodRegistration (
+						"registerNativeMembers",
+						RegisterNativeMembersSignature,
+						(Action<IntPtr, IntPtr, IntPtr, IntPtr, IntPtr>) RegisterNativeMembers)
 			);
 		}
 
@@ -30,7 +37,7 @@ static ManagedPeer ()
 
 		internal static void Init ()
 		{
-			// Present so that JavaVM has _something_ to reference to
+			// Present so that JniRuntime has _something_ to reference to
 			// prompt invocation of the static constructor & registration
 		}
 
@@ -170,6 +177,36 @@ static object[] GetValues (JniRuntime runtime, JniObjectReference values, Type[]
 
 			return pvalues;
 		}
+
+		const   string  RegisterNativeMembersSignature  = "(Ljava/lang/Class;Ljava/lang/String;Ljava/lang/String;)V";
+
+		static void RegisterNativeMembers (
+				IntPtr jnienv,
+				IntPtr klass,
+				IntPtr n_nativeClass,
+				IntPtr n_assemblyQualifiedName,
+				IntPtr n_methods)
+		{
+			var envp = new JniTransition (jnienv);
+			try {
+				var r_nativeClass   = new JniObjectReference (n_nativeClass);
+				var nativeClass     = new JniType (ref r_nativeClass, JniObjectReferenceOptions.Copy);
+
+				var assemblyQualifiedName   = JniEnvironment.Strings.ToString (new JniObjectReference (n_assemblyQualifiedName));
+				var methods                 = JniEnvironment.Strings.ToString (new JniObjectReference (n_methods));
+
+				var type    = Type.GetType (assemblyQualifiedName, throwOnError: true);
+
+				JniEnvironment.Runtime.TypeManager.RegisterNativeMembers (nativeClass, type, methods);
+			}
+			catch (Exception e) {
+				Debug.WriteLine (e.ToString ());
+				envp.SetPendingException (e);
+			}
+			finally {
+				envp.Dispose ();
+			}
+		}
 	}
 
 	class JniLocationException : Exception {

src/Java.Interop/Tests/Java.Interop/CallVirtualFromConstructorDerived.cs

@@ -9,7 +9,7 @@ public class CallVirtualFromConstructorDerived : CallVirtualFromConstructorBase
 		new internal    const   string          JniTypeName = "com/xamarin/interop/CallVirtualFromConstructorDerived";
 		static  readonly        JniPeerMembers  _members    = new JniPeerMembers (JniTypeName, typeof (CallVirtualFromConstructorDerived));
 
-		static CallVirtualFromConstructorDerived ()
+		static void __RegisterNativeMembers (JniType type, string members)
 		{
 			_members.JniPeerType.RegisterNativeMethods (
 					new JniNativeMethodRegistration ("calledFromConstructor", "(I)V", (Action<IntPtr, IntPtr, int>) CalledFromConstructorHandler));