Update iOS device queue to 18.x iOS

[matouskozak]

We need to ensure XHarness is compatible with 18.x (and iOS 17.x) helix queues to be able to successfully complete dotnet/runtime#113325.

• iOS 17.5 devices are in osx.1200.amd64.iphone.open queue
• iOS 18.1 devices are in osx.amd64.iphone.scouting.open queue

TODO

• Test Xharness on the above-mentioned queues
• Update Xharness to use iOS 18.x queue (together with completion of [infra][iOS][tvOS] Update Apple mobile OS versions used for CI testing to 18.x runtime#113325)

---

Note

The same iOS queue is used for servicing branches. Consequently, any changes to the queue must be validated to be compatible with servicing CI infrastructure.

[matouskozak]

Based on locally testing the queues are not working with current Xharness.

The latest mlaunch <PackageVersion Include="Microsoft.Tools.Mlaunch" Version="1.1.11" /> is not compatible with iOS 17+ devices.

[01:23:08.6266650] /private/tmp/helix/working/B64E09AC/p/microsoft.dotnet.xharness.cli/10.0.0-dev/tools/net8.0/any/../../../runtimes/any/native/mlaunch/bin/mlaunch --sdkroot /Applications/Xcode_16.1.app --logdev --devname Dncengosx162
[01:23:09.5567310] error MT0000: Unexpected error - Please file a bug report at https://github.com/xamarin/xamarin-macios/issues/new
[01:23:09.5582790] System.InvalidCastException: Unable to cast object of type 'Xamarin.Hosting.DeviceCtlDevice' to type 'Xamarin.Hosting.RealDevice'.
[01:23:09.5582830]    at Xamarin.Hosting.Logger.LogAsync(String, TextWriter) in /Users/builder/azdo/_work/9/s/src/Xamarin.Hosting/Xamarin.Hosting/Logger.cs:line 152
[01:23:09.5582840]    at Xamarin.Hosting.Logger.LogToConsoleAsync(String) in /Users/builder/azdo/_work/9/s/src/Xamarin.Hosting/Xamarin.Hosting/Logger.cs:line 172
[01:23:09.5582850]    at Xamarin.Hosting.Logger.<>c__DisplayClass5_0.<<LogToConsole>b__0>d.MoveNext() in /Users/builder/azdo/_work/9/s/src/Xamarin.Hosting/Xamarin.Hosting/Logger.cs:line 179
[01:23:09.5582850] --- End of stack trace from previous location ---
[01:23:09.5582860]    at System.Threading.Tasks.Task.<>c.<ThrowAsync>b__128_0(Object state)
[01:23:09.5582870]    at Foundation.NSAsyncSynchronizationContextDispatcher.Apply() in /Users/builder/azdo/_work/1/s/xamarin-macios/src/Foundation/NSAction.cs:line 176
[01:23:09.5582880]    at InvokeStub_NSAsyncSynchronizationContextDispatcher.Apply(Object, Object, IntPtr*)
[01:23:09.5583020]    at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object, BindingFlags)
[01:23:09.5583030] --- End of stack trace from previous location ---
[01:23:09.5583040]    at ObjCRuntime.Runtime.InvokeMethod(MethodBase, Object, IntPtr) in /Users/builder/azdo/_work/1/s/xamarin-macios/src/ObjCRuntime/Runtime.CoreCLR.cs:line 777
[01:23:09.5583050]    at ObjCRuntime.Runtime.InvokeMethod(MonoObject*, MonoObject*, IntPtr) in /Users/builder/azdo/_work/1/s/xamarin-macios/src/ObjCRuntime/Runtime.CoreCLR.cs:line 667
[01:23:09.5583050]    at ObjCRuntime.Runtime.bridge_runtime_invoke_method(MonoObject* method, MonoObject* instance, IntPtr parameters, IntPtr* exception_gchandle) in /Users/builder/azdo/_work/1/s/xamarin-macios/runtime/Delegates.generated.cs:line 1296
[01:23:09.5583060]    at ObjCRuntime.Messaging.void_objc_msgSend_NativeHandle(IntPtr, IntPtr, NativeHandle)
[01:23:09.5583070]    at Foundation.NSRunLoop.RunUntil(NSDate) in /Users/builder/azdo/_work/1/s/xamarin-macios/src/build/dotnet/macos/generated-sources/Foundation/NSRunLoop.g.cs:line 225
[01:23:09.5583210]    at Xamarin.Utils.NSRunLoopExtensions.RunUntilEvent(NSRunLoop, ManualResetEvent, Nullable`1 ) in /Users/builder/azdo/_work/9/s/src/Xamarin.Hosting/Xamarin.Utils/Extensions.cs:line 16
[01:23:09.5583220]    at Xamarin.Hosting.Services.RunMainLoopUntil(ManualResetEvent) in /Users/builder/azdo/_work/9/s/src/Xamarin.Hosting/Xamarin.Hosting/Services.cs:line 1750
[01:23:09.5583220]    at Xamarin.Hosting.Logger.LogToConsole(String) in /Users/builder/azdo/_work/9/s/src/Xamarin.Hosting/Xamarin.Hosting/Logger.cs:line 182
[01:23:09.5583230]    at Xamarin.Launcher.DevController.ShowLog(String) in /Users/builder/azdo/_work/9/s/src/Xamarin.Hosting/Xamarin.Launcher/controller-device.cs:line 46
[01:23:09.5583270]    at Xamarin.Launcher.Driver.MainAsync() in /Users/builder/azdo/_work/9/s/src/Xamarin.Hosting/Xamarin.Launcher/Main.cs:line 452
[01:23:09.5583280]    at Xamarin.Utils.NSRunLoopExtensions.RunUntilTaskCompletion[T](NSRunLoop, Task`1) in /Users/builder/azdo/_work/9/s/src/Xamarin.Hosting/Xamarin.Utils/Extensions.cs:line 29
[01:23:09.5583280]    at Xamarin.Launcher.Driver.Main2(String[]) in /Users/builder/azdo/_work/9/s/src/Xamarin.Hosting/Xamarin.Launcher/Main.cs:line 441
[01:23:09.5583320]    at Xamarin.Launcher.Driver.Main(String[]) in /Users/builder/azdo/_work/9/s/src/Xamarin.Hosting/Xamarin.Launcher/Main.cs:line 128
[01:23:09.5644180] 

xharness/src/Microsoft.DotNet.XHarness.iOS.Shared/Execution/Arguments.cs

Lines 43 to 48 in 4d96da7

	public sealed class LogDevArgument : OptionArgument
	{
	public LogDevArgument() : base("logdev")
	{
	}
	}

mlaunch issue: dotnet/macios#22394

[ivanpovazan]

What about version: 1.0.272, which is used within macios repo?

[matouskozak]

What about version: 1.0.272, which is used within macios repo?

I think this version of mlaunch is missing retrieval of DeviceIdentifier which was fixed in subsequent versions

[04:26:51] crit: System.ArgumentNullException: Value cannot be null. (Parameter 'argumentValue')
                    at Microsoft.DotNet.XHarness.iOS.Shared.Execution.SingleValueArgument..ctor(String argumentName, String argumentValue, Boolean useEqualSign) in <PATH_TO_XHARNESS>/xharness/src/Microsoft.DotNet.XHarness.iOS.Shared/Execution/MLaunchArguments.cs:line 39
                    at Microsoft.DotNet.XHarness.iOS.Shared.Execution.DeviceNameArgument..ctor(IDevice device) in <PATH_TO_XHARNESS>/xharness/src/Microsoft.DotNet.XHarness.iOS.Shared/Execution/Arguments.cs:line 91
                    at Microsoft.DotNet.XHarness.Apple.AppUninstaller.UninstallDeviceApp(IHardwareDevice device, String appBundleId, CancellationToken cancellationToken) in <PATH_TO_XHARNESS>/xharness/src/Microsoft.DotNet.XHarness.Apple/AppOperations/AppUninstaller.cs:line 42
                    at Microsoft.DotNet.XHarness.Apple.BaseOrchestrator.UninstallApp(TestTarget target, String bundleIdentifier, IDevice device, Boolean isPreparation, CancellationToken cancellationToken) in <PATH_TO_XHARNESS>/xharness/src/Microsoft.DotNet.XHarness.Apple/Orchestration/BaseOrchestrator.cs:line 443
                    at Microsoft.DotNet.XHarness.Apple.BaseOrchestrator.OrchestrateOperationInternal(TestTargetOs target, String deviceName, Boolean includeWirelessDevices, Boolean resetSimulator, Boolean enableLldb, GetAppBundleInfoFunc getAppBundle, ExecuteMacCatalystAppFunc executeMacCatalystApp, ExecuteAppFunc executeApp, CancellationToken cancellationToken) in <PATH_TO_XHARNESS>/xharness/src/Microsoft.DotNet.XHarness.Apple/Orchestration/BaseOrchestrator.cs:line 287
                    at Microsoft.DotNet.XHarness.Apple.BaseOrchestrator.OrchestrateOperation(TestTargetOs target, String deviceName, Boolean includeWirelessDevices, Boolean resetSimulator, Boolean enableLldb, GetAppBundleInfoFunc getAppBundle, ExecuteMacCatalystAppFunc executeMacCatalystApp, ExecuteAppFunc executeApp, CancellationToken cancellationToken) in <PATH_TO_XHARNESS>/xharness/src/Microsoft.DotNet.XHarness.Apple/Orchestration/BaseOrchestrator.cs:line 96
                    at Microsoft.DotNet.XHarness.Apple.TestOrchestrator.OrchestrateTest(GetAppBundleInfoFunc getAppBundlePath, TestTargetOs target, String deviceName, TimeSpan timeout, TimeSpan launchTimeout, CommunicationChannel communicationChannel, XmlResultJargon xmlResultJargon, IEnumerable`1 singleMethodFilters, IEnumerable`1 classMethodFilters, Boolean includeWirelessDevices, Boolean resetSimulator, Boolean enableLldb, Boolean signalAppEnd, IReadOnlyCollection`1 environmentalVariables, IEnumerable`1 passthroughArguments, CancellationToken cancellationToken) in <PATH_TO_XHARNESS>/xharness/src/Microsoft.DotNet.XHarness.Apple/Orchestration/TestOrchestrator.cs:line 189
                    at Microsoft.DotNet.XHarness.CLI.Commands.Apple.AppleAppCommand`1.Invoke(ILogger logger) in <PATH_TO_XHARNESS>/xharness/src/Microsoft.DotNet.XHarness.CLI/Commands/Apple/AppleAppCommand.cs:line 68
                    at Microsoft.DotNet.XHarness.CLI.Commands.XHarnessCommand`1.Invoke(IEnumerable`1 arguments) in <PATH_TO_XHARNESS>/xharness/src/Microsoft.DotNet.XHarness.CLI/Commands/XHarnessCommand.cs:line 145

[matouskozak]

Currently blocked by dotnet/macios#22632 (comment), latest run at #1345. cc: @kotlarmilos

[akoeplinger]

I think for the main use case of xharness we should be fine with using devicectl/simctl to copy the testResults.xml back to the host. The TCP tunnel is a source of unreliability too so it'd probably be better to get rid of it anyway.