Updated Spectator View for Mobile for MRTK V2.

[Yoyozilla]

Adding spectator view mobile support for V2 with basic doc.

[RoyRoden]

Is this available in the mrtk release branch? will it work with hololens gen 1?

[chrisfromwork]

Right now this code is in feature/spectatorView. We are anticipating to have the code integrated into the mrtk_development and mrtk_release branches by the end of March.

The spectatorView functionality currently works with the original HoloLens. We are in the process of building support for HoloLens 2 but it will take longer to release. There may end up being some differences in functionality around marker detection, but the bulk logic for setting up the shared coordinate system should remain the same.

[RoyRoden]

Thanks @chrisfromwork
I just tried to move the MixedRealityToolkit.Extensions folder from feature/spectatorView into a local copy of mrtk_developement. There were a couple of namespace conflicts, but apart from that I managed to test it on HoloLens 1. It connected to both an Android and an iPhone each showing a marker on the screen, but it didn't go past the 'locating marker' step.

[chrisfromwork]

If the HoloLens is attempting to locate markers, it sounds like your network connection is working. Did you build the OpenCV plugin dlls for marker detection and include them in your Unity project? These dlls are only used on the HoloLens, but without them, you won't be able to detect markers.

See the feature/spectatorView branch in the MixedRealityToolkit repo:
https://github.com/Microsoft/MixedRealityToolkit/tree/feature/spectatorView/SpectatorViewPlugin
(Note: there are some instructions on the landing page provided above that should help with the steps related to building the needed OpenCV plugin)

We are in the process of updating documentation for this new spectator view refactor. We do currently have a few wiki pages related to the code. They will likely migrate to our GitHub documentation portal but can be found here:
https://github.com/Microsoft/MixedRealityToolkit-Unity/wiki/Spectator-View-Setup
https://github.com/Microsoft/MixedRealityToolkit-Unity/wiki/Spectator-View-Architecture
https://github.com/Microsoft/MixedRealityToolkit-Unity/wiki/Spectator-View-Debugging

[RoyRoden]

Thanks for the documentation @chrisfromwork
I added the generated DLLs to Assets\MixedRealityToolkit.Extensions\SpectatorView\Plugins\WSA\x86 but based on the Setup doc you shared, it seems I should've created a Plugins folder in the root of the Assets folder. I'll try again.

[chrisfromwork]

Hm, if you added the dlls to any folder named "Plugins", I think Unity will include them in your visual studio build (for more information see https://docs.unity3d.com/Manual/SpecialFolders.html). If you look at the directory created when building your Unity app for visual studio and see the SpectatorViewPlugin.dll and its dependencies in there, things have likely worked correctly.

There should be some error logging in the visual studio debug output console that might be able to better identify what is going wrong. It should generate errors if the dlls aren't resolved correctly. It may also be that you are hitting an unexpected boundary case when setting up the HoloLens PV camera for capturing images. If you can share a Unity log file from both your HoloLens and one of the mobile devices, we can try and help identify what is going wrong.

[Yoyozilla]

Spectator view is moving to a separate repo. closing this here.