Use local NPU/GPU for text generation and other capabilities when possible

[chrisglein]

Summary

When this app was originally created ChatGPT was brand new and online queries behind paid API keys were the only option. Since then we have the advent of CoPilot+ PCs which have hardware that can do some of this work. There's also the option to download local models, but I don't want to get into that quite yet. Instead I'd like to pursue an option for when the hardware supports it to not need an API key and use the local hardware capabilities.

From the AI Dev Gallery you can see many examples of this. And for text completion it looks like that's Phi Silica: https://learn.microsoft.com/en-us/windows/ai/apis/phi-silica

There's an example here:
https://learn.microsoft.com/en-us/windows/ai/apis/phi-silica-tutorial
https://github.com/microsoft/WindowsAppSDK-Samples/tree/main/Samples/WindowsAIFoundry/cs-winui

Most of the default examples shown here are in C#, but current the code behind for this app is in C++. There would need to be a TurboModule to expose the right methods to JavaScript, with the implementation in C++. If you dig into the specific methods you'll find C++/winrt examples too: https://learn.microsoft.com/en-us/windows/windows-app-sdk/api/winrt/microsoft.windows.ai.text.languagemodel.generateresponseasync?view=windows-app-sdk-1.8#microsoft-windows-ai-text-languagemodel-generateresponseasync(system-string)

Per the AI Dev Gallery, this may be needed as a Nuget dependency: https://www.nuget.org/packages/Microsoft.Extensions.AI

For implementing this I would assume that the existing JavaScrsipt-only OpenAI pure web implementation is fine as is, but there'd need to be a native module equivalent so that we can drop into C++ code and fill in whatever. If that's Phi Silica, great. In the future if that's something else, also fine. But I'd imagine the JavaScript contract is stable independent of that.

Of note, this app doesn't just do text generation; it also does image generation. Both are candidates here, but for now let's focus on text generation.