Apple Silicon support

[nhirschey]

When planning your roadmap, please consider adding apple silicon support. I know that adding support is not trivial, but I think there’s value in supporting the platform.

Apple silicon support is important to allow TorchSharp use in an educational setting. My unscientific survey is that ~50% of students in university classes use apple silicon. It would also make it easier for downstream libraries such as Diffsharp¹ and ML.NET's new TextClassification trainer to support Apple silicon.

libtorch-cpu on apple silicon would be sufficient, but gpu-acceleration is also available now in PyTorch 1.12.

Footnotes

1. It is possible to use the DiffSharp reference backend on apple silicon. ↩

[dayo05]

Maybe same with #643

[NiklasGustafsson]

We haven't upgraded to 1.12 yet -- does the Pytorch backend support Mac silicon? It didn't in 1.11, AFAIK. TorchSharp is just a wrapper on top of libtorch.dylib on MacOS, so any silicon support will come from the native implementation.

[dayo05]

import torch

mps = torch.device("mps")
print(torch.cuda.is_available())
print(mps)

x = torch.ones(1).to(mps)
print(x.cpu())

I can execute this code on my 14inch macbook pro which provides M1 Pro without rosetta2.

Also, this requires PyTorch 1.11 nightly

[nhirschey]

Here’s the 1.12 link for Apple silicon gpu backend support link.

My understanding (as day05) illustrates was that cpu training on ARM64 Apple silicon was working already.

[NiklasGustafsson]

Also, this requires PyTorch 1.11 nightly

So, it's not in the stable build, then? We'll definitely wait until it is before picking it up. If it is available in a 1.11 stable build, then we can update the libtorch NuGet packages, and then add the relevant device API logic (device type, index, etc.)

[NiklasGustafsson]

Also, I don't have access to a M1-based Mac... Once we're on 1.12, I may have to ask someone in the community to do the integration.

[nhirschey]

I mainly use windows but I have access to an M1 Mac and can help test.

Also, I think that it is in 1.11 stable. My understanding from PyTorch forums is that the version number distinguishes stable from nightly. Stable is clean "1.11.0", nightly is something like "1.11.0.dev20201109".

This is what I get after installing 1.11 on an m1 Mac:

> pip install torch==1.11.0`
Collecting torch==1.11.0
  Downloading torch-1.11.0-cp39-none-macosx_11_0_arm64.whl (43.1 MB)
     |████████████████████████████████| 43.1 MB 40.3 MB/s 

Then in Python:

Python 3.9.12 (main, Apr  5 2022, 01:52:34) 
[Clang 12.0.0 ] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> x = torch.rand(5,3)
>>> print(x)
tensor([[0.2262, 0.6853, 0.9713],
        [0.1158, 0.4903, 0.1279],
        [0.4842, 0.3445, 0.8207],
        [0.4095, 0.3618, 0.7080],
        [0.0212, 0.3360, 0.3603]])
>>> print(torch.__version__)
1.11.0
>>> 

[nhirschey]

For what it's worth, this is what happens when I try to load LibTorch manually on M1:

> open System.Runtime.InteropServices

> NativeLibrary.Load(@"/Users/xxxx/opt/miniconda3/lib/python3.9/site-packages/torch/lib/libtorch.dylib")
val it: nativeint = 8618041536n

> #r "nuget: TorchSharp"
> open TorchSharp
> let lin1 = torch.nn.Linear(1000, 100)

Binding session to '/Users/xxx/.nuget/packages/torchsharp/0.97.0/lib/netcoreapp3.1/TorchSharp.dll'...

error FS0193: Could not load file or assembly 'TorchSharp, Version=0.97.0.0, Culture=neutral, PublicKeyToken=67b0e5e1da516a1c'.

error FS0193: Could not load file or assembly 'TorchSharp, Version=0.97.0.0, Culture=neutral, PublicKeyToken=67b0e5e1da516a1c, processorArchitecture=AMD64'. The given assembly name or codebase was invalid. (0x80131047)

[NiklasGustafsson]

For what it's worth, this is what happens when I try to load LibTorch manually on M1:

> open System.Runtime.InteropServices

> NativeLibrary.Load(@"/Users/xxxx/opt/miniconda3/lib/python3.9/site-packages/torch/lib/libtorch.dylib")
val it: nativeint = 8618041536n

For what it's worth, I don't think you can point at a conda distribution's native libs. If you are going to replace LibTorch, I believe you have to download and unzip the C/C++ distribution.

[NiklasGustafsson]

Also, I've started work on moving on to 1.12 binaries. Running into some problems, though. Hopefully, we can be on 1.12 next week some time.

[NiklasGustafsson]

We will skip 1.12 compat and integrate with 1.13 when that comes out later. That should give us M1 support, too.

[nhirschey]

Sounds good, thank you Niklas.

[ericstj]

@NiklasGustafsson would that give us ARM64 on linux too? I see that's mentioned here: https://pytorch.org/blog/prototype-features-now-available-apis-for-hardware-accelerated-mobile-and-arm64-builds/#arm64-builds-for-linux

[NiklasGustafsson]

@nhirschey, @ericstj, @dayo05

Now that 1.13 is out, we should be getting closer to additional platform support. I will release the first version with 1.13 support as 0.99.0, but not do anything to add support for M1. I'll have to ask someone who has access to a machine with M1 support to work on that.

[dayo05]

Reminder: I've just tried to compile libtorchsharp, I got cpuid.h is only exists on x86 system. Maybe we should resolve this