Installation failure on Julia ≥ 1.4 & Mac

[mcabbott]

I can't seem to make this work on Julia 1.4 or nightly, dev or add appears to install, but it freezes when I ask it to calculate anything:

(@v1.5) pkg> add IntelVectorMath
   Updating registry at `~/.julia/registries/General`
   Updating git-repo `https://github.com/JuliaRegistries/General.git`
┌ Warning: Some registries failed to update:
│     — /Users/me/.julia/registries/General — failed to fetch from repo
└ @ Pkg.Types /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.5/Pkg/src/Types.jl:1118
  Resolving package versions...
   Updating `~/.julia/environments/v1.5/Project.toml`
   c8ce9da6 + IntelVectorMath v0.4.0
   Updating `~/.julia/environments/v1.5/Manifest.toml`
   1d5cc7b8 + IntelOpenMP_jll v2018.0.3+0
   c8ce9da6 + IntelVectorMath v0.4.0
   856f044c + MKL_jll v2020.0.166+0

(@v1.5) pkg> build IntelVectorMath

julia> using IntelVectorMath
[ Info: Precompiling IntelVectorMath [c8ce9da6-5d36-5c03-b118-5a70151be7bc]

julia> pkgdir(IntelVectorMath)
"/Users/me/.julia/packages/IntelVectorMath/BuWXf"

julia> exp.(ones(2))
2-element Array{Float64,1}:
 2.718281828459045
 2.718281828459045

julia> IVM.exp(ones(2))
^CKilled: 9

$ cd /Users/me/.julia/packages/IntelVectorMath/BuWXf
$ ls
LICENSE.md   Project.toml README.md    benchmark    src          test

There seems to be no /deps/build.log file, and perhaps its failure to write here means something?

I can install it on Julia 1.3, on which I have MKL.jl installed. I can try other combinations of versions & MKL but haven't yet. (My 1.4 is the official download.) However in the deps/build.log which I think belongs to the 1.3 version, there are a bunch of errors printed:

$ ls
BuWXf r9uot uZB3Y
$ cd r9uot
$ ls
LICENSE.md   Project.toml README.md    benchmark    deps         src          test
$ cd deps
$ ls
build.jl  build.log usr
$ cat build.log
[ Info: Downloading https://github.com/JuliaPackaging/Yggdrasil/releases/download/MKL-v2019.0.117/MKL.v2019.0.117.x86_64-apple-darwin14.tar.gz to /Users/me/.julia/packages/IntelVectorMath/r9uot/deps/usr/downloads/MKL.v2019.0.117.x86_64-apple-darwin14.tar.gz...
ERROR: LoadError: Could not download https://github.com/JuliaPackaging/Yggdrasil/releases/download/MKL-v2019.0.117/MKL.v2019.0.117.x86_64-apple-darwin14.tar.gz to /Users/me/.julia/packages/IntelVectorMath/r9uot/deps/usr/downloads/MKL.v2019.0.117.x86_64-apple-darwin14.tar.gz:
TaskFailedException(Task (failed) @0x00000001069f1210)
Stacktrace:
 [1] error(::String) at /Applications/Julia-1.5.app/Contents/Resources/julia/lib/julia/sys.dylib:?
 [2] download(::String, ::String; verbose::Bool) at /Users/me/.julia/packages/BinaryProvider/kcGxO/src/PlatformEngines.jl:502
 [3] download_verify(::String, ::String, ::String; verbose::Bool, force::Bool, quiet_download::Bool) at /Users/me/.julia/packages/BinaryProvider/kcGxO/src/PlatformEngines.jl:571
 [4] install(::String, ::String; prefix::Prefix, tarball_path::String, force::Bool, ignore_platform::Bool, verbose::Bool) at /Users/me/.julia/packages/BinaryProvider/kcGxO/src/Prefix.jl:314
 [5] top-level scope at /Users/me/.julia/packages/IntelVectorMath/r9uot/deps/build.jl:51
 [6] include(::String) at /Applications/Julia-1.5.app/Contents/Resources/julia/lib/julia/sys.dylib:?
 [7] top-level scope at none:5
in expression starting at /Users/me/.julia/packages/IntelVectorMath/r9uot/deps/build.jl:49
caused by [exception 1]
TaskFailedException:
InterruptException:
Stacktrace:
 [1] poptaskref(::Base.InvasiveLinkedListSynchronized{Task}) at /Applications/Julia-1.5.app/Contents/Resources/julia/lib/julia/sys.dylib:?
 [2] wait(::Base.GenericCondition{Base.Threads.SpinLock}) at /Applications/Julia-1.5.app/Contents/Resources/julia/lib/julia/sys.dylib:?
 [3] wait_readnb(::Base.PipeEndpoint, ::Int64) at /Applications/Julia-1.5.app/Contents/Resources/julia/lib/julia/sys.dylib:?
 [4] eof at ./stream.jl:48 [inlined]
 [5] eof at ./io.jl:361 [inlined]
 [6] readuntil_many(::Pipe, ::Array{Char,1}) at /Users/me/.julia/packages/BinaryProvider/kcGxO/src/OutputCollector.jl:22
 [7] macro expansion at /Users/me/.julia/packages/BinaryProvider/kcGxO/src/OutputCollector.jl:49 [inlined]
 [8] (::BinaryProvider.var"#1#3"{Pipe,Base.GenericCondition{Base.AlwaysLockedST},Array{Tuple{Float64,String},1}})() at ./task.jl:358
Stacktrace:
 [1] wait at ./task.jl:267 [inlined]
 [2] fetch at ./task.jl:282 [inlined]
 [3] wait(::OutputCollector) at /Users/me/.julia/packages/BinaryProvider/kcGxO/src/OutputCollector.jl:155
 [4] download(::String, ::String; verbose::Bool) at /Users/me/.julia/packages/BinaryProvider/kcGxO/src/PlatformEngines.jl:495
 [5] download_verify(::String, ::String, ::String; verbose::Bool, force::Bool, quiet_download::Bool) at /Users/me/.julia/packages/BinaryProvider/kcGxO/src/PlatformEngines.jl:571
 [6] install(::String, ::String; prefix::Prefix, tarball_path::String, force::Bool, ignore_platform::Bool, verbose::Bool) at /Users/me/.julia/packages/BinaryProvider/kcGxO/src/Prefix.jl:314
 [7] top-level scope at /Users/me/.julia/packages/IntelVectorMath/r9uot/deps/build.jl:51
 [8] include(::String) at /Applications/Julia-1.5.app/Contents/Resources/julia/lib/julia/sys.dylib:?
 [9] top-level scope at none:5
AVX2 support detected, vml_avx2 selected

[KristofferC]

I can't seem to install this on Julia 1.4 or nightly, dev or add leads me to it freezing when I ask it to calculate anything:

You can install it, it just hangs when you call a function. I can repro, and it seems like this happens on 1.4 but not 1.3?

[KristofferC]

Here is an MWE that hangs for me:

using MKL_jll

gamma1(x) = (x = copy(x); ccall((:vdTGamma, libmkl_rt), Cvoid, (Int32, Ptr{Float64}, Ptr{Float64}), length(x), x, x); x);
gamma1([1.0, 2.0])

[mcabbott]

You can install it, it just hangs when you call a function.

Yes, sorry, proofreading failure too. (Now fixed.)

Your example crashes immediately for me on 1.4, while mine freezes and needs to be ctrl-C-ed.

[KristofferC]

Can you try with 1.3? And with the nightly?

[mcabbott]

Your example also works on my 1.3 (which has MKL.jl), and also fails on my 1.5. Both are MKL_jll v2020.0.166+0. (1.5 is 1.5.0-DEV.585 (2020-04-07) Commit 16652a6655* (13 days old master) with some packagecompiler-ing of plots & revise, but 1.4 is the default downloaded binary. Neither has MKL.jl.)

(@v1.5) pkg> st MKL_jll
     Status `~/.julia/environments/v1.5/Project.toml`
   856f044c MKL_jll v2020.0.166+0

julia> using MKL_jll

julia> gamma1(x) = (x = copy(x); ccall((:vdTGamma, libmkl_rt), Cvoid, (Int32, Ptr{Float64}, Ptr{Float64}), length(x), x, x); x);

julia> gamma1([1.0, 2.0])

^CKilled: 9

[KristofferC]

Okay thanks. So there has been some regression here in 1.3 to 1.4. I tried 1.4-RC2 and it seems to work there but it failed on 1.4.0.

I noticed that it works when I use my standard installation of MKL. So I looked at the difference between the libraries and there was one file that had a different hash: libmkl_intel_thread.dylib). When I copied that file from my local installation to the one MKL_jll uses it works.

So my conclusion is that the libmkl_intel_thread.dylib that MKL_jll provides on mac is somehow failty.

[KristofferC]

Opened JuliaPackaging/Yggdrasil#915

[KristofferC]

Can you try run

pkg> add https://github.com/KristofferC/MKL_jll.jl

and try again?

[mcabbott]

OK, that appears to fix things on 1.4:

  Resolving package versions...
   Updating `~/.julia/environments/v1.4/Project.toml`
  [856f044c] + MKL_jll v2020.0.166+1 #master (https://github.com/KristofferC/MKL_jll.jl)
   Updating `~/.julia/environments/v1.4/Manifest.toml`
  [1d5cc7b8] + IntelOpenMP_jll v2018.0.3+0
  [856f044c] + MKL_jll v2020.0.166+1 #master (https://github.com/KristofferC/MKL_jll.jl)

julia> using MKL_jll
[ Info: Precompiling MKL_jll [856f044c-d86e-5d09-b602-aeab76dc8ba7]
Downloading artifact: MKL

Downloading artifact: MKL

julia> gamma1(x) = (x = copy(x); ccall((:vdTGamma, libmkl_rt), Cvoid, (Int32, Ptr{Float64}, Ptr{Float64}), length(x), x, x); x);

julia> gamma1([1.0, 2.0])
2-element Array{Float64,1}:
 1.0
 1.0

(@v1.4) pkg> add IntelVectorMath
  Resolving package versions...
   Updating `~/.julia/environments/v1.4/Project.toml`
  [c8ce9da6] + IntelVectorMath v0.4.0
   Updating `~/.julia/environments/v1.4/Manifest.toml`
  [c8ce9da6] + IntelVectorMath v0.4.0

(@v1.4) pkg> build IntelVectorMath

julia> using IntelVectorMath
[ Info: Precompiling IntelVectorMath [c8ce9da6-5d36-5c03-b118-5a70151be7bc]

julia> IVM.exp(ones(3))
3-element Array{Float64,1}:
 2.718281828459045
 2.718281828459045
 2.718281828459045

And on 1.5 too. Many thanks!

[Crown421]

This was an issue for the last couple of months with the github actions CI for nightly, I mentioned it on the slack channel.
Thanks for finding a fix @KristofferC

[KristofferC]

This should be fixed with the new release of MKL_jll. We should make sure we run CI on 1.4.

[mcabbott]

Great, can confirm that MKL_jll v2020.0.166+1 solves the problem here. Thanks again for figuring this stuff out!

[xuangu-fang]

Can you try run

pkg> add https://github.com/KristofferC/MKL_jll.jl

and try again?

it helps to fix on 1.6 on win10