Again, the releases don't have the libraries.

[0wwafa]

./build/bin/llama-quantize: error while loading shared libraries: libllama.so: cannot open shared object file: No such file or directory

it already happened in the past...

[0wwafa]

Please test the binaries on colab :D if it works there it works everywhere.

[0wwafa]

Version 40xx work..
Version 42xx don't

I tried a few. don't know ehen it "EFFED UP" :D

[dagbs]

Version b4406 built successfully with no libllama.so error.

I setup a script to automatically go through and test until I found one that works. Although, yes... this is currently an issue with latest versions.

To be used with: https://gist.github.com/dagbs/1258a52addeb61cbf26e24f75b2ad7d9

#!/bin/bash

# Starting version and minimum version
start_version=4416
min_version=4000

current_version=$start_version

while [ $current_version -ge $min_version ]; do
    echo "Attempting to build and test version b$current_version"
    
    # Download the specified version
    ./download_latest.sh "b$current_version"
    
    # Build and check if llama-cli works without errors
    if ./build/bin/llama-cli --help 2>&1 | grep -q 'libllama.so'; then
        echo "Error: libllama.so not found in version b$current_version."
    else
        echo "Version b$current_version built successfully with no libllama.so error."
        break # Exit the loop if a successful build is found
    fi
    
    # Decrease the version number and wait 5 seconds before the next attempt
    current_version=$((current_version - 1))
    sleep 5
done

if [ $current_version -lt $min_version ]; then
    echo "No suitable version found. All versions from b$start_version to b$min_version had issues."
fi

[bandoti]

@0wwafa Do you mean to say that when downloading llama.cpp it is not including the shared libraries? If so where are you downloading from? Or are you building, installing, and then running the application and those shared libraries are missing?

If this happened before it is helpful to include a link to a previous issue or PR that resolved the problem you are seeing.

Folks use llama.cpp in various ways so it is necessary to have the extra context so we can help resolve the issue accordingly!

[pjh64]

I ran into the same issue and I remember it to also happen on other platforms such as x86_64.
There seem to be two inconveniences for me. The first is, that cmake does seem to be necessary now. I really appreciated the ease of just running make on even the most basic systems to compile llama.cpp. But that aside, this change will certainly have its reasons. The library issue however seems to be much more of a problem.
It works however when calling the binaries directly from their build dir.

Here is the entire build log from my last compiling attempt on the arm64 platform (but the same happens on x86_64):
:
https://pastebin.com/EK6vNMn0

[bandoti]

@pjh64 thanks for the detailed logs!

So, what I gather is that you are running on a Raspberry Pi. In that case, here are the considerations when using the dynamic build, and I will offer some possible resolutions after.

When Llama.cpp is being installed on your system, it is using a default prefix of /usr/local/bin and /usr/local/lib. The backend libraries are getting installed to the libs directory, and the applications to the bin directory. Now, my first guess is that these paths are not in the default loader path so the system doesn't know where to find them.

And we cannot guarantee that all operating systems will use the same default paths although perhaps there's a smarter way to query that info at build time and some improvements can be made. However, what I would suggest is:

1. If you want to keep the current install path simply prefix running export LD_LIBRARY_PATH="/usr/local/lib:$LD_LIBRARY_PATH" before running llama-cli &co.
2. Change the install prefix to match the system you are running on using cmake --build . --prefix /usr/lib (or similar) instead of the default make install.
3. Compile Llama.cpp statically if you know you are going to use the same backend cmake .. -DBUILD_SHARED_LIBS=OFF

You will have to verify where your system typically loads shared libraries by default and go from there!

[0wwafa]

I mean that if I do:

latest=`curl -qsI "https://github.com/ggerganov/llama.cpp/releases/latest"|grep location|cut -d " " -f 2`
latest=`basename $latest|tr -d "\r"`
#latest='b4000' # <<<<<<<<<<<<<<<< if I uncomment this it will work.
echo Downloading llama.cpp binaries
wget &>/dev/null "https://github.com/ggerganov/llama.cpp/releases/download/$latest/llama-$latest-bin-ubuntu-x64.zip" -O llama.zip
echo Unzipping llama.cpp binaries
unzip &>/dev/null llama.zip

then quantize won't work!
in b4000 it works.
(on google colab)

[0wwafa]

I setup a script to automatically go through and test until I found one that works. Although, yes... this is currently an issue with latest versions.

This is easier to download the latest or a specific version:

latest=`curl -qsI "https://github.com/ggerganov/llama.cpp/releases/latest"|grep location|cut -d " " -f 2`
latest=`basename $latest|tr -d "\r"`
#latest='b4000' # <<<<<<<<<<<<<<<< if I uncomment this it will work.
echo Downloading llama.cpp binaries
wget &>/dev/null "https://github.com/ggerganov/llama.cpp/releases/download/$latest/llama-$latest-bin-ubuntu-x64.zip" -O llama.zip
echo Unzipping llama.cpp binaries
unzip &>/dev/null llama.zip

[ggerganov]

The CI is packing only the files from the build/bin folder:

https://github.com/ggerganov/llama.cpp/blob/84aef8dfa8b12be297c3ec127f599e8135775bd7/.github/workflows/build.yml#L215-L220

I think it has to also pack the binaries from build/lib too. Please fill free to improve this. I don't use the release packages and so I don't know what are the issues with those. It would be best if people that use them help us maintain these.

[0wwafa]

just compile static or do as you did before (version 4400 works)

[egw]

just compile static or do as you did before (version 4400 works)

I'm not familiar enough with what the ci build process is doing, but if you are, you can submit a pull request that reverts the changes from commit f66f582 to .github/workflows/build.yml. The only relevant commit comment is '* ci : remove BUILD_SHARED_LIBS=OFF', which is a fine explanation of what it's doing, but doesn't say why ;)

Anyhow, if you do this, make sure you only revert the changes to build.yml and not to the other sixty(!) files in the commit!

[bandoti]

@egw This change is because the shared library builds now allow multiple backends.

Instead of reverting this change it makes sense to bundle the libraries as well. It is a couple shell script changes mentioned in the link by ggerganov.

I am happy to advise if you want to learn some actions workflow 😊

[TashaSkyUp]

build process wont work, releases don't work..okay must find alternative to llama.cpp

[bandoti]

@TashaSkyUp sorry this issue is causing frustration. It is my goal to improve the user experience in the llama tooling, including the build/release experience. So if you would be so kind and refer me to an open issue describing the problem with your build I can try and help.

[hrez]

@bandoti this is the open issue describing the problem.
This repo's readme says "The main product of this project is the llama library". Further saying "Possible methods for obtaining the binaries: ... Download pre-built binaries from releases".
Yet, release binaries do not have llama library. While it's supposedly the main product of this repo. Those binaries that are provided unusable without this library, hence the frustration:
./llama-server: error while loading shared libraries: libllama.so: cannot open shared object file: No such file or directory