PyMuPDF no longer builds in Alpine Linux

[apyrgio]

Description of the bug

Installing PyMuPDF on Alpine Linux via PyPI fails. It seems that this happens while building MuPDF 1.23.10. The error log is pretty big, filled with lines like these:

      (+2m01.0s): -b: 1: __main__.py:1638:build: platform/c++/implementation/functions.cpp: In function 'pdf_obj* mupdf::ll_pdf_dict_getlv(pdf_obj*, __va_list_tag*)':
      (+2m01.0s): -b: 1: __main__.py:1638:build: platform/c++/implementation/functions.cpp:379:24: error: 'll_pdf_dict_get' was not declared in this scope; did you mean 'll_pdf_dict_getl'?
      (+2m01.0s): -b: 1: __main__.py:1638:build:   379 |                 dict = ll_pdf_dict_get( dict, key);
      (+2m01.0s): -b: 1: __main__.py:1638:build:       |                        ^~~~~~~~~~~~~~~
      (+2m01.0s): -b: 1: __main__.py:1638:build:       |                        ll_pdf_dict_getl
      (+2m01.3s): -b: 1: __main__.py:1638:build: [returned e=1]

How to reproduce the bug

Steps to reproduce the bug:

1. Execute into an Alpine Linux environment:

   docker run --rm -it python:alpine ash

2. Install necessary helpers:

   apk add gcc python3-dev musl-dev linux-headers make g++

3. Install PyMuPDF from PyPI:

   $ pip install pymupdf
   [...]
            (+2m01.0s): -b: 1: __main__.py:1638:build: platform/c++/implementation/functions.cpp: In function 'pdf_obj* mupdf::ll_pdf_dict_getlv(pdf_obj*, __va_list_tag*)':
            (+2m01.0s): -b: 1: __main__.py:1638:build: platform/c++/implementation/functions.cpp:379:24: error: 'll_pdf_dict_get' was not declared in this scope; did you mean 'll_pdf_dict_getl'?
            (+2m01.0s): -b: 1: __main__.py:1638:build:   379 |                 dict = ll_pdf_dict_get( dict, key);
            (+2m01.0s): -b: 1: __main__.py:1638:build:       |                        ^~~~~~~~~~~~~~~
            (+2m01.0s): -b: 1: __main__.py:1638:build:       |                        ll_pdf_dict_getl
            (+2m01.3s): -b: 1: __main__.py:1638:build: [returned e=1]
            Traceback (most recent call last):
                scripts/mupdfwrap.py:6:<module>(): wrap.__main__.main()
                scripts/wrap/__main__.py:2902:main(): jlib.exception_info()
                ^except raise:
                scripts/wrap/__main__.py:2900:main(): main2()
                scripts/wrap/__main__.py:2310:main2(): build( build_dirs, swig_command, args, vs_upgrade)
                scripts/wrap/__main__.py:1638:build(): jlib.build(
                scripts/jlib.py:2213:build(): system( command, out=out, verbose=verbose, executable=executable, caller=2)
                scripts/jlib.py:1682:system(): raise Exception( message)
            Exception: Command failed: c++ \
                -o build/PyMuPDF-x86_64-shared-tesseract-release/libmupdfcpp.so \
             \
                -O2 -DNDEBUG \
                -fPIC -shared \
                -I /tmp/pip-install-pwa29jj3/pymupdf_d1ee6c3319d8472c856164f1a06dcb0c/mupdf-1.23.10-source/include \
                -I /tmp/pip-install-pwa29jj3/pymupdf_d1ee6c3319d8472c856164f1a06dcb0c/mupdf-1.23.10-source/platform/c++/include \
                 platform/c++/implementation/classes.cpp platform/c++/implementation/classes2.cpp platform/c++/implementation/exceptions.cpp platform/c++/implementation/extra.cpp platform/c++/implementation/functions.cpp platform/c++/implementation/internal.cpp \
                -L build/PyMuPDF-x86_64-shared-tesseract-release -l mupdf -Wl,-rpath,'$ORIGIN',-z,origin
            Traceback (most recent call last):
              File "/usr/local/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
                main()
              File "/usr/local/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
                json_out['return_val'] = hook(**hook_input['kwargs'])
                                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
              File "/usr/local/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 152, in prepare_metadata_for_build_wheel
                whl_basename = backend.build_wheel(metadata_directory, config_settings)
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
              File "/tmp/pip-install-pwa29jj3/pymupdf_d1ee6c3319d8472c856164f1a06dcb0c/pipcl.py", line 642, in build_wheel
                items = self._call_fn_build(config_settings)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
              File "/tmp/pip-install-pwa29jj3/pymupdf_d1ee6c3319d8472c856164f1a06dcb0c/pipcl.py", line 804, in _call_fn_build
                ret = self.fn_build()
                      ^^^^^^^^^^^^^^^
              File "/tmp/pip-install-pwa29jj3/pymupdf_d1ee6c3319d8472c856164f1a06dcb0c/setup.py", line 560, in build
                mupdf_build_dir = build_mupdf_unix( mupdf_local, env_extra, build_type)
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
              File "/tmp/pip-install-pwa29jj3/pymupdf_d1ee6c3319d8472c856164f1a06dcb0c/setup.py", line 803, in build_mupdf_unix
                subprocess.run( command, shell=True, check=True)
              File "/usr/local/lib/python3.12/subprocess.py", line 571, in run
                raise CalledProcessError(retcode, process.args,
            subprocess.CalledProcessError: Command 'cd /tmp/pip-install-pwa29jj3/pymupdf_d1ee6c3319d8472c856164f1a06dcb0c/mupdf-1.23.10-source && XCFLAGS=-DTOFU_CJK_EXT XCXXFLAGS=-DTOFU_CJK_EXT /usr/local/bin/python ./scripts/mupdfwrap.py -d build/PyMuPDF-x86_64-shared-tesseract-release -b all && echo /tmp/pip-install-pwa29jj3/pymupdf_d1ee6c3319d8472c856164f1a06dcb0c/mupdf-1.23.10-source/build/PyMuPDF-x86_64-shared-tesseract-release: && ls -l /tmp/pip-install-pwa29jj3/pymupdf_d1ee6c3319d8472c856164f1a06dcb0c/mupdf-1.23.10-source/build/PyMuPDF-x86_64-shared-tesseract-release' returned non-zero exit status 1.
            [end of output]
      
        note: This error originates from a subprocess, and is likely not a problem with pip.
      error: metadata-generation-failed
      
      × Encountered error while generating package metadata.
      ╰─> See above for output.
      
      note: This is an issue with the package mentioned above, not pip.
      hint: See above for details.

PyMuPDF version

1.23.26

Operating system

Linux

Python version

3.12

[julian-smith-artifex-com]

Unfortunately this is a known problem for which i don't know a simple solution. It's caused by the recent (two days ago) release of a new version 18 of the libclang module on pypi.org, which has slightly different behaviour from earlier versions, and breaks the generation of MuPDF C++ bindings.

The next PyMuPDF release will use an updated MuPDF that works with libclang 18, so should build fine for you. I'm hoping that this release will be made later this week.

Ideally there would be a way to tell pip to use a particular version (libclang 16.0.6 is known to work) of a prerequisite when building+installing from an sdist. I've tried pip -c <constraint-file> with <constraint-file> containing libclang==16.0.6, but pip still installs libclang 18. The only way i know to fix things would be to manually download the sdist, extract it, and edit pyproject.toml changing libclang to libclang==16.0.6, the run pip install . in the extracted PyMuPDF-... directory.

Separate from this, i'm curious why pip doesn't install from a wheel. Is there something about your Alpine system that makes it incompatible with the manylinux PyMuPDF wheels on pypip.org?

[apyrgio]

Thanks for the prompt reply, looking forward to the new release!

Separate from this, i'm curious why pip doesn't install from a wheel. Is there something about your Alpine system that makes it incompatible with the manylinux PyMuPDF wheels on pypip.org?

That's probably because Alpine Linux uses musl instead of glibc, and almost everything is statically compiled.

[julian-smith-artifex-com]

Fixed in 1.24.0.

[waza-ari]

Just for the people coming here to look for a solution and want to copy, one additional dependency is missing from what @apyrgio wrote. clang-dev must be installed and python3-dev is not needed. So the result that was working for me will be (python 3.12-alpine and pymupdf==1.24.2):

docker run --rm -it python:3.12-alpine ash
apk add gcc musl-dev linux-headers make g++ clang-dev
pip install pymupdf

[JanPokorny]

@waza-ari Looks like python3-dev actually is needed, at least with the current version.

Full Dockerfile that worked for me:

FROM docker.io/alpine:3.20
RUN apk add --no-cache gcc musl-dev linux-headers make g++ clang-dev python3 python3-dev py3-pip && \
    pip install --target /out pymupdf

[julian-smith-artifex-com]

This issue was fixed in 1.24.0.

Please do not post here unless it is to reopen this issue, otherwise it's difficult to keep track of what's going on. Instead please create a new issue or discussion.

Thanks.