pip 25.1 crashes when installing PySide6

[aplzr]

Description

I'm running a CI pipeline that updates pip to the latest version as one of the first steps, followed by installation of various packages, including PySide6. This pipeline is failing now with pip 25.1 during installation of PySide6. Versions of PySide6 and all its dependencies have not changed between the last suceeding pipeline and the currently failing ones. Pinning pip to 25.0.1 resolves the problem.

Expected behavior

CI pipeline doesn't fail when using pip 25.1.

pip version

25.1

Python version

3.9.5

OS

Debian 10 Buster

How to Reproduce

Minimal reproducible example of the failing CI pipeline on a Gitlab runner

stages:
  - test

test_pip:
  stage: test
  variables:
    GIT_STRATEGY: clone
  tags:
    - docker
    - hsa
  image: python:3.9.5
  before_script:
    - pip install pip==25.1
  script:
    - pip install PySide6

Minimal reproducible example of the succeeding CI pipeline with pip pinned to 25.0.1

stages:
  - test

test_pip:
  stage: test
  variables:
    GIT_STRATEGY: clone
  tags:
    - docker
    - hsa
  image: python:3.9.5
  before_script:
    - pip install pip==25.0.1
  script:
    - pip install PySide6

Output

Output of the failing pipeline with pip 25.1

Running with gitlab-runner 17.6.0 (374d34fd)
  on olserv05 Docker-Runner oHoJyH23J, system ID: s_343cc2df7956
Preparing the "docker" executor 00:03
Using Docker executor with image python:3.9.5 ...
Pulling docker image python:3.9.5 ...
Using docker image sha256:9b0d330dfd02b60939072b19a42eedf31064bc32ae642d4fca7468778421c787 for python:3.9.5 with digest python@sha256:2ff7f45e91d65fc36bee74e48692a5b1877c973a040ed2468aef4223edf9ccac ...
Preparing environment 00:01
Running on runner-ohojyh23j-project-1808-concurrent-0 via olserv05...
Getting source from Git repository 00:02
Fetching changes with git depth set to 20...
Initialized empty Git repository in /builds/hsa/tools/mnt/mntdatasets/.git/
Created fresh repository.
Checking out df147188 as detached HEAD (ref is test-pip)...
Skipping Git submodules setup
Executing "step_script" stage of the job script 00:16
Using docker image sha256:9b0d330dfd02b60939072b19a42eedf31064bc32ae642d4fca7468778421c787 for python:3.9.5 with digest python@sha256:2ff7f45e91d65fc36bee74e48692a5b1877c973a040ed2468aef4223edf9ccac ...
$ pip install pip==25.1
Collecting pip==25.1
  Downloading pip-25.1-py3-none-any.whl (1.8 MB)
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 21.1.3
    Uninstalling pip-21.1.3:
      Successfully uninstalled pip-21.1.3
Successfully installed pip-25.1
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
$ pip install PySide6
Collecting PySide6
  Downloading PySide6-6.9.0-cp39-abi3-manylinux_2_28_x86_64.whl.metadata (5.5 kB)
Collecting shiboken6==6.9.0 (from PySide6)
  Downloading shiboken6-6.9.0-cp39-abi3-manylinux_2_28_x86_64.whl.metadata (2.7 kB)
Collecting PySide6-Essentials==6.9.0 (from PySide6)
  Downloading PySide6_Essentials-6.9.0-cp39-abi3-manylinux_2_28_x86_64.whl.metadata (3.9 kB)
Collecting PySide6-Addons==6.9.0 (from PySide6)
  Downloading PySide6_Addons-6.9.0-cp39-abi3-manylinux_2_28_x86_64.whl.metadata (4.2 kB)
Downloading PySide6-6.9.0-cp39-abi3-manylinux_2_28_x86_64.whl (558 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 558.1/558.1 kB 12.5 MB/s eta 0:00:00
Downloading PySide6_Addons-6.9.0-cp39-abi3-manylinux_2_28_x86_64.whl (166.3 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 166.3/166.3 MB 81.1 MB/s eta 0:00:00
Downloading PySide6_Essentials-6.9.0-cp39-abi3-manylinux_2_28_x86_64.whl (94.2 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 94.2/94.2 MB 80.5 MB/s eta 0:00:00
Downloading shiboken6-6.9.0-cp39-abi3-manylinux_2_28_x86_64.whl (206 kB)
Installing collected packages: shiboken6, PySide6-Essentials, PySide6-Addons, PySide6
ERROR: Exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/py_compile.py", line 144, in compile
    code = loader.source_to_code(source_bytes, dfile or file,
  File "<frozen importlib._bootstrap_external>", line 918, in source_to_code
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/usr/local/lib/python3.9/site-packages/PySide6/scripts/deploy_lib/android/recipes/PySide6/__init__.tmpl.py", line 39
    {% for module in qt_modules %}  # noqa: E999
     ^
SyntaxError: invalid syntax
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/compileall.py", line 238, in compile_file
    ok = py_compile.compile(fullname, cfile, dfile, True,
  File "/usr/local/lib/python3.9/py_compile.py", line 150, in compile
    raise py_exc
py_compile.PyCompileError:   File "/usr/local/lib/python3.9/site-packages/PySide6/scripts/deploy_lib/android/recipes/PySide6/__init__.tmpl.py", line 39
    {% for module in qt_modules %}  # noqa: E999
     ^
SyntaxError: invalid syntax
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/pip/_internal/cli/base_command.py", line 105, in _run_wrapper
    status = _inner_run()
  File "/usr/local/lib/python3.9/site-packages/pip/_internal/cli/base_command.py", line 96, in _inner_run
    return self.run(options, args)
  File "/usr/local/lib/python3.9/site-packages/pip/_internal/cli/req_command.py", line 68, in wrapper
    return func(self, options, args)
  File "/usr/local/lib/python3.9/site-packages/pip/_internal/commands/install.py", line 459, in run
    installed = install_given_reqs(
  File "/usr/local/lib/python3.9/site-packages/pip/_internal/req/__init__.py", line 83, in install_given_reqs
    requirement.install(
  File "/usr/local/lib/python3.9/site-packages/pip/_internal/req/req_install.py", line 867, in install
    install_wheel(
  File "/usr/local/lib/python3.9/site-packages/pip/_internal/operations/install/wheel.py", line 728, in install_wheel
    _install_wheel(
  File "/usr/local/lib/python3.9/site-packages/pip/_internal/operations/install/wheel.py", line 614, in _install_wheel
    success = compileall.compile_file(path, force=True, quiet=True)
  File "/usr/local/lib/python3.9/compileall.py", line 255, in compile_file
    msg = err.msg.encode(sys.stdout.encoding,
TypeError: encode() argument 'encoding' must be str, not None
Cleaning up project directory and file based variables 00:01
ERROR: Job failed: exit code 1

I've noticed that the first exception happens in a code path that seems to be related to Android based on the file system path of the offending file:

  File "/usr/local/lib/python3.9/site-packages/PySide6/scripts/deploy_lib/android/recipes/PySide6/__init__.tmpl.py", line 39
    {% for module in qt_modules %}  # noqa: E999

This seems strange as the pipeline has nothing to do with Android, but I don't know if this is relevant to the issue.

Output of the succeeding pipeline with pip pinned to 25.0.1

Running with gitlab-runner 17.6.0 (374d34fd)
  on olserv05 Docker-Runner oHoJyH23J, system ID: s_343cc2df7956
Preparing the "docker" executor 00:03
Using Docker executor with image python:3.9.5 ...
Pulling docker image python:3.9.5 ...
Using docker image sha256:9b0d330dfd02b60939072b19a42eedf31064bc32ae642d4fca7468778421c787 for python:3.9.5 with digest python@sha256:2ff7f45e91d65fc36bee74e48692a5b1877c973a040ed2468aef4223edf9ccac ...
Preparing environment 00:01
Running on runner-ohojyh23j-project-1808-concurrent-0 via olserv05...
Getting source from Git repository 00:02
Fetching changes with git depth set to 20...
Initialized empty Git repository in /builds/hsa/tools/mnt/mntdatasets/.git/
Created fresh repository.
Checking out b459e379 as detached HEAD (ref is test-pip)...
Skipping Git submodules setup
Executing "step_script" stage of the job script 00:20
Using docker image sha256:9b0d330dfd02b60939072b19a42eedf31064bc32ae642d4fca7468778421c787 for python:3.9.5 with digest python@sha256:2ff7f45e91d65fc36bee74e48692a5b1877c973a040ed2468aef4223edf9ccac ...
$ pip install pip==25.0.1
Collecting pip==25.0.1
  Downloading pip-25.0.1-py3-none-any.whl (1.8 MB)
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 21.1.3
    Uninstalling pip-21.1.3:
      Successfully uninstalled pip-21.1.3
Successfully installed pip-25.0.1
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
$ pip install PySide6
Collecting PySide6
  Downloading PySide6-6.9.0-cp39-abi3-manylinux_2_28_x86_64.whl.metadata (5.5 kB)
Collecting shiboken6==6.9.0 (from PySide6)
  Downloading shiboken6-6.9.0-cp39-abi3-manylinux_2_28_x86_64.whl.metadata (2.7 kB)
Collecting PySide6-Essentials==6.9.0 (from PySide6)
  Downloading PySide6_Essentials-6.9.0-cp39-abi3-manylinux_2_28_x86_64.whl.metadata (3.9 kB)
Collecting PySide6-Addons==6.9.0 (from PySide6)
  Downloading PySide6_Addons-6.9.0-cp39-abi3-manylinux_2_28_x86_64.whl.metadata (4.2 kB)
Downloading PySide6-6.9.0-cp39-abi3-manylinux_2_28_x86_64.whl (558 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 558.1/558.1 kB 15.7 MB/s eta 0:00:00
Downloading PySide6_Addons-6.9.0-cp39-abi3-manylinux_2_28_x86_64.whl (166.3 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 166.3/166.3 MB 85.4 MB/s eta 0:00:00
Downloading PySide6_Essentials-6.9.0-cp39-abi3-manylinux_2_28_x86_64.whl (94.2 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 94.2/94.2 MB 86.9 MB/s eta 0:00:00
Downloading shiboken6-6.9.0-cp39-abi3-manylinux_2_28_x86_64.whl (206 kB)
Installing collected packages: shiboken6, PySide6-Essentials, PySide6-Addons, PySide6
Successfully installed PySide6-6.9.0 PySide6-Addons-6.9.0 PySide6-Essentials-6.9.0 shiboken6-6.9.0
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
[notice] A new release of pip is available: 25.0.1 -> 25.1
[notice] To update, run: pip install --upgrade pip
Cleaning up project directory and file based variables 00:00
Job succeeded

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[ichard26]

Ugh alright, this is 100% my fault. I can't reproduce the failure, but it's certainly caused by 502d08c. I'd confirmed that Python 3.9 compileall would handle an instance of StringIO just fine when I wrote that PR, but I missed that it was added to the 3.9 branch during its bugfix cycle. The 3.9 backport PR was only merged on July 30, 2021: python/cpython#27489

That means, according to the Python 3.9 release schedule, pip will crash if compileall raises an error while compiling a Python module on Python 3.9.6 or older.

I'll file a PR to revert the StringIO portion of that commit. I will say that this probably doesn't warrant a hotfix release since AFAICT, this will only crash if compileall fails to compile a module. If everything compiles fine, then it won't crash.

[aplzr]

Hi @ichard26, I just reran the minimal pipeline referenced above with docker images python:3.9.6 and python:3.9.7. I can't reproduce the problem with either of them.

Edit: sorry, I was too hasty and tested with 25.0.1 instead of 25.1. When running the test again with 25.1, python:3.9.6 is failing while python:3.9.7 is succeeding, confirming what you said.

[pfmoore]

I missed that it was added to the 3.9 branch during its bugfix cycle

That sucks - it could be argued that this shouldn't have been backported to 3.9, as it feels more like a new feature than a bugfix. But I guess that's water under the bridge now.

I will say that this probably doesn't warrant a hotfix release since AFAICT, this will only crash if compileall fails to compile a module. If everything compiles fine, then it won't crash.

Thanks for the analysis. I'll take your recommendation on this, but please do merge the fix, and if we do a bugfix release for other reasons, this can go into it.

[dmac-au]

I'm having a similar issue to the above in Python 3.9.2 in Debian bullseye. I have pinned to 24.1.1 as that was the last version I used on my servers but will see if 25.0.1 works, let me know when this fix is deployed so I can upgrade.

[eliehalimi]

I have the same issue on Python 3.9.2 in Debian bullseye-slim. it works fine with 25.0.1 but doesn't as of 25.1.

[Jamie-]

I've run into this too when installing gherkin-official==29.0.0 and invoke==1.7.3 on Python 3.9 in Debian Bullseye.
I can confirm that 25.0.1 works and 25.1 fails for both of these as above.

What I found interesting though is that after running pip install ... and seeing the fail, if I run the same command again, the install then passes with the message below, it does not fail again.

Requirement already satisfied: invoke==1.7.3 in /tmp/lib/python3.9/site-packages (1.7.3)