2.2.2: codespell script installed in wrong location

[kloczek]

During package 2.2.2 I found that codespell script is not installed in bindir.

Provides: codespell = 2.2.2-3.fc35 python3.8dist(codespell) = 2.2.2 python3dist(codespell) = 2.2.2
Requires(rpmlib): rpmlib(CompressedFileNames) <= 3.0.4-1 rpmlib(FileDigests) <= 4.6.0-1 rpmlib(PayloadFilesHavePrefix) <= 4.0-1
Requires: /usr/bin/python3 python(abi) = 3.8
Checking for unpackaged file(s): /usr/lib/rpm/check-files /home/tkloczko/rpmbuild/BUILDROOT/codespell-2.2.2-3.fc35.x86_64
error: Installed (but unpackaged) file(s) found:
   /usr/lib/python3.8/site-packages/bin/codespell

[kloczek]

Looks as well like test suite is installed as well (conten of the codespell_lib/tests/)

[kloczek]

Because in pyproject.toml already is [project.scripts] section first issue can be solved by delete bin/ directiry and include bin/codespell from MANIFEST.in.

[DimitriPapadopoulos]

Is this an issue with codespell itself, or the Fedora package of codespell?

[kloczek]

I'm not using Fedora.
Fedora package is not using pep517 build procedure which I'm using to generate .whl archive.
If you want to test that procedure please use python3 -sBm build -w --no-isolation command to testwhat lands in generated .whl archove.

[kloczek]

Just checked and in Fedora spec file in %install is line which removes test suite from buildroot.
https://src.fedoraproject.org/rpms/codespell/blob/rawhide/f/codespell.spec#_33

[DimitriPapadopoulos]

Sorry, I noticed the fc35 in Provides: codespell = 2.2.2-3.fc35 and assumed this was somehow occurring while packaging on or for Fedora 35, or something similar. My wrong.

I am not sure how to reproduce this issue, or where to find documentation on duplicates in MANIFEST.in and pyproject.toml. Hopefully, someone else will review this.

[kloczek]

Sorry, I noticed the fc35 in Provides: codespell = 2.2.2-3.fc35 and assumed this was somehow occurring while packaging on or for Fedora 35, or something similar. My wrong.

And my wrong as well. I shoud change %dist macro value as well to not confuse peopel like you 😋

I am not sure how to reproduce this issue, or where to find documentation on duplicates in MANIFEST.in and pyproject.toml. Hopefully, someone else will review this.

My typical build proceduyte which uses pap517 consist from:

• python3 -sBm build -w --no-isolation
• because I'm calling build with --no-isolation I'm using during all processes only locally installed modules
• install .whl file in </install/prefix>
• run pytest with PYTHONPATH pointing to sitearch and sitelib inside </install/prefix>

[DimitriPapadopoulos]

The command line option -s is not an option on my machine:

Logs of "python3.8 -Bm build -w --no-isolation", directly on Ubuntu 18.04

$ python3.8 -sBm build -w --no-isolation
/usr/bin/python3.8: No module named build
$ 
$ python3.8 -Bm build -w --no-isolation
[...]
adding 'bin/codespell'
adding 'codespell_lib/__init__.py'
adding 'codespell_lib/__main__.py'
adding 'codespell_lib/_codespell.py'
adding 'codespell_lib/_version.py'
adding 'codespell_lib/data/__init__.py'
adding 'codespell_lib/data/dictionary.txt'
adding 'codespell_lib/data/dictionary_code.txt'
adding 'codespell_lib/data/dictionary_en-GB_to_en-US.txt'
adding 'codespell_lib/data/dictionary_informal.txt'
adding 'codespell_lib/data/dictionary_names.txt'
adding 'codespell_lib/data/dictionary_rare.txt'
adding 'codespell_lib/data/dictionary_usage.txt'
adding 'codespell_lib/data/linux-kernel.exclude'
adding 'codespell_lib/tests/__init__.py'
adding 'codespell_lib/tests/test_basic.py'
adding 'codespell_lib/tests/test_dictionary.py'
adding 'codespell-2.2.3.dev65+g3f05fd62.dist-info/METADATA'
adding 'codespell-2.2.3.dev65+g3f05fd62.dist-info/WHEEL'
adding 'codespell-2.2.3.dev65+g3f05fd62.dist-info/entry_points.txt'
adding 'codespell-2.2.3.dev65+g3f05fd62.dist-info/top_level.txt'
adding 'codespell-2.2.3.dev65+g3f05fd62.dist-info/RECORD'
removing build/bdist.linux-x86_64/wheel
Successfully built codespell-2.2.3.dev65+g3f05fd62-py3-none-any.whl
$ 
$ unzip -t dist/codespell-2.2.3.dev65+g3f05fd62-py3-none-any.whl 
Archive:  dist/codespell-2.2.3.dev65+g3f05fd62-py3-none-any.whl
    testing: bin/codespell            OK
    testing: codespell_lib/__init__.py   OK
    testing: codespell_lib/__main__.py   OK
    testing: codespell_lib/_codespell.py   OK
    testing: codespell_lib/_version.py   OK
    testing: codespell_lib/data/__init__.py   OK
    testing: codespell_lib/data/dictionary.txt   OK
    testing: codespell_lib/data/dictionary_code.txt   OK
    testing: codespell_lib/data/dictionary_en-GB_to_en-US.txt   OK
    testing: codespell_lib/data/dictionary_informal.txt   OK
    testing: codespell_lib/data/dictionary_names.txt   OK
    testing: codespell_lib/data/dictionary_rare.txt   OK
    testing: codespell_lib/data/dictionary_usage.txt   OK
    testing: codespell_lib/data/linux-kernel.exclude   OK
    testing: codespell_lib/tests/__init__.py   OK
    testing: codespell_lib/tests/test_basic.py   OK
    testing: codespell_lib/tests/test_dictionary.py   OK
    testing: codespell-2.2.3.dev65+g3f05fd62.dist-info/METADATA   OK
    testing: codespell-2.2.3.dev65+g3f05fd62.dist-info/WHEEL   OK
    testing: codespell-2.2.3.dev65+g3f05fd62.dist-info/entry_points.txt   OK
    testing: codespell-2.2.3.dev65+g3f05fd62.dist-info/top_level.txt   OK
    testing: codespell-2.2.3.dev65+g3f05fd62.dist-info/RECORD   OK
No errors detected in compressed data of dist/codespell-2.2.3.dev65+g3f05fd62-py3-none-any.whl.
$ 

I do not see bin/codespell missing from the resulting wheel.

I do see the tests in the resulting wheel, but I don't know whether they should be excluded or not from binary distributions such as wheels. The Python Packaging User Guide seems to imply they should not be excluded from source distributions at least:

The following files are included in a source distribution by default:

• ...
• all files matching the pattern test/test*.py

I am surprised changing MANIFEST.in could affect binary distributions such as wheels, because the Python Packaging User Guide states:

Note: MANIFEST.in does not affect binary distributions such as wheels.

I cannot keep the current pace of changes in Python packaging, it is a poorly documented mess. Do you know of any documentation on migrating from setup.py to pyproject.toml and how MANIFEST.in is supposed to evolve in this context?

[DimitriPapadopoulos]

To clarify, is this a regression, meaning you can reproduce this issue with version 2.2.2 but not earlier versions?

[kloczek]

To clarify, is this a regression, meaning you can reproduce this issue with version 2.2.2 but not earlier versions?

Yes. Prev version had no such issues.

[DimitriPapadopoulos]

Have you made sure pip, setuptools and wheel are updated to the latest versions before packaging?

As I wrote, I cannot easily reproduce this issue on my machine (maybe because I cannot use the -s option), and the documentation suggests MANIFEST.in does not affect binary distributions such as wheels.

[kloczek]

Have you made sure pip, setuptools and wheel are updated to the latest versions before packaging?

Here is list of modules with versions installed in build env

Package              Version
-------------------- --------------
appdirs              1.4.4
attrs                22.1.0
Brlapi               0.8.3
build                0.9.0
chardet              5.0.0
contourpy            1.0.6
cssselect            1.1.0
cycler               0.11.0
distro               1.8.0
dnspython            2.2.1
extras               1.0.0
fixtures             4.0.0
fonttools            4.38.0
gpg                  1.17.1-unknown
importlib-resources  5.9.0
iniconfig            1.1.1
kiwisolver           1.4.4
libcomps             0.1.19
louis                3.23.0
lxml                 4.9.1
matplotlib           3.6.2
numpy                1.23.1
olefile              0.46
packaging            21.3
pbr                  5.9.0
pep517               0.13.0
Pillow               9.3.0
pip                  22.2.2
pkgutil_resolve_name 1.3.10
platformdirs         2.5.2
pluggy               1.0.0
py                   1.11.0
PyGObject            3.42.2
pyparsing            3.0.9
pyrsistent           0.19.2
pytest               7.1.3
python-dateutil      2.8.2
rpm                  4.17.0
scour                0.38.2
setuptools           65.5.1
setuptools-scm       7.0.5
six                  1.16.0
testtools            2.5.0
tomli                2.0.1
traitlets            5.4.0
typing_extensions    4.3.0
wheel                0.37.1
zipp                 3.9.0

[DimitriPapadopoulos]

Version 2.2.2 moved from setup.py to setuptools.toml, see 754b9a5 / #2523. Chances are this issue is related. Unfortunately, I cannot reproduce it.

[DimitriPapadopoulos]

Again, my suggestion is to update Python packages. In your case, pip can be upgraded from 22.2.2 to 22.3.1, and
wheel from 0.37.1 to 0.38.4. Please do try these suggestions before reporting back.

See Python Packaging User Guide:

Ensure pip, setuptools, and wheel are up to date

While pip alone is sufficient to install from pre-built binary archives, up to date copies of the setuptools and wheel projects are useful to ensure you can also install from source archives:

python3 -m pip install --upgrade pip setuptools wheel