Poetry Update always re-installs *all* dependencies -- again!

[philipsd6]

• I am on the latest Poetry version. (The very latest!)
• I have searched the issues of this repo and believe that this is not a duplicate. (Well actually, it is, but I can't reopen a closed issue.)
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: Ubuntu 21.10 (impish)
• Poetry version: 1.2.0a2
• Link of a Gist with the contents of your pyproject.toml file: pyproject.toml

Issue

This is the same symptom reported in #2534, but that was closed, and this is with the latest alpha. Running poetry update reinstalls the same versions of all the dependencies:

philipsd6@testvm ~/d/t/testing-poetry> poetry install
Creating virtualenv testing-poetry-7qidRo_f-py3.10 in /home/philipsd6/.cache/pypoetry/virtualenvs
Updating dependencies
Resolving dependencies... (21.1s)

Writing lock file

Package operations: 30 installs, 0 updates, 0 removals

  • Installing backports.entry-points-selectable (1.1.0)
  • Installing cached-property (1.5.2)
  • Installing distlib (0.3.3)
  • Installing filelock (3.3.1)
  • Installing platformdirs (2.4.0)
  • Installing six (1.16.0)
  • Installing typing-extensions (3.10.0.2)
  • Installing asgiref (3.4.1)
  • Installing aspy.refactor-imports (2.2.0)
  • Installing cfgv (3.3.1)
  • Installing click (8.0.3)
  • Installing h11 (0.12.0)
  • Installing identify (2.3.1)
  • Installing mypy-extensions (0.4.3)
  • Installing nodeenv (1.6.0)
  • Installing pathspec (0.9.0)
  • Installing pydantic (1.8.2)
  • Installing pyyaml (6.0)
  • Installing regex (2021.10.23)
  • Installing ruamel.yaml (0.17.16)
  • Installing starlette (0.14.2)
  • Installing toml (0.10.2)
  • Installing tomli (1.2.1)
  • Installing virtualenv (20.9.0)
  • Installing black (21.9b0)
  • Installing fastapi (0.68.2)
  • Installing pre-commit (2.15.0)
  • Installing pre-commit-hooks (4.0.1)
  • Installing reorder-python-imports (2.6.0)
  • Installing uvicorn (0.15.0)

philipsd6@testvm ~/d/t/testing-poetry> poetry update
Updating dependencies
Resolving dependencies... (14.9s)

Package operations: 0 installs, 30 updates, 0 removals

  • Updating backports.entry-points-selectable (1.1.0 /home/philipsd6/.cache/pypoetry/artifacts/fd/b4/52/733441a9d7392e38759bc000f27ec89e3792f2c85e7ef7459875b29689/backports.entry_points_selectable-1.1.0-py2.py3-none-any.whl -> 1.1.0)
  • Updating cached-property (1.5.2 /home/philipsd6/.cache/pypoetry/artifacts/9a/ce/00/993ada1ed134ec75f82f4039e6a8b8496e9c7ef5ca99aebf446c9480a3/cached_property-1.5.2-py2.py3-none-any.whl -> 1.5.2)
  • Updating distlib (0.3.3 /home/philipsd6/.cache/pypoetry/artifacts/89/ca/13/f159abbc862ecf53f9e9df45085cb68d453e528222b7a482b224800878/distlib-0.3.3-py2.py3-none-any.whl -> 0.3.3)
  • Updating filelock (3.3.1 /home/philipsd6/.cache/pypoetry/artifacts/87/16/bb/bf772a0fb18f2ac9af1bfe64df668b2fa5239757aaed3ec9476b3f0731/filelock-3.3.1-py3-none-any.whl -> 3.3.1)
  • Updating platformdirs (2.4.0 /home/philipsd6/.cache/pypoetry/artifacts/e7/bd/93/75d8df177f4ad6ada153c8c7ec5b3894cb698333a51ddd6071f7b7f74a/platformdirs-2.4.0-py3-none-any.whl -> 2.4.0)
  • Updating six (1.16.0 /home/philipsd6/.cache/pypoetry/artifacts/e6/5b/74/61941a3d223fc41bf7621167df83f0eedef520cd021f62404cc2481d36/six-1.16.0-py2.py3-none-any.whl -> 1.16.0)
  • Updating typing-extensions (3.10.0.2 /home/philipsd6/.cache/pypoetry/artifacts/c1/e7/9a/f29765b6dcb2022436353e8e1d67fc04432f104e8583c54b995acf1000/typing_extensions-3.10.0.2-py3-none-any.whl -> 3.10.0.2)
  • Updating asgiref (3.4.1 /home/philipsd6/.cache/pypoetry/artifacts/7b/5e/5f/20bf06e839b09b168768c01a88c3461b24adb4a4689a5d92af41499fca/asgiref-3.4.1-py3-none-any.whl -> 3.4.1)
  • Updating aspy.refactor-imports (2.2.0 /home/philipsd6/.cache/pypoetry/artifacts/7b/6b/3c/5dbd479cd09c42f474d0bfc3a662a4d8ea5828879a1f80b3c13df5b4e4/aspy.refactor_imports-2.2.0-py2.py3-none-any.whl -> 2.2.0)
  • Updating cfgv (3.3.1 /home/philipsd6/.cache/pypoetry/artifacts/f6/cf/5e/a23c7a442f58abf93f5b40f81986ab47f43b021c081020645197772a3f/cfgv-3.3.1-py2.py3-none-any.whl -> 3.3.1)
  • Updating click (8.0.3 /home/philipsd6/.cache/pypoetry/artifacts/1a/87/db/9b4d416b0982de26aa207a5b530917c76b1b7439f72a4773aca942d315/click-8.0.3-py3-none-any.whl -> 8.0.3)
  • Updating h11 (0.12.0 /home/philipsd6/.cache/pypoetry/artifacts/53/1f/93/d0ae2e18e312958649c53b791adcb80b1575e47dddeef29651ec1c1224/h11-0.12.0-py3-none-any.whl -> 0.12.0)
  • Updating identify (2.3.1 /home/philipsd6/.cache/pypoetry/artifacts/96/ee/a6/1c4bb0f20e402d0b39478bfa950758de6a477582676dd7fb3d028623cb/identify-2.3.1-py2.py3-none-any.whl -> 2.3.1)
  • Updating mypy-extensions (0.4.3 /home/philipsd6/.cache/pypoetry/artifacts/4f/6f/d0/c461463ab1f56dc3f6545dd9674717ab5cca5e7282440baf533ad5fef5/mypy_extensions-0.4.3-py2.py3-none-any.whl -> 0.4.3)
  • Updating nodeenv (1.6.0 /home/philipsd6/.cache/pypoetry/artifacts/00/8a/8a/794fdccd7e4f8b25f9721ad0ee1a131aba981cbedbc8e3fc5d77f498c4/nodeenv-1.6.0-py2.py3-none-any.whl -> 1.6.0)
  • Updating pathspec (0.9.0 /home/philipsd6/.cache/pypoetry/artifacts/af/62/31/1144a446465869affd13338fb8295d81390edb27ca5d11c96c239b7c48/pathspec-0.9.0-py2.py3-none-any.whl -> 0.9.0)
  • Updating pydantic (1.8.2 /home/philipsd6/.cache/pypoetry/artifacts/0a/4b/bf/b36af9dddc4cfc7f52a9aef7b7fa54ace5b6bb486bd5032330fc2017e4/pydantic-1.8.2-py3-none-any.whl -> 1.8.2)
  • Updating pyyaml (6.0 /home/philipsd6/.cache/pypoetry/artifacts/3d/cd/04/0f3b92a70b015a9919b1a461301ee77442ea9a611defeb7b4939731158/PyYAML-6.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl -> 6.0)
  • Updating regex (2021.10.23 /home/philipsd6/.cache/pypoetry/artifacts/e2/94/5e/f37575ed6b129be5456c44320f0d0fd13a19044832d58209f3f1018a7b/regex-2021.10.23-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl -> 2021.10.23)
  • Updating ruamel.yaml (0.17.16 /home/philipsd6/.cache/pypoetry/artifacts/4f/1e/ac/f14d1643f8b5504b87609f8c6f18db6ac4ba08b270e1bf7de4a5f46190/ruamel.yaml-0.17.16-py3-none-any.whl -> 0.17.16)
  • Updating starlette (0.14.2 /home/philipsd6/.cache/pypoetry/artifacts/62/69/ae/8e2ae7bb032dca64e3cb90f8574872646895d03e57531080751568987b/starlette-0.14.2-py3-none-any.whl -> 0.14.2)
  • Updating toml (0.10.2 /home/philipsd6/.cache/pypoetry/artifacts/b9/71/b6/c5f38fe37b701edd09651f93b93090d624ad05b438321c4adb0a143470/toml-0.10.2-py2.py3-none-any.whl -> 0.10.2)
  • Updating tomli (1.2.1 /home/philipsd6/.cache/pypoetry/artifacts/e3/bc/44/3b4087800f77e8f24aa6cb919c8bda7349a47c174b8771f83014ecd39e/tomli-1.2.1-py3-none-any.whl -> 1.2.1)
  • Updating virtualenv (20.9.0 /home/philipsd6/.cache/pypoetry/artifacts/e6/77/16/9920019604e7e5b8ce863e555816ad7d8524662623b37e49c396f30dfa/virtualenv-20.9.0-py2.py3-none-any.whl -> 20.9.0)
  • Updating black (21.9b0 /home/philipsd6/.cache/pypoetry/artifacts/66/99/c2/9c8a81799e6f193d7c841366cb21988eaaa1342589911ea31d116ba349/black-21.9b0-py3-none-any.whl -> 21.9b0)
  • Updating fastapi (0.68.2 /home/philipsd6/.cache/pypoetry/artifacts/aa/52/7f/c3020e310376cd98099f8b4a56e30e8c2ef966e78e161466f17a98e537/fastapi-0.68.2-py3-none-any.whl -> 0.68.2)
  • Updating pre-commit (2.15.0 /home/philipsd6/.cache/pypoetry/artifacts/ae/f1/25/50d98351fc400c5d75960b7fafe00682853a3bb895fcf3bf34ec289c6d/pre_commit-2.15.0-py2.py3-none-any.whl -> 2.15.0)
  • Updating pre-commit-hooks (4.0.1 /home/philipsd6/.cache/pypoetry/artifacts/9a/8a/1e/1bc8d7989188ae2d59dfccaebac974738fcadeb9157d40c4906fdeee02/pre_commit_hooks-4.0.1-py2.py3-none-any.whl -> 4.0.1)
  • Updating reorder-python-imports (2.6.0 /home/philipsd6/.cache/pypoetry/artifacts/e3/db/a0/b30fa57ae5e7f515385d9f9d2bc9a9fc59ceca55f5bb082fc09924ceec/reorder_python_imports-2.6.0-py2.py3-none-any.whl -> 2.6.0)
  • Updating uvicorn (0.15.0 /home/philipsd6/.cache/pypoetry/artifacts/82/01/49/abffbbfb4ef56176605e91cdbe8b0214dd1363f00ec9f242e9a2af0737/uvicorn-0.15.0-py3-none-any.whl -> 0.15.0)

[philipsd6]

I have the output of `poetry install -vvv' in a gist but I don't see any errors there.

[abn]

This typically happens (based on your output) when poetry thinks that the existing package was installed as file dependency and the new lock file tells it that it should be installed as a regular dependency, hence the (<version> < cached wheel file path> -> <version>). Was should not happen anymore with the changes to editable installer.

I attempted to reproduce this using the following, but was not successful. Am I missing something?

podman run --rm -i --entrypoint bash python:3.9 <<EOF
set -xe
python -m pip install -q --pre poetry==1.2.0a2
poetry --version
install -d foobar
pushd foobar
cat > pyproject.toml <<TOML
[tool.poetry]
name = "testing-poetry"
description = "poetry update reinstalls same versions"
version = "0.1.0"
authors = ["Me <my-email@example.com>"]
readme = "README.md"

[tool.poetry.dependencies]
python = "^3.9"
fastapi = "^0.68.0"
pydantic = "^1.8.2"
uvicorn = "^0.15.0"

[tool.poetry.dev-dependencies]
black = "^21.9b0"
pre-commit = "^2.15.0"
pre-commit-hooks = "^4.0.1"
reorder-python-imports = "^2.6.0"

[build-system]
requires = ["poetry>=1.0.0"]
build-backend = "poetry.core.masonry.api"
TOML
poetry install
poetry update
EOF

If not, then it could be something to do with your environment. I'd recommend nuking the virtual environment and recreating it.

[abn]

The only variable I could think of here is the custom repository usage. (Related: #4385)

[abn]

Might also be related to #4669.

[philipsd6]

Duplicate of #4385

I ran your podman test, and it works fine until I add:

[[tool.poetry.source]]
name = "artifactory"
default = true
url = "https://our-internal-repository/api/pypi/pypi/simple"

...and then it shows the behavior I described, and as described in #4385

[valkenburg]

I find that if I replace default = true by secondary = true, only the packages from the custom source are installed repeatedly, all packages from pypi.org are correctly skipped.

Tracing this down, I find that only those dependencies that have a section

[package.source]
type = "legacy"
url = "..."
reference = "..."

in poetry.lock, get re-installed upon every call to poetry install.

For now hence I use secondary = true to minimize the number of re-installs, but that really is not a solution.

[radoering]

Duplicate: #5360
Resolved by: #5362

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.