Builds broken with `poetry` 1.8

[etianen]

Following the install dependencies with poetry guide, I get the following error with poetry 1.8:

no theme named 'furo' found (missing theme.conf?)

However, the theme is clearly installed!

- Installing furo (2024.1.29)

Specifically installing poetry 1.7 with fixes the issue:

- pip install 'poetry~=1.7.0'

I don't see anything in the poetry 1.8 release notes to as a cause for the breakage. If you can pin down what the problem is, I'll gladly follow-up with an issue on the poetry issue tracker. 🙇

Details

• Read the Docs project URL: https://readthedocs.org/projects/logot/
• Build URL (if applicable): https://readthedocs.org/projects/logot/builds/23555282/
• Read the Docs username (if applicable): https://readthedocs.org/profiles/etianen/

Expected Result

The build to pass with poetry 1.8 👍

Actual Result

The build fails with poetry 1.8 👎

[ewjoachim]

Looks like we have the same issue: #11149

[ewjoachim]

I'm noting that 1.8.0 has been released in december, but it only started failing recently, so something changed, that is different in Poetry 1.8.0 and 1.7.0 but chances are it's not poetry itself.

[ewjoachim]

Hmpf, I completely misread. Poetry 1.8 was released 5 hours ago.

[ewjoachim]

So there's python-poetry/poetry#9025 for tracking the issue in Poetry.

So far, it seems it's a consequence of python-poetry/poetry#8970 that makes detection of venv more strict. Because RTD doesn't expose VIRTUAL_ENV, Poetry doesn't consider this a virtualenv, and installs to the base Python instead of the one in the venv.

Poetry folks indicate that RTD should not advocate using poetry config virtualenvs.create false, it only worked because of a bug.

So what now ?

• Should we use a different technique for installing poetry deps ? (e.g. poetry export + pip install ?)
• Should we make it so that the build sets the VIRTUAL_ENV envvar as part of the yaml instructions ?
• Should the VIRTUAL_ENV variable be set by RTD itself for every build ?

Quoting the Poetry discord:

I don't think there is an official way to use a specific python, which is not an activated venv. You can only fake an activated venv by setting PATH and VIRTUAL_ENV.

[etianen]

I'd vote for (3). I always set this when activating a venv programmatically.

…

On Sun, 25 Feb 2024 at 18:49, Joachim Jablon ***@***.***> wrote: So there's python-poetry/poetry#9025 <python-poetry/poetry#9025> for tracking the issue in Poetry. So far, it seems it's a consequence of python-poetry/poetry#8970 <python-poetry/poetry#8970> that makes detection of venv more strict. Because RTD doesn't expose VIRTUAL_ENV, Poetry doesn't consider this a virtualenv, and installs to the base Python instead of the one in the venv. Poetry folks indicate that RTD should not advocate using poetry config virtualenvs.create false, it only worked because of a bug. So what now ? - Should we use a different technique for installing poetry deps ? (e.g. poetry export + pip install ?) - Should we make it so that the build sets the VIRTUAL_ENV envvar as part of the yaml instructions ? - Should the VIRTUAL_ENV variable be set by RTD itself for every build ? — Reply to this email directly, view it on GitHub <#11150 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABEKCFDMACV6UZ55N5B74DYVOBTPAVCNFSM6AAAAABDZAMGXWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNRTGAZDOMBXGU> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[dimbleby]

Should the VIRTUAL_ENV variable be set by RTD itself for every build ?

If that means that the variable is also set at the moment that you execute pip install poetry - then this also is not correct. Poetry must be in a different environment than the project under management: else installing the project dependencies can break the environment for poetry.

I do not see a solution that can avoid changing the instructions.

(This probably means that both poetry and readthedocs will be dealing with duplicate reports of this for some time to come: until fixed instructions eventually work their way through the system...)

[etianen]

How about making pipx be available on runners (maybe it is already?!). Installing poetry via pipx is how it's done in GitHub actions, and it works like a charm!

…

On Sun, 25 Feb 2024 at 19:34, David Hotham ***@***.***> wrote: Should the VIRTUAL_ENV variable be set by RTD itself for every build ? If that means that the variable is also set at the moment that you execute pip install poetry - then this also is not correct. Poetry must be in a different environment than the project under management: else installing the project dependencies can break the environment for poetry. I do not see a solution that can avoid changing the instructions. (This probably means that both poetry and readthedocs will be dealing with duplicate reports of this for some time to come: until fixed instructions eventually work their way through the system...) — Reply to this email directly, view it on GitHub <#11150 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABEKCBSA4BROAK3O5FM5ADYVOG35AVCNFSM6AAAAABDZAMGXWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNRTGAZTQNRRHA> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[humitos]

How about making pipx be available on runners (maybe it is already?!).

It's not, but it can be installed via

python3 -m pip install --user pipx
python3 -m pipx ensurepath

(from https://pipx.pypa.io/stable/#on-linux)

If that's the best way to install and execute poetry, we can update the documentation to reflect that.

[humitos]

Poetry folks indicate that RTD should not advocate using poetry config virtualenvs.create false

This is required because there are other Python commands executed by Read the Docs (e.g. sphinx-build) that need to be executed using the current virtualenv where all the requirements were installed --that's why poetry and the requirements for building the documentation has to be installed in the same virtualenv.

These commands are executed by Read the Docs,

python -m pip install --upgrade --no-cache-dir pip setuptools              
python -m pip install --upgrade --no-cache-dir sphinx readthedocs-sphinx-ext
python -m sphinx -T -b html -d _build/doctrees -D language=en . $READTHEDOCS_OUTPUT/html

and these should work without any modification. Note that we cannot prepend them with pipx run or similar.

[etianen]

Installing via pipx is one of the recommended ways to install poetry: https://python-poetry.org/docs/ (Sorry I can't link more specifically on mobile, docs seem to be a SPA.) So a pipx installation coupled with the additional venv environmental variable would work. It would be lovely to have pipx already installed. It's very mainstream and opens the door to installing all sorts of CLI tools without polluting the main venv. GHA comes with it installed, for example.

…

On Sun, 25 Feb 2024 at 20:27, Manuel Kaufmann ***@***.***> wrote: How about making pipx be available on runners (maybe it is already?!). It's not, but it can be installed via python3 -m pip install --user pipx python3 -m pipx ensurepath *(from https://pipx.pypa.io/stable/#on-linux <https://pipx.pypa.io/stable/#on-linux>)* If that's the best way to install and execute poetry, we can update the documentation to reflect that. — Reply to this email directly, view it on GitHub <#11150 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABEKCDBLQFA2YLJLRW2TJDYVONCJAVCNFSM6AAAAABDZAMGXWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNRTGA2TCNJYGQ> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[humitos]

I'm happy to update our documentation with a different solution if you have one that does work for this particular use case.

It would be lovely to have pipx already installed

Unfortunately, we can't do this in the short term at least; but it's easy to install it as I showed before.

[humitos]

@ewjoachim

Poetry folks indicate that RTD should not advocate using poetry config virtualenvs.create false, it only worked because of a bug.

BTW, do you know what's the bug that they fixed that changed this behavior? Their documentation for virtualenvs.create (https://python-poetry.org/docs/configuration#virtualenvscreate) still says that it should work in the way we expect to work: don't create a new virtualenv when installing dependencies --but it seems it doesn't work like that anymore.