Skip to content

Conversation

@sandydoo
Copy link
Member

@sandydoo sandydoo commented Oct 22, 2025

This prepares us for the arrival of several upstream patches (including NixOS/nixpkgs#442540) that will make it possible to drop the python wrapper.

This also changes the way the old wrapper is applied.

  • withPackages are now propagated through the wrapper
  • We no longer override LD_LIBRARY_PATH, which should resolve a bunch of issues with python subprocesses. Okay, this one was too good to be true

TODO: adjust the issue list
Fixes #1807.
Fixes #1792.
Fixes #1486. (possibly earlier)
Fixes #1335.
Fixes #1218.
Fixes #1111.
Fixes #773.

@sandydoo sandydoo added the bug Something isn't working label Oct 22, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Oct 22, 2025

🔍 Suggested Reviewers

Based on git blame analysis of the changed lines, the following contributors have significant experience with the modified code:

  • @domenkozar - 46.3% of changed lines (19 lines)
  • @gabyx - 39.0% of changed lines (16 lines)

Please consider reviewing this PR as you have authored significant portions of the code being modified. Your expertise would be valuable! 🙏

This comment was automatically generated by git-blame-auto-reviewer

Last updated: 2025-10-23T16:49:17.396Z

@sandydoo sandydoo changed the title python: prepare to drop legacy venv wrapper python: prepare to drop legacy venv wrapper and adjust how the legacy wrapper is applied Oct 22, 2025
@cloudflare-workers-and-pages
Copy link

cloudflare-workers-and-pages bot commented Oct 22, 2025

Deploying devenv with  Cloudflare Pages  Cloudflare Pages

Latest commit: 74f437e
Status: ✅  Deploy successful!
Preview URL: https://f216645d.devenv.pages.dev
Branch Preview URL: https://fix-1218.devenv.pages.dev

View logs

@@ -582,7 +657,7 @@ in
};

enterShell = ''
export PYTHONPATH="$DEVENV_PROFILE/${package.sitePackages}''${PYTHONPATH:+:$PYTHONPATH}"
export PYTHONPATH="$DEVENV_PROFILE/${cfg.package.sitePackages}''${PYTHONPATH:+:$PYTHONPATH}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: Why do we even need to set PYTHONPATH. We should never tinker with that if possible?

Copy link
Contributor

@gabyx gabyx Oct 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As I see this resolves to DEVENV_PROFILE/lib/python.../site-packages. for what is that path and what does it contain?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No clue. Maybe an old attempt at doing what the wrapper does?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

f151daf#diff-f5bb89f776eda1a758db67c6c75b19ff1665b2f10998d17d1d98479835fc9e2eR482

This is from a large tasks PR. Let's try removing it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PYTHONPATH is critical in making site packages (withPackages) available to venvs created with venv.enable.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah weird: ok, so its an empty folder which gets filled by venv.enable I guess

{
languages.python = {
enable = true;
package = pkgs.python3.withPackages (ps: [
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

awesome!

@sandydoo sandydoo changed the title python: prepare to drop legacy venv wrapper and adjust how the legacy wrapper is applied python: prepare to drop legacy wrapper and adjust how it's applied Oct 23, 2025
@sandydoo
Copy link
Member Author

Added a test to verify sys.base_prefix and the interaction between withPackages and venv.enable.

We now also patch buildEnv in a way that avoids nesting envs.

This PR still needs to be tested against the upstream nixpkgs PR. Once it's out of staging, we can verify this easily without rebuilding the universe.

ps.tkinter
]);
patches.buildEnv.enable = true;
venv.enable = true;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: could we add a uv test as well? with uv.enable = true and uv.sync = true or so

pyproject.toml

[project]
name = "mymodule"
description = "test"
version = "0.0.1"

dependencies = [
    "request",
    "pytest",
]

# Check that sys.base_prefix points to the -env buildEnv, not the bare interpreter
assert "-env" in sys.base_prefix, \
f"sys.base_prefix ({sys.base_prefix}) should point to python-env with packages, not bare interpreter"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome!

@domenkozar
Copy link
Member

Shall we merge this?

@gabyx
Copy link
Contributor

gabyx commented Nov 2, 2025

suggestion: we need another uv test as uv is the predominant tool
in the future

@gabyx
Copy link
Contributor

gabyx commented Nov 2, 2025

but should not hold it back!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

4 participants