Add dependency-group cli flags to `uv pip install` and `uv pip compile` (`--group`, `--no-group`, `--only-group`, `--all-groups`) #10861

Gankra · 2025-01-22T14:55:18Z

Ultimately this is a lot of settings plumbing and a couple minor pieces of Actual Logic (which are so simple I have to assume there's something missing, but maybe not!).

Note this "needlessly" use DevDependencyGroup since it costs nothing, is more futureproof, and lets us maintain one primary interface (we just pass false for all the dev arguments).

Fixes #8590
Fixes #8969

Gankra · 2025-01-22T14:57:40Z

No description provided.

Gankra · 2025-01-22T15:06:53Z

crates/uv-settings/src/settings.rs

+    /// Include optional dependencies from the specified group; may be provided more than once.
+    ///
+    /// Only applies to `pyproject.toml`, `setup.py`, and `setup.cfg` sources.
+    #[option(
+        default = "[]",
+        value_type = "list[str]",
+        example = r#"
+            group = ["dev", "docs"]
+        "#
+    )]
+    pub group: Option<Vec<GroupName>>,


Whether these were supposed to pass through the shared PipOptions and not a side-thing was an uncertainty for me. Overall I assumed they're equivalentish to the extra settings and those are here, so, seems fine?

Seems right to me, though the option abstractions are not my forte.

crates/uv-cli/src/lib.rs

zanieb · 2025-01-22T17:05:00Z

crates/uv-configuration/src/dev.rs

+    /// Returns `true` if the specification will have no effect.
+    pub fn is_empty(&self) -> bool {
+        let GroupsSpecification::Include {
+            include: IncludeGroups::Some(includes),
+            exclude,
+        } = self
+        else {
+            return false;
+        };
+        includes.is_empty() && exclude.is_empty()
+    }
 }


I'm not entirely sure about the correctness of this. Is this relying on some assumptions about the construction of GroupsSpecification?

It is, but it's a relatively mundane assumption imo.

Might be worth a comment saying what the assumption is, but don't feel strongly.

crates/uv-requirements/src/sources.rs

zanieb · 2025-01-22T17:07:28Z

crates/uv-settings/src/settings.rs

+    /// Include optional dependencies from the specified group; may be provided more than once.
+    ///
+    /// Only applies to `pyproject.toml`, `setup.py`, and `setup.cfg` sources.
+    #[option(
+        default = "[]",
+        value_type = "list[str]",
+        example = r#"
+            group = ["dev", "docs"]
+        "#
+    )]
+    pub group: Option<Vec<GroupName>>,


Seems right to me, though the option abstractions are not my forte.

zanieb · 2025-01-22T17:10:05Z

crates/uv/tests/it/pip_install.rs

 }
+
+#[test]
+fn install_group() -> Result<()> {


What happens if multiple pyproject.toml files are provided? What happens if a group is present in one but not the other? If a group is present in both?

Similarly, what about uv pip install <dir> --group <name>?

I believe as currently implemented we:

don't complain if we fail to find a group in the toml

would independently match and include groups from every toml

This falls out of the implementation basically being "for each sourcefile, for each group in that sourcefile, if the selector built from cli flags matches the name of this group, append that group to the requirements".

not sure about the dir usecase

I think we should fail (or at the very least, warn) if we cannot find a group.

I don't mind including groups from every toml, but we should have test coverage.

We fail in uv sync fwiw

❯ uv sync --group test error: Group `test` is not defined in the project's `dependency-group` table

Added warnings.

zanieb · 2025-01-22T17:13:24Z

docs/reference/settings.md

+#### [`all-groups`](#pip_all-groups) {: #pip_all-groups }
+<span id="all-groups"></span>
+
+Include all groups.
+
+Only applies to `pyproject.toml`, `setup.py`, and `setup.cfg` sources.
+
+**Default value**: `false`
+
+**Type**: `bool`
+
+**Example usage**:
+
+=== "pyproject.toml"
+
+    ```toml
+    [tool.uv.pip]
+    all-groups = true
+    ```
+=== "uv.toml"
+
+    ```toml
+    [pip]
+    all-groups = true
+    ```
+
+---
+


Honestly I'm not sure why we support requesting extras in settings file like this. It seems correct to match the extras options for groups though?

Perhaps @charliermarsh understands the use-case?

qvalentin · 2025-01-23T14:12:09Z

Thanks @Gankra

henryiii · 2025-01-23T16:54:26Z

Not that this is does not match the proposed API for pip, so pip and uv pip would diverge. pypa/pip#13065 is going to be merged soon.

Gankra · 2025-01-23T17:06:38Z

Wow incredible timing. This won't get released in uv until we're happy with our interop story (will back this out if need be).

zanieb · 2025-01-23T17:08:10Z

This is intended to be compatible with the proposed pip interface, i.e., we can implement it once it's released. We have to deal with a bit more complexity because we allow -r pyproject.toml which pip has said they will not support.

Filimoa · 2025-01-31T15:19:19Z

Is there another way of compiling development groups dependencies into a requirements.txt if this is no longer planned? I don't see anything in the docs or issues so just wondering if it's possible.

charliermarsh · 2025-01-31T15:24:19Z

If you use uv lock, you can then uv export into a requirements.txt today. (We still plan to ship this, we just deferred it for a bit.)

henryiii · 2025-01-31T15:51:13Z

And pypa/pip#13065 should be about ready to go in, it's already got an approval. You can also use uvx dependency-groups > requirements.txt if you want them unlocked.

…pile` (#11686) This is a minimal redux of #10861 to be compatible with `uv pip`. This implements the interface described in: pypa/pip#13065 (comment) for `uv pip install` and `uv pip compile`. Namely `--group <[path:]name>`, where `path` when not defined defaults to `pyproject.toml`. In that interface they add `--group` to `pip install`, `pip download`, and `pip wheel`. Notably we do not define `uv pip download` and `uv pip wheel`, so for parity we only need to implement `uv pip install`. However, we also support `uv pip compile` which is not part of pip itself, and `--group` makes sense there too. ---- The behaviour of `--group` for `uv pip` commands makes sense for the cases upstream pip supports, but has confusing meanings in cases that only we support (because reading pyproject.tomls is New Tech to them but heavily supported by us). **Specifically case (h) below is a concerning footgun, and case (e) below may get complaints from people who aren't well-versed in dependency-groups-as-they-pertain-to-wheels.** ## Only Group Flags Group flags on their own work reasonably and uncontroversially, except perhaps that they don't do very clever automatic project discovery. a) `uv pip install --group path/to/pyproject.toml:mygroup` pulls up `path/to/project.toml` and installs all the packages listed by its `mygroup` dependency-group (essentially treating it like another kind of requirements.txt). In this regard it functions similarly to `--only-group` in the rest of uv's interface. b) `uv pip install --group mygroup` is just sugar for `uv pip install --group pyproject.toml:mygroup` (**note that no project discovery occurs**, upstream pip simply hardcodes the path "pyproject.toml" here and we reproduce that.) c) `uv pip install --group a/pyproject.toml:groupx --group b/pyproject.toml:groupy`, and any other instance of multiple `--group` flags, can be understood as completely independent requests for the given groups at the given files. ## Groups With Named Packages Groups being mixed with named packages also work in a fairly unsurprising way, especially if you understand that things like dependency-groups are not really supposed to exist on pypi, they're just for local development. d) `uv pip install mypackage --group path/to/pyproject.toml:mygroup` much like multiple instances of `--group` the two requests here are essentially completely independent: pleases install `mypackage`, and please also install `path/to/pyproject.toml:mygroup`. e) `uv pip install mypackage --group mygroup` is exactly the same, but this is where it becomes possible for someone to be a little confused, as you might think `mygroup` is supposed to refer to `mypackage` in some way (it can't). But no, it's sourcing `pyproject.toml:mygroup` from the current working directory. ## Groups With Requirements/Sourcetrees/Editables Requirements and sourcetrees are where I expect users to get confused. It behaves *exactly* the same as it does in the previous sections but you would absolutely be forgiven for expecting a different behaviour. *Especially* because `--group` with the rest of uv *does* do something different. f) `uv pip install -r a/pyproject.toml --group b/pyproject.toml:mygroup` is again just two independent requests (install `a/pyproject.toml`'s dependencies, and `b/pyproject.toml`'s `mygroup`). g) `uv pip install -r pyproject.toml --group mygroup` is exactly like the previous case but *incidentally* the two requests refer to the same file. What the user wanted to happen is almost certainly happening, but they are likely getting "lucky" here that they're requesting something simple. h) `uv pip install -r a/pyproject.toml --group mygroup` is again exactly the same but the user is likely to get surprised and upset as this invocation actually sources two different files (install `a/pyproject.toml`'s dependencies, and `pyproject.toml`'s `mygroup`)! I would expect most people to assume the `--group` flag here is covering all applicable requirements/sourcetrees/editables, but no, it continues to be a totally independent reference to a file with a hardcoded relative path. ------ Fixes #8590 Fixes #8969

This PR switches from using optional-dependencies to dependency-groups for several development dependencies This depends on: - tox-dev/tox#3409 - astral-sh/uv#8272 - astral-sh/uv#8590 - astral-sh/uv#8969 - astral-sh/uv#10861 - astral-sh/uv#11686 (actually incorporated in uv 0.6.8) - readthedocs/readthedocs.org#11766, will be solved by readthedocs/readthedocs.org#11710

Gankra commented Jan 22, 2025

View reviewed changes

zanieb reviewed Jan 22, 2025

View reviewed changes

Gankra added 9 commits January 22, 2025 13:48

WIP: it works?

d39a562

add positive pip_install --group tests

25167a5

add no_default_groups

d5af9f8

generate-all

915e8e6

restrict to pyproject.tomls

1235604

don't support --no-default-groups

9022093

fix ordering of doc comment

b02b04b

regen snapshots

3ce6e53

regen more snapshots

15e22dc

Gankra force-pushed the gankra/depgroups branch from 264867c to 15e22dc Compare January 22, 2025 18:52

Gankra added 3 commits January 22, 2025 13:53

omg clippy chill

cd13972

fix

ccc1d71

add tests for multi-install

9247f2f

Gankra mentioned this pull request Jan 22, 2025

feat: error on non-existent extra #10869

Closed

Gankra added 3 commits January 22, 2025 15:34

Add warnings for undefined groups

510c5f0

add tests for pip_compile

a1e3b53

stabilize output

3d043f1

Gankra merged commit 53706a1 into main Jan 23, 2025
68 checks passed

Gankra deleted the gankra/depgroups branch January 23, 2025 13:47

cthoyt mentioned this pull request Jan 23, 2025

Implement dependency groups cthoyt/cookiecutter-snekpack#32

Merged

notatallshaw mentioned this pull request Jan 23, 2025

Feature Proposal: PEP 735 "Dependency Groups" Support pypa/pip#12963

Closed

1 task

zanieb mentioned this pull request Jan 23, 2025

temporarily disable new uv pip dependency-group flags #10909

Merged

Gankra mentioned this pull request Feb 21, 2025

add pip-compatible --group flag to uv pip install and uv pip compile #11686

Merged

zanieb mentioned this pull request Jul 14, 2025

How can I do soething like this uv pip install -r pyproject.toml --group dev #14590

Closed

Add dependency-group cli flags to uv pip install and uv pip compile (--group, --no-group, --only-group, --all-groups) #10861

Add dependency-group cli flags to uv pip install and uv pip compile (--group, --no-group, --only-group, --all-groups) #10861

Uh oh!

Conversation

Gankra commented Jan 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Gankra commented Jan 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

qvalentin commented Jan 23, 2025

Uh oh!

henryiii commented Jan 23, 2025

Uh oh!

Gankra commented Jan 23, 2025

Uh oh!

zanieb commented Jan 23, 2025

Uh oh!

Filimoa commented Jan 31, 2025

Uh oh!

charliermarsh commented Jan 31, 2025

Uh oh!

henryiii commented Jan 31, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

Add dependency-group cli flags to `uv pip install` and `uv pip compile` (`--group`, `--no-group`, `--only-group`, `--all-groups`) #10861

Add dependency-group cli flags to `uv pip install` and `uv pip compile` (`--group`, `--no-group`, `--only-group`, `--all-groups`) #10861

Gankra commented Jan 22, 2025 •

edited

Loading

Gankra commented Jan 22, 2025 •

edited

Loading