`cargo package --no-verify` fails if a package's version req is too high for the registry (but works locally)

[weihanglo]

Problem

#13447 dd698ff has some unintentional consequences:

cargo package after that generates a lockfile also when packaging libraries.
However, if the crate being packaged depends on some unpublished dependencies. When generating the lockfile Cargo then can't find unpublished deps on registry index, so it bailed out.

Steps

This could be simplly reproduced in rust-lang/cargo as of today (2025-01-13)

1. Checkout to 54df3c7
2. Bump cargo-test-support to an unpublished version such as 99.99.99
3. cargo +1.83.0 package -p cargo-test-support --no-verify — succeeded
4. cargo +1.84.0 package -p cargo-test-support --no-verify — failed

Possible Solution(s)

Assume if we have passed the local registry unconditionally, we shouldn't have the issue today.

cargo/src/cargo/ops/cargo_package/mod.rs

Lines 219 to 225 in f15df8f

	let mut local_reg = if ws.gctx().cli_unstable().package_workspace {
	let reg_dir = ws.target_dir().join("package").join("tmp-registry");
	sid.map(|sid| TmpRegistry::new(ws.gctx(), reg_dir, sid))
	.transpose()?
	} else {
	None
	};

Version

• cargo 1.84.0 (66221abde 2024-11-19)
• cargo 1.86.0-nightly (088d49608 2025-01-10)

[epage]

So this is a problem that existed before 54df3c7 but only for some crates. Now all see it, right?

[weihanglo]

Yeah I believe it was only bugging for packages having [[bin]] crates, but now every package.

[epage]

@Eh2406 our resolve call is

cargo/src/cargo/ops/cargo_package/mod.rs

Lines 655 to 664 in 54df3c7

	let mut new_resolve = ops::resolve_with_previous(
	&mut tmp_reg,
	&tmp_ws,
	&CliFeatures::new_all(true),
	HasDevUnits::Yes,
	orig_resolve.as_ref(),
	None,
	&[],
	true,
	)?;

Is there a way to say "prefer existing but don't lock them? Would keep = |_| false do that? Seems like what we'd want (have to double check yanked) and then cargo package already has logic to warn in these situations which imo is good because its easy to accidentally release a package and not realize you forgot to release a dependency.

[weihanglo]

In terms of a swift fix: How stable is TmpRegistry?
It might be helpful if we could use it internally without stabilizing the entire -Zpackage-workspace.

[epage]

I'm not sure how that would fix it. We wouldn't be putting the package into TmpRegistry to be able to be used unless the user requests it at which point it will work anyways.

[weihanglo]

More like Cargo automatically publishes unpublished dependencies crates when needed to TmpRegistry before building the lock?

[epage]

I suspect getting it right to sometimes publish to TmpRegistry can cause us other problems and would get in the way of stabilizing workspace packaging that I feel like that less likely to be our best option for a swift fix.

if we feel the need to backport to beta, a revert is likely the best route. If not, we should at least understand what it would take on the resolver side.

[sehz]

We are running into same issue. Can this is be reverted?

[epage]

I cannot reproduce this if I bump the package's version but don't bump the version requirement. I also missed the detail that --no-verify is required..

This means that cargo is able to adjust the lockfile correctly as needed and the problem is that we moved "dependency exists" checks from the verify step to the packaging step, forcing it to always happen.

The documentation for cargo publish --no-verify just says

Don’t verify the contents by building them.

I think this still leaves room for other verification. Historically, we've done some registry and dependency verification and we recently added "is this published?" (#14448).

cargo/src/cargo/ops/registry/publish.rs

Lines 96 to 139 in 54df3c7

	let just_pkgs: Vec<_> = pkgs.iter().map(|p| p.0).collect();
	let reg_or_index = match opts.reg_or_index.clone() {
	Some(r) => {
	validate_registry(&just_pkgs, Some(&r))?;
	Some(r)
	}
	None => {
	let reg = super::infer_registry(&just_pkgs)?;
	validate_registry(&just_pkgs, reg.as_ref())?;
	if let Some(RegistryOrIndex::Registry(ref registry)) = &reg {
	if registry != CRATES_IO_REGISTRY {
	// Don't warn for crates.io.
	opts.gctx.shell().note(&format!(
	"found `{}` as only allowed registry. Publishing to it automatically.",
	registry
	))?;
	}
	}
	reg
	}
	};
	// This is only used to confirm that we can create a token before we build the package.
	// This causes the credential provider to be called an extra time, but keeps the same order of errors.
	let source_ids = super::get_source_id(opts.gctx, reg_or_index.as_ref())?;
	let (mut registry, mut source) = super::registry(
	opts.gctx,
	&source_ids,
	opts.token.as_ref().map(Secret::as_deref),
	reg_or_index.as_ref(),
	true,
	Some(Operation::Read).filter(|_| !opts.dry_run),
	)?;
	{
	let _lock = opts
	.gctx
	.acquire_package_cache_lock(CacheLockMode::DownloadExclusive)?;
	for (pkg, _) in &pkgs {
	verify_unpublished(pkg, &mut source, &source_ids, opts.dry_run, opts.gctx)?;
	verify_dependencies(pkg, &registry, source_ids.original)?;
	}
	}

Its also not clear to me why you would be publishing packages without dependencies present. Is this a workaround for #4242? We already have a method for this for some cases (leaving off versions for dev-dependencies). If there is a case that doesn't cover, it would be good to make sure its reported in #4242.