Redefine `CARGO_TARGET_DIR` to be only an artifacts directory

[kornelski]

For design, see #14125 (comment)

For tracking the artifact-dir side of this, see #6790

Documentation: https://doc.rust-lang.org/nightly/cargo/reference/unstable.html#build-dir

Testing instructions: #14125 (comment)

FCP: #14125 (comment)

Implementation

• unstable build-dir support (Implemented build.build-dir config option #15104)
• build-dir templating (Added build.build_dir templating support #15236)
  • fix(build-dir): Renamed workspace-manifest-path-hash to workspace-path-hash #15334
• cargo metadata support (Added build_directory field to cargo metadata output #15377)
• Document build_directory cargo metadata field (docs(metadata): Added build_directory to cargo metadata documentation #15410)
• workspace-path-hash should have symlinks resolved (Added symlink resolution for workspace-path-hash #15400)
• error on unmatched {, } (Added validation for unmatched brackets in build-dir template #15414)
• append closest_msg to unknown variable error (and maybe list all variables if no match is found?) (Improved error message when build-dir template var is invalid #15418)
• call for testing (Redefine CARGO_TARGET_DIR to be only an artifacts directory #14125 (comment))
• stabilize build-dir

Stabilizing this would also resolve

• Remap target dir #11156

Known issues:

• build_from_source makes assumptions on internal Cargo details, breaking cargo-miri with new build-dir features RalfJung/rustc-build-sysroot#25

Decisions

• cargo clean will clean both target-dir and build-dir
• Skipping including a CARGO_BUILD_DIR shortcut for CARGO_BUILD_BUILD_DIR (like CARGO_TARGET_DIR is a shortcut for CARGO_BUILD_TARGET_DIR)

artifact-dir files:

• build
  • binary executable for bin crates
  • binary executable for examples
  • depinfo files (.d files) for third party build-system integrations (see https://github.com/rust-lang/cargo/blob/master/src/cargo/core/compiler/fingerprint/mod.rs#L194)
  • Cargo's --timings HTML report
• cargo package's generated .crate files
• cargo doc output (html/css/js/etc)

build-dir files (intermediate artifacts, build state, caches):

• build
  • "pre and non uplifted" binary executables. (ie. bins for examples that contain the hash in the name, bins for benches, proc macros, build scripts)
  • other depinfo files (generated by rustc, fingerprint, etc. See https://github.com/rust-lang/cargo/blob/master/src/cargo/core/compiler/fingerprint/mod.rs#L164)
  • rlibs and debug info from dependencies
  • build script OUT_DIR
  • output from proc macros (previously stored in target/build)
  • incremental build output from rustc
  • fingerprint files used by Cargo for rebuild detection
  • Cache of rustc invocations (.rustc_info.json)
• cargo package's scratchpad used for the verify step
• CARGO_TARGET_TMPDIR files (see rational for this here)
• reports generated from cargo report like future-incompat-report

Open questions

• Are we good with the name build-dir?
  • Yes, discussed on 2025-03-18
• Should we offer CARGO_BUILD_DIR as a shortcut to CARGO_BUILD_BUILD_DIR, like CARGO_TARGET_DIR is for CARGO_BUILD_TARGET_DIR? see Implemented build.build-dir config option #15104 (comment)
  • No, discussed on 2025-03-18
  • End-users are most likely to set this globally in config
  • Env variable is most likely in CI
  • Length and "cleaner name" aren't priorities for the target audience, CI
• Should we stick with the name {workspace-manifest-path-hash} and what should it include? Should we shorten to {workspace-hash} or even just {hash}?
  • We should drop manifest, unsure about dropping either workspace or path. Users may be confused if they don't realize the path is a part of the hash
  • Renamed to workspace-path-hash in fix(build-dir): Renamed workspace-manifest-path-hash to workspace-path-hash #15334
• Should we include the Cargo version in {workspace-manifest-path-hash} so we get unique whole-target directories for easier cleanup (Garbage collect whole target/ #13136) deferred
• Should we resolve symlinks in workspace manifest-path before generating {workspace-manifest-path-hash}? See Added build.build_dir templating support #15236 (comment)
  • The cargo team leans towards doing so right now
  • As the hash is unspecified, we feel this is a two-way door
  • Added symlink resolution for workspace-path-hash #15400
• Should we keep erroring on unknown template variables? Should we start erroring on unmatch {? See Added build.build_dir templating support #15236 (comment)
  • Yes, we should error on unknown template variables. It is the least surprising, even if annoying, and provides better error messages (229489e)
  • Yes, we should error on unmatched {. This includes {{; escaping has been deferred (Added validation for unmatched brackets in build-dir template #15414)
• Should we take the advantage to self-ignore build.build-dir? Make target dir self-ignoring #15061
• Include in cargo metadata? What about that people need to match cargo metadata version against cargo to get it?
  • Yes, it should be included in cargo metadata but only the current one. Changing the hash on every cargo version has been deferred to reduce the need to provide build-dir for all cargo versions
  • Added build_directory field to cargo metadata output #15377, docs(metadata): Added build_directory to cargo metadata documentation #15410
• Should we offer CARGO_BUILD_TMPDIR, deprecating CARGO_TARGET_TMPDIR?
• should we offer the workspace base name either as a prefix to the hash or as its own variable for making it easier for humans to find delete specific build dirs

Deferred

• {workspace-manifest-hash} also hashes the cargo version
  • Excludes release channel information, including which nightly
  • Allows easier cleanup on rustup update (Garbage collect whole target/ #13136)
  • Makes very clear that the hash's meaning is unstable
  • Blocked on Garbage collect whole target/ #13136 (maybe more?) to reduce the need for third-party clean up tools which will need to know the path to every build-dir for a workspace
• In templates, {{ is an escaped {
  • With us erroring on {, we have the room to allow escaping in the future
  • We'd like escaping support to be based on use cases and not speculatively developed as { in paths is rare and this isn't a general path

Original Issue:

---

Problem

There are a couple of issues with the CARGO_TARGET_DIR that are seemingly in conflict with each other:

1. Multiple locations of target dirs complicate excluding them from backups and full-disk search, cleanup of the temp files, moving temp files to dedicated partitions, out of slow network drives or container mounts, etc. Users don't like that the target dir is huge, and multiple instances of it add up to lot of disk space. Users would prefer a central location to ease management of the temp files, and also to dedupe/reuse dependencies across many projects.

2. People (and tools) are relying on a relative ./target directory being present to copy or run built files out of there. Additionally, users may not want to configure a shared CARGO_TARGET_DIR due to risk of file name conflicts between projects.

However, the dilemma between 1 and 2 exists only because Cargo uses CARGO_TARGET_DIR for two different roles:

1. A cache for all intermediate build products (a place where crates.io crates are built, where compiler-private temp files are) which aren't project-specific, and/or files that users don't need to access directly.
2. A location for user-facing final build products (artifacts) that users expect to be there and need to access.

Proposed Solution

So to satisfy both uses, I suggest to change the thinking about what the role of CARGO_TARGET_DIR should be. Instead of thinking where to put the same huge all-purpose mixed CARGO_TARGET_DIR, think how to deduplicate and slim CARGO_TARGET_DIR, and move everything non-user-facing out of it.

Instead of merging or sharding the CARGO_TARGET_DIR as-is with all of its current content, and adding --artifact-dir as a separate place where final products are being copied to — make CARGO_TARGET_DIR to be the artifact dir (without copying).

As long as the CARGO_TARGET_DIR dir is the place for all of the build files, of all crates including all the crates.io and local builds, with all the caches, all the temp junk, then this is going to be a problematic large directory that needs to be managed. But if the purpose of the ./target dir was changed to be only for user-facing files (files that users can name, and would access via ./target path themselves), then this directory would be relatively small, with a good reason to stay workspace-relative.

What isn't an intermediate build product? (and should stay in ./target)

• linked (and stripped) binaries of the current workspace, including binaries for the examples,
• libraries of the current workspace as .a/.so, where lib.crate-type calls for them. Possibly .rlib/.rmeta in the future if there's a stable ABI.
• linked binaries for tests and benches of the current workspace (to make it easy to launch them under a debugger/profiler, and so they can use relative file paths to read workspace assets).
• debug symbols for all of the above.
• .d files for all of the above (so that IDEs and other build systems know when to rebuild the artifacts).
• if Cargo adds some "staging" directory (a non-private OUT_DIR for build.rs, see Allow build scripts to stage final artifacts #13663), then for build scripts belonging to the current workspace it would be inside ./target as well.

So generally files that users build intentionally, and may want to access directly (run themselves, or package up for distribution) and files that users may need configure their IDE and debugger to find inside the project.

Crates in [patch.crates-io] with a path are a gray area, an might also have their artifacts included in the ./target dir (but in some way that avoids clobbering workspaces' files).

What isn't a final build product, and doesn't belong to ./target:

• anything related to building crates from crates.io, or any other registry (packages with source = "registry+…")
• all .fingerprint and incremental dir content of all crates. These are implementation details of the compiler, and nobody should be accessing these directly via ./target/….
• .o files. Users are not supposed to use them directly either (Rust has static libs for this).
• proc macro libs. They're not useful without rustc present.

All of these should be built in some other shared build cache dir (one that is not inside CARGO_TARGET_DIR), configurable by a new option/env var.

Registry dependencies would get unique paths derived from rustc version + package IDs + enabled features (so that different crates using different features don't invalidate each others' caches all the time). This would enable sharing built crates.io dependencies across all projects for the same local user, without also causing local workspaces to clobber each others' CARGO_TARGET_DIR/profile/product paths. Temp directories for local projects would need some hashed paths in the shared build/temp dir too.

Advantages

• Such split removes about 90% of the weight from ./target dirs (for cargo itself, it makes ./target/debug with binaries and tests take 415MB, instead of 4.2GB). This makes cleanup of all the scattered target dirs less of a pressing problem.
• The ./target keeps relatively few files, and removes high-frequency-churning files out of it, which makes it less of a problem for real-time disk indexing (like search and backups on macOS).
• I/O latency of ./target stops being critical for build speeds, unlike I/O of the incremental cache and rewrites of thousands of .o files. It becomes feasible to have project directory on a network drive without overriding CARGO_TARGET_DIR (network filesystems are used by non-Linux systems where tools like Vagrant and Docker have to run full-fat VMs, and can't cheaply share the file system).
• It makes ./target contain only workspace-unique files, which makes it justified for every workspace to have one.
• It enables moving registry deps to a shared build directory, without side effect of local projects overwriting each others' files. Sharing of dependencies matches users' expectation that the same dependencies shouldn't be redundantly rebuilt for each local project.
• It's almost entirely backwards compatible. Users can get the benefits without breaking their existing workflows, post-build scripts, and integrations. It doesn't invalidate documentation/books/tutorials that refer to target/release/exe etc.
• It could be the default behavior, so it could benefit all users without friction of adding --artifact-dir or .cargo/config.

Notes

No response

[kornelski]

In terms of incompatibilities, only a few come to my mind:

• Hacks that recursively search ./target looking for a file built by some build script in its OUT_DIR. Ideally this should be solved by adding proper support for building arbitrary assets in Cargo, but for the sake of keeping compatibility with CARGO_TARGET_DIR, this could be made to work by keeping OUT_DIRs as subdirectories or symlinks in ./target.

• Binaries trying to link to .so/.dylib of their Rust dependencies, where the deps are built by Cargo, and are not installed on the system. This is currently basically unsupported in Cargo, because there's no way to set rpath to anything that works (absolute path is too build-specific, and relative doesn't work for tests and examples that are in a different dir than bins). But if users are hacking around it with custom post-build steps, they may expect to find the 3rd party shared libs in ./target. This again would be better solved by having Rust/Cargo aware of dylib dependencies, and copy/hardlink dylibs as needed and set rpath accordingly. Not many deps have crate-type = "cdylib", so these also could be kept/symlinked in ./target for back-compat.

• Build containers could have very strict read-only filesystems with only CARGO_TARGET_DIR and CARGO_HOME subdirs being writeable. This could be made backwards-compatible by reverting back to the current everything-in-one-place behavior when CARGO_TARGET_DIR is set, and picking other env vars for configuring locations of the trimmed ./target and the intermediate products cache dir.

[epage]

@poliorcetics from rust-lang/rfcs#3664 (comment)

Yes, the issue should be moved to cargo I think.

I'm not convinced at all this won't break backwards compatibility in some way.

It makes ./target contain only workspace-unique files, which makes it justified for every workspace to have one.

And I don't want one in any cargo project while still keeping isolation, which is entirely different from what you are proposing.

It enables moving registry deps to a shared build directory, without side effect of local projects overwriting each others' files. Sharing of dependencies matches users' expectation that the same dependencies shouldn't be redundantly rebuilt for each local project.

Once again, the RFC I wrote and original issue I inspired myself from do not ask for that, it asks for the opposite: myself and many others want separate targets dirs for every project.

There are probably as many reasons as there are users for it but common ones are different sets of features amongst projects, sharing of build caches for specific projects, CI builds wanting to separate projects for security, or one project pinning 1.2.3 in a dep B of a dep A and the other project pinning 1.2.4: A can have the same version for both but it's dependencies won't and cargo is not made to handle the case at the moment.

You are fundamentally solving a different issue, one that the RFC I posted is not trying to solve.

[epage]

Overall, I see this as a solution alternative to #6790 and had recommended we have that conversation there (or on internals).

What isn't an intermediate build product? (and should stay in ./target)

This is likely going to be the most difficult topic to work through and we'll need to make sure we get wide input on this from #6790 users and others.

Registry dependencies would get unique paths derived from rustc version + package IDs + enabled features (so that different crates using different features don't invalidate each others' caches all the time). This would enable sharing built crates.io dependencies across all projects for the same local user, without also causing local workspaces to clobber each others' CARGO_TARGET_DIR/profile/product paths. Temp directories for local projects would need some hashed paths in the shared build/temp dir too.

imo this is out of scope for this proposal (see #5931) and we should keep this focused so as not to get distracted.

[epage]

I see this as a re-framing of the problem, addressing #6790 and rust-lang/rfcs#3371

Instead of us defining a new artifact-dir, we say target-dir is the artifact directory and move everything else out into a "working directory".

• The move can be gradual as these files are considered implementation details.
• We avoid "yet another flag" problem of --artifact-dir because controlling of the location of the "working directory" is too low level for the CLI and should be reserved for config
• A big downside compared to --artfact-dir is that target-dir is not the final directory files are put under but the parent directory. This is especially annoying for cargo build vs cargo build --target, dealing with profile names, etc

Potential names

• working-dir (arbitrarily chose this one to refer to the concept moving forward)
• build-dir

Cargo script would default its target-dir as its working-dir

This would need input from

• artifact-dir users
• cache / cache clean up users
• random other systems that dig into the target dir

[epage]

This would need an audit of ways we publicly treat the target dir as a working dir, like exposing CARGO_TARGET_TMPDIR

[ensc]

Sharing sources over NFS adds another incompatibility when the ability to move ./target out of the sources is eliminated.

Every modern buildsystem allows to keep sources and build results separated (and users and tools do not have problems with it). I do not think that cargo should go the way back and enforce a fixed ./target directory.

[kornelski]

I'm not suggesting to force it to always be ./target. The CARGO_TARGET_DIR can continue to move this directory elsewhere.
The main point is to reduce severity of problems that the current high-churn high-volume content of this dir causes.

[epage]

We talked about this in today's Cargo team meeting.

Our care abouts include

• Moving intermediate artifacts away
  • Having a transition plan for tool authors
• Providing a profile-agnostic location for final artifacts
• Resolving the "elide target when host" within the final artifact path
• Having a coherent story for users, for example
  • Users today mostly interact with target-dir for dealing with final artifacts, so we likely want to preserve that
  • If we create a new directory for intermediate artifacts, we might not want to templatize target-dir as that will push people down one path moving intermediate artifacts and then we completely change it on the user what path they should go, rather than having a stable story for how to handle this

While we acknowledged the potential for user confusion with CARGO_TARGET_TMPDIR, we were fine with it not being associated with target-dir

The general shape of what we proposed in the meeting is...

Shiny future

target-build-dir: Home of intermediate artifacts

• Goals:
  • Move intermediate artifacts out of source
  • Don't force a transition for what term/variable people reach out to to access final artifacts
• name is a placeholder
• No CLI flag for target-build-dir
• Precedence (note the lack of target-dir)
  • target-build-dir config
  • target-build-dir default ("{cargo-cache}/target/{workspace-manifest-path-hash}")

target-artifact-dir: Home of final artifacts

• Goals:
  • Provide profile-agnostic path
  • Provide a way for users to opt-out of the target-platform from being elided when building for the host-platform
• name is a placeholder (e.g. do we want the target prefix)
• --artifact-dir CLI flag is added
• will error if {platform} or {legacy-platform} is not present with multi-target builds
• Precedence
  • target-artifact-dir config / cli
  • target-dir config / cli
  • target-artifact-dir default ("{workspace-root}/target/{legacy-platform}/{profile}")

Legacy target-dir

• --target-dir is hidden on CLI. Maybe shows up in man pages
• No templating supported

Other

• CARGO_TARGET_TMPDIR points within target-build-dir

Path to Shiny Future

target-build-dir

In theory, we could trivially do this by

• locking target-build-dir and target-dir (if different)
• doing all work in target-build-dir and only when we do the "hardlink or cp" do we reference target-dir

Initial default is "{workspace-root}/target"

Template supports

• {workspace-root}
• {cargo-cache} (pointing to CARGO_HOME for now)
• {workspace-manifest-path-hash}

Steps

1. Implement
2. Call for testing
3. Stabilize with opt-in to new location
4. Call for testing
5. Switch to opt-out

Notes:

• Throughout those steps, we would work closely with tool authors to make sure they had a smooth transiti
• The opt-in/opt-out could either be
  • .cargo/config.toml target-build-dir field
  • A one-off environment variable if we don't want to stabilize the above yet

target-artifact-dir

Assumption: target-build-dir takes some pressure off of target-artifact-dir

Defaulted to final location ("{workspace-root}/target/{legacy-platform}/{profile}")

Template supports

• {workspace-root}
• {cargo-cache} (pointing to CARGO_HOME for now)
• {workspace-manifest-path-hash}
• {platform}
• {legacy-platform} (elides host-target)

Needs all of the details in the tracking issue to be finalized.

Contingencies

If target-build-dir takes more than N time (1 year?) to stabilize, then we re-evaluate approving rust-lang/rfcs#3371. This is to try to balance the needs of the people who want something like rust-lang/rfcs#3371 now vs (1) the long-term inapplicability of that RFC and (2) the lack of stable "blessed" workflow for users (telling users to use solution X for several months and then telling them that is no longer "right" and they need to use solution Y).

Alternatives

• Maintain target-dir as the "artifact base dir" and provide a query command (like buck2 analyze) to ask what the "artifact dir" is
• Don't expose target-artifact-dir in CLI, leaving it for more advanced uses
• Have only target-dir and target-artifact-dir
  • Downside: harder end-user transition as they reach out to target-dir for final artifacts. While we eventually de-emphasize target-dir, those workflows will work just fine
• Have only target-dir and target-build-dir
  • Downside: access to final artifacts is still annoying
  • Only append final directories ({platform}/{profile}) if no templates are available
    • Downside: this would be confusing
    • We do this for {dl} but that is a more limited / off in the weeds use case rather than front and center for users
• Change the meaning of target-dir if target-build-dir or target-artifact-dir is present
  • This gets confusing