Skip to content

Conversation

@alexarchambault
Copy link
Collaborator

@alexarchambault alexarchambault commented Jun 17, 2025

This makes Mill read all environment variables and Java properties used by coursier via a Task.Input.

In more detail, this adds a CoursierConfigModule external module in scalalib. It contains a single Task.Input, that reads all environment variables used by coursier via Task.env and all Java properties used by coursier via sys.props.

Helper tasks compute default values for various parameters (repositories, cache location, etc.). CoursierConfigModule.coursierConfig returns most of these parameters in a case class defined in Mill, CoursierConfig. For convenience, down-the-line, we pass CoursierConfig instances around.

Fixes #5134 (by managing coursier env vars / Java properties via Task.Input)

Fixes #5229 (via coursier/coursier#3426 from coursier 2.1.25-M14 and the small rework of default repository handling)

@alexarchambault alexarchambault marked this pull request as ready for review June 23, 2025 09:47
@alexarchambault alexarchambault requested review from lefou and lihaoyi June 23, 2025 09:47
Copy link
Member

@lefou lefou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The most tricky case with long running daemon is the case where the env variable is defined initially, but later gets removed from the env, in which case we need to fall back to the other defaults. Do we have this in a test case already?

Copy link
Member

@lefou lefou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Found two other nits.

Could you also clean installation-ide.adoc which references the issues you just fixed?

NOTE: Currently, you may need to `./mill shutdown` before calling Mill with the environment variable to make sure it gets properly propagated to Mill's background daemon.
There is an https://github.com/com-lihaoyi/mill/issues/5134[open issue] to let Mill detect these changes automatically.

/**
* Aggregates coursier parameters read from the environment
*/
def coursierConfig: Task[mill.util.CoursierConfig] = Task.Anon {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like a candidate for caching, to me.

Suggested change
def coursierConfig: Task[mill.util.CoursierConfig] = Task.Anon {
def coursierConfig: Task[mill.util.CoursierConfig] = Task {

Copy link
Collaborator Author

@alexarchambault alexarchambault Jun 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've been trying that. Some abstract classes are involved (coursier.core.Repository, coursier.params.Mirror, coursier.credentials.Credentials in particular), which can be worked around, given this class is parsed from a bunch of strings, and the class instances are actually only simple easy-to-serialize classes.

But it seems this can also involve credentials (parsed from files on disk in particular). This raises security concerns: we don't adjust permissions of files where task results are written, so we could end up writing passwords in files with too loose permissions.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's leave it as-is then. Would be nice to have some support for secret caching (e.g. via encryption, or just in memory), but we currently don't.

@lihaoyi
Copy link
Member

lihaoyi commented Jun 27, 2025

I think this looks fine, once the last set of review comments have been resolve we can merge this

 Conflicts:
	core/util/src/mill/util/Jvm.scala
	mill-build/src/millbuild/Deps.scala
@lefou lefou merged commit d03fd45 into com-lihaoyi:main Jun 27, 2025
39 checks passed
@lefou lefou added this to the 1.0.0-RC3 milestone Jun 27, 2025
lihaoyi added a commit to lihaoyi/mill-1 that referenced this pull request Jul 1, 2025
lihaoyi added a commit that referenced this pull request Jul 1, 2025
souvlakias pushed a commit to vaslabs-ltd/mill that referenced this pull request Jul 1, 2025
@alexarchambault alexarchambault deleted the coursier-env branch July 8, 2025 14:45
alexarchambault added a commit that referenced this pull request Sep 4, 2025
This is a new attempt at fixing
#5134. Turns out setting the
context class loader when evaluating user code addresses the issue while
not running into #5398 again.

---

Description from #5350 that
still applies:

This makes Mill read all environment variables and Java properties used
by coursier via a `Task.Input`.

In more detail, this adds a `CoursierConfigModule` external module in
scalalib. It contains a single `Task.Input`, that reads all environment
variables used by coursier via `Task.env` and all Java properties used
by coursier via `sys.props`.

Helper tasks compute default values for various parameters
(repositories, cache location, etc.).
`CoursierConfigModule.coursierConfig` returns most of these parameters
in a case class defined in Mill, `CoursierConfig`. For convenience,
down-the-line, we pass `CoursierConfig` instances around.

Fixes #5134 (by managing
coursier env vars / Java properties via `Task.Input`)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Coursier default repositories break mill init Properly handle coursier-supported env variables in Mill daemon

3 participants