Spring boot plugin rewrites jar contents, making the build not reproducible unless tests are run

[hgschmie]

The spring boot maven plugin rewrites the jars that a build creates when executing the spring-boot:process-test-aot goal.

This goal is dependent on the maven-wide 'skipTests' and 'maven.test.skip' properties. If either of these is set, the execution is skipped.

For modern java projects, it is common to provide reproducible builds (See https://reproducible-builds.org/docs/jvm/ for details). In a reproducible build cycle, the CI does

• a full build of a project executing all tests (neither 'skipTests' nor 'maven.test.skip' is set). The resulting jars are installed in the local repository

• a second build using (for maven) the artifact:compare target. For that second build, usually tests and other checks are skipped as the goal of that build is simply to validate that the resulting jars are bitwise identical.

Currently, it is impossible to use the spring-boot:process-test-aot in this reproducible build cycle. Adding the plugin to the build results in the plugin being executed in the first build, but skipped in the second as it looks at the skipTests and maven.test.skip which are set to actually skip the tests while the spring-boot target should be executed independent of whether the tests are run or not.

Not being able to control the execution of this goal independent of the actual test execution (whether through maven-surefire, maven-native or any other maven test plugin) is a bug. This goal is not a test specific goal but a test-build specific goal.

The current situation makes it impossible for to provide reproducible builds if one wants to support native builds and also support spring. Dropping native build support is not an option.

One possible solution is making the execution of the spring-boot:process-test-aot not directly dependent on the properties mentioned above but have a configuration option to skip or run the goal and have its default set from the proprties but allow users to override this in the maven pom file. This would allow for reproducible builds while maintaining full compatibility to the current functionality.

[wilkinsona]

Currently, it is impossible to use the spring-boot:process-test-aot in this reproducible build cycle. Adding the plugin to the build results in the plugin being executed in the first build, but skipped in the second

Why does that matter in the context of artifact:compare? I understand the benefits of
a build's "production" artifacts being reproducible but does that comparison consider test classes (and possibly a test-specific native image too)?

[hgschmie]

because the artifact gets published (like all other maven artifacts). Users tend to take a dim view if we tell them "you can verify these 34 artifacts but not those three". :-)

In this specific case, we are shipping an (OSS) library to support spring as part of our project (https://jdbi.org/#_spring_framework). We have > 200 artifacts and all of them are reproducible for regular and native builds. Except the spring test jars.

[wilkinsona]

Publishing tests is, in my experience, rather unusual. That would have been useful information to provide up front.

To help us to understand exactly what you're doing, please provide a minimal sample that exhibits the non-reproducible behavior that you've described.

[hgschmie]

Thanks. Not publishing the test jars is a good point, something to think about. I am not sure whether artifact:compare can skip jars that will be installed but not deployed (it may need some additional logic). We do need to install the jars locally because we have integration tests that depend on some of the test jars.

Skipping them for some sub-projects may be an option. The wider issue (that the spring-boot:process-test-aot is a goal that is run as part of the test build process and not the test execution process and therefore should be available if tests are built but not executed) still remains, even though we may find ways to work around this.

[wilkinsona]

We can't make any progress here without understanding exactly how you've set up Maven. To that end, please provide a minimal sample that exhibits the non-reproducible behavior that you've described.