Simplified dhis-core maven module structure

[jbee]

Problems

A collection of the issues experienced with the current maven modularisation.

• not all model types are located in a common model package
  • forces either to place dependent classes in the same module or split out another module with a dependency
  • causes creation of many small modules (=> multiplication matrix of modules based on their dependencies)
• modules are motivated by external dependencies (mostly service layer)
  • causes creation of many small modules (=> multiplication matrix of modules based on their dependencies)
• dependency graph is too complex to reason about
  • even with a plugin checking dependencies unwanted dependencies are introduced by explicitly adding them as it is no longer clear what is and isn't a clean solution)

Goals

Immediate

• easy to understand dependency graph that has only contains simple paths to the root (no splitting)
• dependencies motivated by dhis2 application layers
• modules that align with products and teams
• modules that make it easy to pick the right module for a class
• modules that allow to easily identify in a review that a class does not belong into a certain module (risk would be to put the type too close to the root, or in other words modules closer to the root should change less and be checked more critical)

Long term

• create preconditions for use unique package names only (ready for java modules)

Proposal

Layers

Layers are allowed to depend on all layers above:

1. api
2. service
3. web

Modules

Modules in below tree may only depend on parents in the tree, except

• dhis-service-* may all depend on dhis-service-common
• dhis-web-* may all depend on dhis-web-common:

Module Tree (test modules excluded):

• dhis-api-common (or just dhis-api): common model and utilities
  • dhis-service-common: shared service implementation utilities (roughly all of what now is dhis-support-*)
    • dhis-web-common: shared web base classes and utilities
  • dhis-api-platform: platform only model and utilities
    • dhis-service-platform: platform only API implementation
      • dhis-web-platform: platform only web implementation
  • dhis-api-tracker: tracker only model and utilities
    • dhis-service-tracker: tracker only API implementation
      • dhis-web-tracker: tracker only web implementation
  • dhis-api-analytics: analytics only model and utilities
    • dhis-service-analytics: analytics only API implementation
      • dhis-web-analytics: analytics only web implementation

Additionally dhis-web-applciation depends on all dhis-web-* dependencies to join everything to a web application.

Optionally,

• if deemed beneficial a dhis-utils could be extracted from dhis-api containing utilities that only depend on JDK types, like date/time helpers or string helpers

Questions

• Does the embedded jetty need a module on its own or can it be in dhis-web-application (packaging issues?)
• Should the web layers have access to service or is all access going via interfaces defined in api?
• Is getting rid of struts (non-linked web module part) a precondition?

Transition

A step by step transition in a dozen or so PRs.

1. Prepare target: Create the goal module structure as "empty" modules (alongside the existing modules)
2. Move: move classes to their goal module, PR scope of one module or package (in a module) at a time, keeping their package unchanged
3. Cleanup: Once a "old" module is empty it gets deleted
4. Organize (internally): each of the "new" non-api modules gets cleaned up package wise internally as that should not affect any other module
5. Consolidate: Wait until all "old" modules are moved
6. Organize: each of the "new" api modules can get cleaned up package wise
7. Align: establish contracts (APIs) between modules where such APIs do not exist but should

[teleivo]

To me modules that align with products and teams is the most important. As I want us to be able to agree on contracts/interfaces between teams that rarely change while the implementation is free to change.

I am not aware of how our types might be consumed by others. Meaning to me tracker provides APIs that are consumed via HTTP. From that perspective I wonder why we don't model as a single dhis-tracker module. I am just naively stating this here so you can explain the rationale behind the api/service/web split 😄

[jbee]

I wonder why we don't model as a single dhis-tracker module

This is purely to keep access to web layer dependencies out of the application layer. We cannot avoid adding such dependencies for the web stack but we do not want them used in the application layer. It might happen easily if we merge all into a single module. In the same way we do not want to give access to implementation details which is why each team would have its api module to define its "contracts" with the "outside" world in a way that is easily enforced purely on the dependency level.

[jbee]

Update: We had a conversation today with tracker and platform present where we concluded that we might want to make an even simple module structure where there is basically one Java module per team which also is one maven module. We then can utilize the features of the Java module system to expose API and hide implementation. Until we get there we just make this clear using packages (as the unit that exports in the Java module system is a package as well).

[jbee]

Some clarifying thoughts.
Packages are the unit that is controlled using JPMS (Java modules). This means the goal is that each team defined a Java module with an api package, e.g. org.hisp.dhis.tracker.api which is made accessible using an exports statement. The other packages are not exported and thereby "internal". Some tooling outside core might need special access to "internals" which we can grant using export ... to statements. This just requires the using code to also define their code in a java module so we can name it. Reflection access might still be needed so we have to declare some opens statements. For testing we likely need a test module per team module to which we also explicitly use exports ... to to make the internal APIs testable (note that with JPMS test code is not included in the same module but tooling does not yet have good support for test spaces, therefor making a JPMS module for the "unit" tests that does have access to the internals is the easiest way to make it work).