Skip to content

Conversation

@PProfizi
Copy link
Contributor

@PProfizi PProfizi commented Sep 6, 2022

This PR reorganizes tests according to a testing matrix defined by the core developer team.
The goal is to spread different test requirements at different steps of the development process to both increase development ease and ensure a thorough robust testing.

The testing configuration to implement is:

On commit to a draft PR:

  • unit testing on the latest standalone, for Python 3.7, on windows-latest and ubuntu-latest.

On "ready for review", on commit to a non-draft PR, or on push to master, thus required to pass for merging on master:

  • unit testing, on the latest standalone, for Python 3.7, on windows-latest and ubuntu-latest.
  • examples, on the latest standalone, for Python 3.7, on windows-latest and ubuntu-latest.
  • compatibility with the latest PyDPF-Post, on the latest standalone, for Python 3.7, on windows-latest and ubuntu-latest.
  • unit testing, on every older standalone, for Python 3.7, on windows-latest and ubuntu-latest.
  • documentation, on the latest standalone, for Python 3.7, on windows-latest.

On push of a release tag, thus required to pass for releasing:

  • unit testing, on the latest standalone, for all supported Python versions, on windows-latest and ubuntu-latest.
  • examples, on the latest standalone, for all supported Python versions, on windows-latest and ubuntu-latest.
  • compatibility with the latest PyDPF-Post, on the latest standalone, for Python 3.7, on windows-latest and ubuntu-latest.
  • unit testing, on every older standalone, for Python 3.7, on windows-latest and ubuntu-latest.
  • compatibility with the latest PyDPF-Post, on every older standalone, for Python 3.7, on windows-latest and ubuntu-latest.
  • documentation, on the latest standalone, for Python 3.7, on windows-latest.
  • gatebin test, on the latest standalone, for Python 3.7, on windows-latest and ubuntu-latest.

On a weekly basis, run automatically:

  • The same as for a release (without the part for drafting a release).

We also give the capability to run manually, with choice of standalone, OS, and Python versions to test:

  • Unit tests.
  • Examples.
  • Compatibility with the latest PyDPF-Post
  • Documentation
  • Gatebin test.

@PProfizi PProfizi added the CI/CD Related to CI/CD label Sep 6, 2022
@PProfizi PProfizi self-assigned this Sep 6, 2022
@codecov
Copy link

codecov bot commented Sep 6, 2022

Codecov Report

Merging #470 (a33faa5) into master (c7f29e4) will increase coverage by 0.07%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #470      +/-   ##
==========================================
+ Coverage   88.00%   88.07%   +0.07%     
==========================================
  Files          68       68              
  Lines        7568     7574       +6     
==========================================
+ Hits         6660     6671      +11     
+ Misses        908      903       -5     

@PProfizi PProfizi marked this pull request as ready for review September 6, 2022 10:10
required: true
default: 'master'
type: string
default: "231"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@PProfizi shouldn't we use a global variable for all the "231" references that might avoid to have a lot to change each release?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@anslpa I do not think I can put an expression in the default value of workflow inputs.
The pipelines will have to be updated in a few places anyway.
Maybe NOT putting a default value is the solution, so that only the CI, and CI_release pipelines should be updated.

@PProfizi PProfizi force-pushed the ci/testing_pipelines branch from 926e893 to a33faa5 Compare October 26, 2022 11:07
@PProfizi PProfizi merged commit 3097c6c into master Oct 27, 2022
@PProfizi PProfizi deleted the ci/testing_pipelines branch October 27, 2022 13:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CI/CD Related to CI/CD

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants