diff --git a/package-structure-code/code-style-linting-format.md b/package-structure-code/code-style-linting-format.md index 27adeaa75..8851058a9 100644 --- a/package-structure-code/code-style-linting-format.md +++ b/package-structure-code/code-style-linting-format.md @@ -1,8 +1,6 @@ # Python Package Code Style, Format and Linters -```{important} - -## Take Aways +```{admonition} Take Aways * pyOpenSci requires authors to follow PEP 8 code format guidelines * Setting up a code formatters like Black and isort will help you enforce PEP 8 style guidelines and also consistent, readable code format @@ -96,7 +94,6 @@ some exceptions. A few examples of those exceptions are below: - Black will not adjust line length in your comments or docstrings. - This tool will not review and fix import order (you need _isort_ or _Ruff_ to do that - see below). - ```{tip} If you are interested in seeing how Black will format your code, you can use the [Black playground](https://black.vercel.app/) @@ -205,6 +202,7 @@ gaining some traction since its release. As such, `ruff` can be used instead of `flake8` and `isort`. `ruff` has some interesting features that distinguish it from other linters: + - Linter configuration in `pyproject.toml` - Several hundred rules included, many of which are automatically fixable - Rules explanation, see [F403](https://beta.ruff.rs/docs/rules/undefined-local-with-import-star/) for an example @@ -236,7 +234,6 @@ Depending on your project, you might want to add the following to sort imports c known-first-party = [""] ``` - ## How to use code formatter in your local workflow ### Linters, code formatters and your favorite coding tools @@ -277,7 +274,7 @@ You type and run: - Once all of the fixes are applied you can re-add (stage) the files to be commit. And re-run your commit. -:::{figure-md} fig-target +:::{figure-md} precommit-hook Diagram showing the steps of a pre-commit workflow from left to right. The pre-commit workflow begins with you adding files that have changes to be diff --git a/package-structure-code/intro.md b/package-structure-code/intro.md index 8271c6679..356d86758 100644 --- a/package-structure-code/intro.md +++ b/package-structure-code/intro.md @@ -55,7 +55,7 @@ package follows best practices for code format? Learn more about the options and ::: :::: -:::{figure-md} fig-target +:::{figure-md} packaging-tools-decision-tree Figure showing a decision tree with the various packaging tool front-end and back-end options. diff --git a/package-structure-code/publish-python-package-pypi-conda.md b/package-structure-code/publish-python-package-pypi-conda.md index 13eeabec8..1a45bd51f 100644 --- a/package-structure-code/publish-python-package-pypi-conda.md +++ b/package-structure-code/publish-python-package-pypi-conda.md @@ -2,13 +2,14 @@ -pyOpenSci requires that your package has an distribution that can be installed from a public community repository such as PyPI or a conda channel such as `bioconda` or `conda-forge` in the Anaconda cloud. +pyOpenSci requires that your package has an distribution that can be installed +from a public community repository such as PyPI or a conda channel such as +`bioconda` or `conda-forge` in the Anaconda cloud. Below you will learn more about the various publishing options for your Python package. -```{important} -## Take Aways +```{admonition} Take Aways * Installing packages in the same environment using both pip and conda can lead to package conflicts. @@ -17,7 +18,7 @@ lead to package conflicts. Below you will learn more specifics about the differences between PyPI and conda publishing of your Python package. ``` -### What is PyPI +## What is PyPI [PyPI](https://pypi.org/) is an online Python package repository that you can use to both find and install and publish your Python package. There is @@ -31,17 +32,15 @@ a package is that conda can install any package regardless of the language(s) that it is written in. Whereas `pip` can only install Python packages. - - ```{tip} On the package build page, we discussed the [two package distribution -types that you will create when making a Python package](link-to-page-on-sdist): SDist (packaged as a .tar.gz or .zip) and +types that you will create when making a Python package](python-package-distribution-files-sdist-wheel): SDist (packaged as a .tar.gz or .zip) and Wheel (.whl) which is really a zip file. Both of those file "bundles" will -be published on PyPI when you use [a standard build tool](link-to-build-page) to build +be published on PyPI when you use [a standard build tool](python-package-build-tools) to build your package. ``` -### What is Anaconda Cloud and conda? +## What is Anaconda Cloud and conda? conda is an open source package and environment management tool. conda can be used to install tools from the [Anaconda Cloud @@ -61,7 +60,8 @@ While conda was originally created to support Python packages, it is now used across all languages. This cross-language support makes it easier for some packages to include and have access to tools written in other languages such as c/c++ (gdal), Julia, or R. -Creating environment that mixes all those packages are usually easier and more consistent with full fledged package managers like conda. +Creating environment that mixes all those packages are usually easier and more +consistent with full fledged package managers like conda. ``` ### conda channels @@ -79,7 +79,7 @@ packages using conda including. **conda-forge** emerged as many of the scientific packages did not exist in the default Anaconda cloud channel. -:::{figure-md} fig-target +:::{figure-md} conda-channels ADD. @@ -96,7 +96,7 @@ repository to publish your Python package. The answer to both questions relates dependency conflicts. -:::{figure-md} fig-target +:::{figure-md} xkcd-dep-conflict Image showing an XKCD comic that shows a web of Python environments and tools and installations. At the bottom is says - My python environment has become so degraded that my laptop has been declared a superfund site. @@ -130,7 +130,7 @@ to use conda to manage their local environments (which many do), you should consider publishing to both PyPI and the conda-forge channel (_more on that below_). -```{tip} +```{admonition} Additional resources * [learn more about why conda-forge was created, here](https://conda-forge.org/docs/user/introduction.html#why-conda-forge) * [To learn more about conda terminology, check out their glossary.](https://docs.conda.io/projects/conda/en/latest/glossary.html ) diff --git a/package-structure-code/pyproject-toml-python-package-metadata.md b/package-structure-code/pyproject-toml-python-package-metadata.md index 987d72051..60283d32a 100644 --- a/package-structure-code/pyproject-toml-python-package-metadata.md +++ b/package-structure-code/pyproject-toml-python-package-metadata.md @@ -9,7 +9,7 @@ The **pyproject.toml** file is written in [TOML (Tom's Obvious, Minimal Language Below that table identifier are key/value pairs that support configuration for that particular table. -### Benefits of using a pyproject.toml file +## Benefits of using a pyproject.toml file Including your package's metadata in a separate human-readable **pyproject.toml** format also allows someone to view the project's metadata in a GitHub repository. @@ -29,7 +29,7 @@ resources working with complex builds in the future. ``` -### Example pyproject.toml for building using PDM +## Example pyproject.toml for building using PDM Below is an example build configuration for a Python project. This example package setup uses: @@ -65,7 +65,7 @@ dependencies = [ Notice that dependencies are specified in this file. -### Example pyproject.toml for building using setuptools +## Example pyproject.toml for building using setuptools The package metadata including authors, keywords, etc is also easy to read. Below you can see the same TOML file that uses a different build system (setuptools). diff --git a/package-structure-code/python-package-build-tools.md b/package-structure-code/python-package-build-tools.md index 64e22bf03..a915c5f6e 100644 --- a/package-structure-code/python-package-build-tools.md +++ b/package-structure-code/python-package-build-tools.md @@ -8,7 +8,7 @@ We focus on pure-python packages in this guide. However, we also highlight tools that currently support packages with C/C++ and other language extensions. -:::{figure-md} fig-target +:::{figure-md} package-decision-tree Decision tree diagram showing the various front and back end packaging tools. You can decide what packaging tool to use by thinking about what features you need. PDM is currently the most flexible tool that also supports both non pure Python projects and also using different build back-ends. As such currently PDM is the tool we think beginners might appreciate most with Poetry being a close second. Poetry is ideal for pure python projects. @@ -19,7 +19,7 @@ NOTE: this is still a DRAFT so i'm not going to spend time truly cleaning it up If you want to know more about Python packages that have extensions written in other languages, [check out the page on complex package builds.](complex-python-package-builds) -### Tools that we review here +## Tools that we review here In this section we have selected tools that were returned as the most popular packaging tools in the PyPA survey. @@ -31,7 +31,7 @@ You will learn more about the following tools on this page: - [PDM](https://pdm.fming.dev/latest/) - [Poetry](https://python-poetry.org/docs/) -### Summary of tools Hatch vs. PDM vs. Poetry (and setuptools) +## Summary of tools Hatch vs. PDM vs. Poetry (and setuptools) If you are looking for a quick summary, read below. diff --git a/package-structure-code/python-package-distribution-files-sdist-wheel.md b/package-structure-code/python-package-distribution-files-sdist-wheel.md index e05742a0a..abc268936 100644 --- a/package-structure-code/python-package-distribution-files-sdist-wheel.md +++ b/package-structure-code/python-package-distribution-files-sdist-wheel.md @@ -18,7 +18,7 @@ Also note that we are not discussing conda build workflows in this section. [You can learn more about conda builds here.](https://conda.io/projects/conda-build/en/latest/user-guide/tutorials/index.html) ``` -### Source Distribution (sdist) +## Source Distribution (sdist) **Source files** are the unbuilt files needed to build your package. These are the "raw / as-is" files that you store on GitHub or whatever @@ -91,7 +91,7 @@ items including a metadata directory and if you use `setuptools_scm` or `hatch_v the SDist may also contain a file that stores the version. ``` -### Wheel (.whl files): +## Wheel (.whl files): A wheel file is a ZIP-format archive whose filename follows a specific format (below) and has the extension `.whl`. The `.whl` archive contains a specific diff --git a/package-structure-code/python-package-structure.md b/package-structure-code/python-package-structure.md index 137ecab80..25a38a960 100644 --- a/package-structure-code/python-package-structure.md +++ b/package-structure-code/python-package-structure.md @@ -79,7 +79,7 @@ While we recommend the **src/** layout we also review the **flat** layout here. ``` -### The src/ layout and testing +## The src/ layout and testing The benefit of using the **src/package** layout, particularly if you are creating a new package, is that it ensures tests are run against the @@ -103,7 +103,7 @@ However, if your package lives in a src/ directory structure **src/package** the Note: Python versions 3.11 and above have a path setting that can be adjusted to ensure the priority is to use installed packages first (e.g. `PYTHONSAFEPATH`). ``` -#### Sometimes tests are needed in a distribution +### Sometimes tests are needed in a distribution We do not recommend including tests as part of your package wheel by default. However, not including tests in your package distribution will make it harder for people other than yourself to test whether your package is functioning correctly on their system. If you have a small test suite (Python files + data), and think your users may want to run tests locally on their systems, you can include tests by moving the `tests/` directory into the **src/package** directory (see example below). diff --git a/package-structure-code/python-package-versions.md b/package-structure-code/python-package-versions.md index f4b82026c..6a20e40c0 100644 --- a/package-structure-code/python-package-versions.md +++ b/package-structure-code/python-package-versions.md @@ -3,8 +3,7 @@ -```{important} -## Key Takeways +```{admonition} Key Takeways * Follow [semantic versioning guidelines (SemVer) rules](https://semver.org/) when bumping (increasing) your Python's package version; for example a major version bump (version 1.0 --> 2.0) equates to breaking changes in your package's code for a user. * You may want to consider using a plugin like hatch_vsc for managing versions of your package - if you want to have a GitHub only release workflow. @@ -26,9 +25,7 @@ with how and when you update your package versions is important as: bump a package version based on standard rules. 3. Consistent version increases following semver rules mean that values of your package version explain the extent of the changes made in the code base from version to version. thus your package version numbers become "expressive" in the same way that naming code variables well can [make code expressive](https://medium.com/@daniel.oliver.king/writing-expressive-code-b69ef7a5a2fa). -```{note} -A note about versioning - +```{admonition} A note about versioning In some cases even small version changes can turn a package update into a breaking change for some users. What is also important is that you document how you version your code and if you can, also @@ -37,7 +34,7 @@ document your deprecation policy for code. -### SemVer rules +## SemVer rules Following SemVer, your bump your package version to a: @@ -63,7 +60,7 @@ pyOpenSci will never require semver in a peer review as long as a package has a reasonable approach to versioning! ``` -### Avoid manually updating Python package version numbers if you can +## Avoid manually updating Python package version numbers if you can Often times you may want to have your package version value in multiple locations. One example of this is that it might be both @@ -86,8 +83,8 @@ Python package versions. ## Tools to manage versions for your Python package There are a handful of tools that are widely used in the scientific ecosystem that you can use to manage your package -versions. Some of these tools are built into or work with your -[packaging build tools that we discuss]() +versions. Some of these tools are built into or work with your chosen +[packaging build tools that discussed in this chapter.](python-package-build-tools) diff --git a/python-packaging/intro.md b/python-packaging/intro.md deleted file mode 100644 index 96db020dc..000000000 --- a/python-packaging/intro.md +++ /dev/null @@ -1,11 +0,0 @@ -# Python Packaging Tools... Coming soon... - - -In this section we will cover: - -* build tools to build your package -* publishing to PyPI and the Anaconda Cloud repositories -* Built front-end tools to support your development workflow -* Managing dependencies - -Stay tuned as this section is currently under development as of January 2023!!