Update Dockerfile #350

CristianCantoro · 2025-09-04T21:40:07Z

This PR does the following:

Update base image from ubuntu:18.04 to ubuntu:24.04 (LTS release)
Update LABELs
Build task-maker-rust from source, since the .deb package is no longer available
Add support for running multiple workers (by default, run <nproc>-1 workers, or you can specify the number with -j)

Usage:
  entrypoint.sh [options]

Entrypoint for task-maker-rust Docker image.

Options:
  -j, --jobs NWORKERS    Number of workers to launch [default: <nproc>-1].
  -h, --help             Show this help and exits.

I think the image could be published on Docker Hub and/or distributed with the GitHub container registry.

h/t to @edomora97 for the guidance on the changes.

CristianCantoro · 2025-09-04T22:03:23Z

A small note, if you build the image passing all the --build-args to populate the labels, the image will compile from scratch every time. It takes ~16.5 minutes on my machine.

docker build \
  --build-arg TM_VERSION=0.6.14 \
  --build-arg VCS_REF="$(git rev-parse HEAD)" \
  --build-arg BUILD_DATE="$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
  -t cristiancantoro/task-maker-rust:0.6.14 \
    .

For dev/testing just build it like this:

docker build \
  --build-arg TM_VERSION=0.6.14 \
  -t cristiancantoro/task-maker-rust:0.6.14 \
    .

edomora97

Overall nice! Thanks for taking the time!

edomora97 · 2025-09-06T08:03:52Z

tools/docker/Dockerfile

This Docker image is:

Forced to run based on a tmr release (it downloads the source of a git tag)

Builds tmr

I think it would be better to either:

Force the build based on a tmr release, but grab the .deb file that is already built by the CI.

The CI needs minor tweaks to publish the debs, but they have already been built

Allow the build from any commit, or even uncommitted changes.

Do not download the source, but take it from the context (i.e., slightly change the Dockerfile and the build command)

Option 1 Option 2
Speed of the build Fast Slow
Customizing what packages are installed Yes Yes
Making code changes Need a release Don't even need a commit
Dockerfile complexity Simple More complex
Maintainability issues Relies on CI building the release into .deb files Relies on tmr compatibility with Rust version¹

¹ If rustup downloads a newer version of Rust, the build may fail.

Given the use case of this Docker image (a quick and easy way of deploying task-maker), I'm biased to solution 1.

Why not both? I have created another Dockerfile where I COPY my local repository and build task-maker-rust. I think it would be nice to provide a image for uses that want to use task-maker-rust as soon as possible and another for the people interested in developing and contributing.

@edomora97 wrote:

Option 1 Option 2

Speed of the build Fast Slow

Regarding the speed of build, this is a bit faster than the previous version (building from source), but it's still not super fast as on my machine it takes ~ 10 minutes (against 16.5 minutes).

edomora97 · 2025-09-06T08:15:24Z

tools/docker/Dockerfile

+# symlink `task-maker` and `tusk-maker-tools` into /usr/local/bin/
+USER root
+RUN ln -s /opt/task-maker-rust/"task-maker-rust-${TM_VERSION}"/target/release/task-maker /usr/local/bin/ \
+   && ln -s /opt/task-maker-rust/"task-maker-rust-${TM_VERSION}"/target/release/task-maker /usr/local/bin/task-maker-rust \
+   && ln -s /opt/task-maker-rust/"task-maker-rust-${TM_VERSION}"/target/release/task-maker-tools /usr/local/bin/


I'd prefer if this is packaged as a deb, and then installed with the package manager. There many files that needs to be installed, and the full list is already defined in code:

https://github.com/edomora97/task-maker-rust/blob/09a9b7ee18d6901b7c1d7825bc34f39b482f6736/Cargo.toml#L17-L37

Also, now that I look at the CI I see that there may be problems with the TM_DATA_DIR not being exported in the release pipeline. @veluca93 why aren't you using the same commands as build_release.sh in the release CI? To be honest, I don't exactly remember how that script was used, probably we had a Docker image to package tmr with old system libraries, instead of building a different version for each OS.

Probably because I didn't really remember that script existed :D

We should probably set the variable...

The thing that was strange to me is that the .deb package installs task-maker-rust and task-maker-tools, instead cargo build --release build task-maker and task-maker-tools, that's why I was creating the symlink for both task-maker and task-maker-rust.

edomora97 · 2025-09-06T08:16:22Z

tools/docker/Dockerfile

+RUN userdel --remove ubuntu
+
+# create a group and a user called taskmaker, create the home directory
+RUN groupadd ${TM_GID:+-g "${TM_GID}"} taskmaker


I think we always have called task-maker with the dash, can you add it also to the user name?

To be fair, in the previous version of the Dockerfile the user was called user :-D. I will change it to task-maker.

edomora97 · 2025-09-06T08:20:20Z

tools/docker/entrypoint.sh

This is getting really complex. Why not migrate it to a Python script? It will be so much cleaner and maintainable 😅

tools/docker/healthcheck.sh

veluca93 · 2025-09-06T08:42:31Z

Why doesn't the dockerfile just add the APT repository and add the package from there? That seems way simpler than most other alternatives, it should boil down to roughly three commands:

echo "deb [signed-by=/etc/apt/keyrings/task-maker-rust.asc] https://artifacts.lucaversari.it/olimpiadi-informatica/task-maker-rust/latest/deb/$(lsb_release -cs) /" | sudo tee /etc/apt/sources.list.d/task-maker-rust.list
curl https://artifacts.lucaversari.it/signing-key.asc | sudo tee /etc/apt/keyrings/task-maker-rust.asc > /dev/null
sudo apt update && sudo apt install task-maker-rust

CristianCantoro · 2025-09-07T18:47:29Z

@veluca93 wrote:

Why doesn't the dockerfile just add the APT repository and add the package from there? That seems way simpler than most other alternatives, it should boil down to roughly three commands:

When I sat down to write the new Dockerfile I decided that I was going to use what I found in the release page and so I downloaded and built task-maker-rust from source without thinking twice. Funnily enough, on my machine I actually installed your .deb package.

edomora97 · 2025-09-08T11:23:22Z

Why doesn't the dockerfile just add the APT repository and add the package from there? That seems way simpler than most other alternatives, it should boil down to roughly three commands:
echo "deb [signed-by=/etc/apt/keyrings/task-maker-rust.asc] https://artifacts.lucaversari.it/olimpiadi-informatica/task-maker-rust/latest/deb/$(lsb_release -cs) /" | sudo tee /etc/apt/sources.list.d/task-maker-rust.list
curl https://artifacts.lucaversari.it/signing-key.asc | sudo tee /etc/apt/keyrings/task-maker-rust.asc > /dev/null
sudo apt update && sudo apt install task-maker-rust

I'm a bit of a noob, but can you install a specific old version of tmr from the APT repository?

veluca93 · 2025-09-08T13:10:07Z

Why doesn't the dockerfile just add the APT repository and add the package from there? That seems way simpler than most other alternatives, it should boil down to roughly three commands:
echo "deb [signed-by=/etc/apt/keyrings/task-maker-rust.asc] https://artifacts.lucaversari.it/olimpiadi-informatica/task-maker-rust/latest/deb/$(lsb_release -cs) /" | sudo tee /etc/apt/sources.list.d/task-maker-rust.list
curl https://artifacts.lucaversari.it/signing-key.asc | sudo tee /etc/apt/keyrings/task-maker-rust.asc > /dev/null
sudo apt update && sudo apt install task-maker-rust
I'm a bit of a noob, but can you install a specific old version of tmr from the APT repository?

Yup, see https://askubuntu.com/a/428778 -- tldr: sudo apt-get install package=version.

- Update base image from ubuntu:18.04 to ubuntu:24.04 (LTS release) - Update LABELs - Build `task-maker-rust` from source, since the `.deb` package is no longer available - Add support for running multiple workers

- install `task-maker-rust` using released deb package - convert entrypoint.sh to entrypoint.py

CristianCantoro · 2025-09-16T02:29:47Z

Sorry for the delay, this is what I have changed:

now task-maker-rust is installed using the deb package from the release page.
I have converted entrypoint.sh to entrypoint.py (with logging)
I have changed the user name from taskmaker to task-maker

I saw that @veluca93's APT repo only contains the latest version of task-maker-rust, I assume it is ok to install the package from the release.

If, for whatever reason, you prefer that the Dockerfile installs the deb from the APT repository then Dockerfile#L71-L78 should be changed to the following:

# install task-maker-rust from APT repo
ARG TM_DEB_VERSION="${TM_VERSION}-1~ubuntu-24.04"
ARG TM_DEB_NAME="task-maker-rust_${TM_DEB_VERSION}_amd64.deb"

RUN (test -n "$TM_VERSION" || (echo "Please use --build-arg TM_VERSION=X.Y.Z" >&2 && exit 1)) \
  && . /etc/os-release \
  && echo "deb [signed-by=/etc/apt/keyrings/task-maker-rust.asc] https://artifacts.lucaversari.it/olimpiadi-informatica/task-maker-rust/latest/deb/${UBUNTU_CODENAME} /" \
      | tee /etc/apt/sources.list.d/task-maker-rust.list \
  && curl https://artifacts.lucaversari.it/signing-key.asc \
      | tee /etc/apt/keyrings/task-maker-rust.asc > /dev/null \
  && apt update \
  && apt install -yy task-maker-rust="${TM_DEB_VERSION}" \
  && rm -rf /var/lib/apt/lists/*

(note that the deb package name is ~ubuntu-XX.YY instead of .ubuntu-XX.YY)

CristianCantoro · 2025-09-16T03:36:28Z

I have also updated healthcheck.sh to check that all expected workers are running.

CristianCantoro force-pushed the master branch 6 times, most recently from ed0ccec to c2fb113 Compare September 5, 2025 21:59

edomora97 reviewed Sep 6, 2025

View reviewed changes

CristianCantoro added 2 commits September 16, 2025 04:14

Update Dockerfile

98cddf0

- Update base image from ubuntu:18.04 to ubuntu:24.04 (LTS release) - Update LABELs - Build `task-maker-rust` from source, since the `.deb` package is no longer available - Add support for running multiple workers

Update Dockerfile

f4f6d47

- install `task-maker-rust` using released deb package - convert entrypoint.sh to entrypoint.py

CristianCantoro force-pushed the master branch from c2fb113 to f4f6d47 Compare September 16, 2025 02:15

Update healthcheck to check if all expected workers are running

9a0dc0b

	Option 1	Option 2
Speed of the build	Fast	Slow
Customizing what packages are installed	Yes	Yes
Making code changes	Need a release	Don't even need a commit
Dockerfile complexity	Simple	More complex
Maintainability issues	Relies on CI building the release into .deb files	Relies on tmr compatibility with Rust version¹

Update Dockerfile #350

Are you sure you want to change the base?

Update Dockerfile #350

Uh oh!

Conversation

CristianCantoro commented Sep 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

CristianCantoro commented Sep 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

edomora97 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

CristianCantoro Sep 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

veluca93 commented Sep 6, 2025

Uh oh!

CristianCantoro commented Sep 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

edomora97 commented Sep 8, 2025

Uh oh!

veluca93 commented Sep 8, 2025

Uh oh!

CristianCantoro commented Sep 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

CristianCantoro commented Sep 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

CristianCantoro commented Sep 4, 2025 •

edited

Loading

CristianCantoro commented Sep 4, 2025 •

edited

Loading

CristianCantoro Sep 7, 2025 •

edited

Loading

CristianCantoro commented Sep 7, 2025 •

edited

Loading

CristianCantoro commented Sep 16, 2025 •

edited

Loading

CristianCantoro commented Sep 16, 2025 •

edited

Loading