Skip to content

Commit 400fffb

Browse files
gregharveygregharvey
authored
Allow branch switching pr 1.x (#167)
* Creating devel containers with Debian 'bookworm' (12). * Moving ansible and boto3 installs to apt. * apt installed ansible is not in /usr/local. * Updating dependencies. * Updating yarn.lock again. * Fixing ansible paths for ce-provision 2.x. * Adding new variables required by controller. * Installing ansible in a virtual environment on the container before running ce-provision. * Missing a slash! * Also missed a &&. * Adding python venv support via apt in base image. * Temporarily adding python3-venv to the controller Dockerfile. * Installing Ansible as the ce-dev user to avoid perms issues. * Upgrading ubuntu in CI to have systemd as default and disabling Ansible upgrades. * Missing an ansible install var. * Documentation for testing containers. * Adding vscode settings to .gitignore. * Adding devel versions of Dockerfiles and config. * Fixing bug caused by set line. * Updating CI for export.sh changes. * Adding ability to switch ce-provision own branch and config branch. * Restoring the 1.x branch. * Missed a couple of 2.x references. * Updating yarn.lock again. * Do not base off the unison container. * Fixing 1.x Ansible installation.
1 parent cb72b14 commit 400fffb

File tree

30 files changed

+518
-106
lines changed

30 files changed

+518
-106
lines changed

‎.github/workflows/README.md

Lines changed: 56 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,26 +1,74 @@
11
# GitHub Actions
2-
32
This document describes the current CI for the ce-dev project. The CI is written for GitHub Actions, the built in GitHub system for orchestration and automation. We operate three separate GitHub Actions workflows.
43

5-
## ce-dev-build
4+
## ce-dev-build-dev
5+
Builds base and controller images using the contents of the `devel` branch on push. The built images get pushed to Docker Hub and tagged with `devel`. This is very useful for pushing test containers to try out locally.
6+
7+
### Testing project images
8+
To test a `devel` container on a project, firstly fetch the container image:
9+
10+
```
11+
docker pull codeenigma/ce-dev-controller-1.x:devel
12+
```
13+
14+
Then edit your project's `ce-dev.compose.yml` file, changing the image for each applicable service to use the `devel` tag instead of the `latest` tag, for example:
15+
16+
```yaml
17+
image: codeenigma/ce-dev-1.x:devel
18+
```
19+
20+
Finally, destroy and recreate your `docker compose` file and containers:
21+
22+
```bash
23+
ce-dev destroy
24+
ce-dev init -t ce-dev.compose.yml
25+
ce-dev start
26+
ce-dev provision
27+
ce-dev deploy
28+
```
29+
30+
### Testing an experimental controller
31+
To test the `devel` version of the `ce-dev-controller` image locally you can do the following:
32+
33+
```bash
34+
docker pull codeenigma/ce-dev-controller-1.x:devel
35+
# Replace the 'latest' tag with your 'devel' image
36+
docker tag codeenigma/ce-dev-controller-1.x:devel codeenigma/ce-dev-controller-1.x:latest
37+
docker kill ce_dev_controller
38+
# Then in any ce-dev project
39+
ce-dev start
40+
ce-dev provision
41+
```
642

7-
This builds the images and binaries, pushes the Docker containers to Docker Hub, pushes the binaries to a GitHub release and builds and commits the ce-dev documentation. It runs when someone accepts a PR for or pushes to the `1.x` branch and has added a valid tag in the format `1.*`.
43+
If you need to hop branches of `ce-provision` or `ce-deploy` on the controller to try things you can hop on to the container and merge the test branch into `1.x` like so:
44+
45+
```bash
46+
docker exec -it ce_dev_controller bash
47+
su ce-dev
48+
cd ~/ce-provision # or ~/ce-deploy
49+
git merge origin my_test_branch
50+
```
51+
52+
Then use `ce-dev provision` and `ce-dev deploy` in the usual way.
853

954
### Known issues
55+
* The golang release needs keeping up to date (there's no 'latest' download we can use)
56+
* `mkcert` is built from source, initially because of a bad release but now it's like that we might as well keep it that way
57+
58+
## ce-dev-build
59+
This builds the images and binaries, pushes the Docker containers to Docker Hub, pushes the binaries to a GitHub release and builds and commits the ce-dev documentation. It runs when someone accepts a PR for or pushes to the `1.x` branch and has added a valid tag in the format `1.*`. These images are tagged `latest` in the Docker repository.
1060

61+
### Known issues
1162
* The golang release needs keeping up to date (there's no 'latest' download we can use)
12-
* mkcert is built from source, initially because of a bad release but now it's like that we might as well keep it that way
63+
* `mkcert` is built from source, initially because of a bad release but now it's like that we might as well keep it that way
1364
* If we start developing in a new version branch we will need to update the `on: push:` YAML in this workflow to allow the new tags or it won't execute
1465

1566
## ce-dev-lint
16-
1767
This runs ESLint over the JavaScript code in the `src` directory. It runs on every pull request. It uses the ESLint config defined in `./.eslintrc` so if you need to change its behaviour, edit that file.
1868

1969
## ce-dev-test
20-
2170
This builds the images and tests the ce-dev stack. It runs on every pull request and daily at 6:30.
2271

2372
### Known issues
24-
2573
* The golang release needs keeping up to date (there's no 'latest' download we can use)
26-
* mkcert is built from source, initially because of a bad release but now it's like that we might as well keep it that way
74+
* `mkcert` is built from source, initially because of a bad release but now it's like that we might as well keep it that way

‎.github/workflows/ce-dev-PR-test.yml

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -30,5 +30,6 @@ jobs:
3030
rm -Rf mkcert
3131
- name: Build and test
3232
run: |
33-
/bin/sh docker-images/export.sh latest
33+
/bin/sh docker-images/export.sh --version latest --image-name ce-dev
34+
/bin/sh docker-images/export.sh --version latest --image-name ce-dev-controller
3435
/bin/sh templates/prebuild.sh

‎.github/workflows/ce-dev-build-dev.yml

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ on:
88

99
jobs:
1010
build:
11-
runs-on: ubuntu-20.04
11+
runs-on: ubuntu-latest
1212
steps:
1313
- uses: actions/checkout@v2
1414
- name: Install modules
@@ -39,4 +39,5 @@ jobs:
3939
- name: Build and push Docker images
4040
run: |
4141
echo "${{ secrets.DOCKER_PASSWORD }}" | docker login -u "${{ secrets.DOCKER_USERNAME }}" --password-stdin
42-
/bin/sh docker-images/export.sh devel --push
42+
/bin/sh docker-images/export.sh --version devel --image-name ce-dev --base-image debian:bookworm-slim --dockerfile-path base-devel --push
43+
/bin/sh docker-images/export.sh --version devel --image-name ce-dev-controller --base-image debian:bookworm-slim --dockerfile-path controller-devel --push

‎.github/workflows/ce-dev-build.yml

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -42,7 +42,8 @@ jobs:
4242
- name: Build and push Docker images
4343
run: |
4444
echo "${{ secrets.DOCKER_PASSWORD }}" | docker login -u "${{ secrets.DOCKER_USERNAME }}" --password-stdin
45-
/bin/sh docker-images/export.sh latest --push
45+
/bin/sh docker-images/export.sh --version latest --image-name ce-dev --dockerfile-path base --push
46+
/bin/sh docker-images/export.sh --version latest --image-name ce-dev-controller --dockerfile-path controller --push
4647
echo "${{ secrets.DOCKER_PASSWORD }}" | docker login -u "${{ secrets.DOCKER_USERNAME }}" --password-stdin
4748
/bin/sh templates/prebuild.sh --push
4849
# Uses https://github.com/marketplace/actions/gh-release

‎.github/workflows/ce-dev-test.yml

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -34,5 +34,6 @@ jobs:
3434
rm -Rf mkcert
3535
- name: Build and test
3636
run: |
37-
/bin/sh docker-images/export.sh latest
37+
/bin/sh docker-images/export.sh --version latest --image-name ce-dev
38+
/bin/sh docker-images/export.sh --version latest --image-name ce-dev-controller
3839
/bin/sh templates/prebuild.sh

‎.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,3 +6,4 @@ lib
66
tmp
77
node_modules
88
oclif.manifest.json
9+
.vscode/*

‎.vscode/settings.json

Lines changed: 0 additions & 5 deletions
This file was deleted.

‎contribute/README.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,8 @@
11
# Contribute to ce-dev
22

3+
## Using test images
4+
See [the GitHub Actions README](https://github.com/codeenigma/ce-dev/blob/1.x/.github/workflows/README.md) for information on using `devel` tagged images pushed by the CI.
5+
36
## Release instructions
47

58
1. [Create a pull request](https://github.com/codeenigma/ce-dev/compare) to the `1.x` branch.
@@ -21,7 +24,6 @@
2124
[GitHub Actions](https://github.com/codeenigma/ce-dev/actions) should now take care of the release for you.
2225

2326
## Rolling back a release
24-
2527
If you need to pull a release, follow these instructions:
2628

2729
1. Locally in your terminal on the `1.x` branch update the tags, e.g. `git pull --tags`

‎docker-images/base-devel/Dockerfile

Lines changed: 103 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,103 @@
1+
FROM debian:bookworm-slim as unison
2+
RUN \
3+
set -x && \
4+
export DEBIAN_FRONTEND=noninteractive && \
5+
apt-get update && \
6+
apt-get install -y -o Dpkg::Options::="--force-confnew" \
7+
p7zip-full \
8+
build-essential \
9+
wget \
10+
ocaml-native-compilers && \
11+
cd /tmp/ && \
12+
wget https://github.com/bcpierce00/unison/archive/v2.53.3.tar.gz && \
13+
tar -xzvf v2.53.3.tar.gz && \
14+
cd /tmp/unison-2.53.3 && \
15+
make
16+
17+
FROM debian:bookworm-slim
18+
19+
RUN \
20+
set -x && \
21+
export DEBIAN_FRONTEND=noninteractive && \
22+
apt-get update && \
23+
apt-get dist-upgrade -y -o Dpkg::Options::="--force-confnew" && \
24+
apt-get install -y -o Dpkg::Options::="--force-confnew" \
25+
anacron \
26+
apt-transport-https \
27+
apt-utils \
28+
aptitude \
29+
bash \
30+
binutils \
31+
cron \
32+
curl \
33+
dirmngr \
34+
gnupg \
35+
rsync \
36+
openssh-server \
37+
postfix \
38+
procmail \
39+
python3-apt \
40+
python3-dev \
41+
python3-pycurl \
42+
python3-pip \
43+
python3-venv \
44+
rsyslog \
45+
sudo \
46+
systemd \
47+
systemd-sysv \
48+
unzip \
49+
vim \
50+
wget && \
51+
apt-get clean && \
52+
update-alternatives --install /usr/bin/python python /usr/bin/python3 1 && \
53+
rm -rf \
54+
/var/lib/apt/lists/* \
55+
/var/log/* \
56+
/tmp/*
57+
58+
RUN \
59+
echo 'UseDNS no' >> /etc/ssh/sshd_config && \
60+
mkdir -p /var/run/sshd && \
61+
rm /usr/sbin/policy-rc.d
62+
63+
RUN \
64+
set -x && \
65+
export DEBIAN_FRONTEND=noninteractive && \
66+
useradd -s /bin/bash ce-dev && \
67+
echo ce-dev:ce-dev | chpasswd -m && \
68+
install -m 755 -o ce-dev -g ce-dev -d /home/ce-dev && \
69+
install -m 700 -o ce-dev -g ce-dev -d /home/ce-dev/.ssh && \
70+
echo root:ce-dev | chpasswd -m && \
71+
echo 'ce-dev ALL=(ALL) NOPASSWD: ALL' > /etc/sudoers.d/ce-dev && \
72+
chmod 0440 /etc/sudoers.d/ce-dev && \
73+
rm -rf /tmp/*
74+
75+
RUN \
76+
rm -f \
77+
/etc/machine-id \
78+
/var/lib/dbus/machine-id
79+
80+
COPY --from=unison /tmp/unison-2.53.3/src/unison /usr/local/bin/
81+
COPY --from=unison /tmp/unison-2.53.3/src/unison-fsmonitor /usr/local/bin/
82+
COPY ./ce-dev-ownership.sh /opt/
83+
COPY ./ce-dev-ssh.sh /opt/
84+
COPY ./unison.sh /opt/
85+
COPY ./unison-startup.sh /opt/
86+
COPY ./procmailrc /etc/procmailrc
87+
88+
RUN \
89+
wget https://github.com/FiloSottile/mkcert/releases/download/v1.4.4/mkcert-v1.4.4-linux-amd64 -O /usr/local/bin/mkcert && \
90+
mkdir -p /home/ce-dev/deploy/live.local /home/ce-dev/.composer/cache /home/ce-dev/.nvm/versions/node /home/ce-dev/.local/share/mkcert && \
91+
chown -R ce-dev:ce-dev /home/ce-dev && \
92+
chmod +x /usr/local/bin/*
93+
94+
RUN \
95+
systemctl mask -- \
96+
dev-hugepages.mount \
97+
sys-fs-fuse-connections.mount
98+
99+
ENV container docker
100+
STOPSIGNAL SIGRTMIN+3
101+
VOLUME [ "/sys/fs/cgroup", "/run", "/run/lock", "/tmp" ]
102+
103+
ENTRYPOINT ["/sbin/init"]

‎docker-images/base-devel/ce-dev-ownership.sh

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
#!/bin/sh
2+
3+
# Ensure user numeric uid/gid matches.
4+
# @param $1
5+
# User id.
6+
# @param $2
7+
# Group id.
8+
ensure_user_ids(){
9+
OLD_UID="$(id -u ce-dev)"
10+
OLD_GID="$(id -g ce-dev)"
11+
if [ "$OLD_UID" = "$1" ] && [ "$OLD_GID" = "$2" ]; then
12+
return
13+
fi
14+
if [ "$OLD_UID" != "$1" ]; then
15+
usermod -u "$1" ce-dev
16+
chown -R --from="$OLD_UID" "$1" /var
17+
echo "User ID changed to $1."
18+
fi
19+
if [ "$OLD_GID" != "$2" ]; then
20+
groupmod -g "$2" ce-dev
21+
chown -R --from=":$OLD_GID" ":$2" /var
22+
echo "Group ID changed to $2."
23+
fi
24+
if [ -d /.x-ce-dev ]; then
25+
chown -R ce-dev:ce-dev /.x-ce-dev
26+
fi
27+
chown -R ce-dev:ce-dev /home/ce-dev
28+
}
29+
30+
# Match ids with host user.
31+
if [ -n "$1" ] && [ -n "$2" ]; then
32+
ensure_user_ids "$1" "$2"
33+
fi

0 commit comments

Comments
 (0)