add Makefile and local setup instruction in README #86
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Not sure why the pelican Dockerfile build fails... |
alamb
reviewed
Jul 8, 2025
There was a problem hiding this comment.
Thank you @kevinjqliu --I tested this locally and it doesn't seem to work for me:
make
6.519 Downloading charset_normalizer-2.0.4-py3-none-any.whl (36 kB)
6.579 ERROR: Could not find a version that satisfies the requirement urllib3==2.5.0 (from versions: 0.3, 1.0, 1.0.1, 1.0.2, 1.1, 1.2, 1.2.1, 1.2.2, 1.3, 1.4, 1.5, 1.6, 1.7, 1.7.1, 1.8, 1.8.2, 1.8.3, 1.9, 1.9.1, 1.10, 1.10.1, 1.10.2, 1.10.3, 1.10.4, 1.11, 1.12, 1.13, 1.13.1, 1.14, 1.15, 1.15.1, 1.16, 1.17, 1.18, 1.18.1, 1.19, 1.19.1, 1.20, 1.21, 1.21.1, 1.22, 1.23, 1.24, 1.24.1, 1.24.2, 1.24.3, 1.25, 1.25.1, 1.25.2, 1.25.3, 1.25.4, 1.25.5, 1.25.6, 1.25.7, 1.25.8, 1.25.9, 1.25.10, 1.25.11, 1.26.0, 1.26.1, 1.26.2, 1.26.3, 1.26.4, 1.26.5, 1.26.6, 1.26.7, 1.26.8, 1.26.9, 1.26.10, 1.26.11, 1.26.12, 1.26.13, 1.26.14, 1.26.15, 1.26.16, 1.26.17, 1.26.18, 1.26.19, 1.26.20, 2.0.0a1, 2.0.0a2, 2.0.0a3, 2.0.0a4, 2.0.0, 2.0.1, 2.0.2, 2.0.3, 2.0.4, 2.0.5, 2.0.6, 2.0.7, 2.1.0, 2.2.0, 2.2.1, 2.2.2, 2.2.3)
6.579 ERROR: No matching distribution found for urllib3==2.5.0
6.699 WARNING: You are using pip version 21.1.3; however, version 25.0.1 is available.
6.699 You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command.
------
3 warnings found (use docker --debug to expand):
- FromAsCasing: 'as' and 'FROM' keywords' casing do not match (line 8)
- LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy "ENV key value" format (line 29)
- LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy "ENV key value" format (line 30)
Dockerfile:50
--------------------
48 | COPY requirements.txt .
49 | # Don't automatically load dependencies; please add them to requirements.txt instead
50 | >>> RUN pip install -r requirements.txt --no-deps
51 |
52 | # Could perhaps be added to requirements.txt but that would affect other uses
--------------------
ERROR: failed to solve: process "/bin/sh -c pip install -r requirements.txt --no-deps" did not complete successfully: exit code: 1
make: *** [build-image] Error 1
alamb
reviewed
Jul 8, 2025
There was a problem hiding this comment.
However, it seems to be something wrong with the infrastructure-actions repo
When I checked out
~/Software/datafusion-site/infrastructure-actions$ git checkout 8aee7a080268198548d8d1b4f1315a4fb94bffeaThen the image builds fine:
andrewlamb@Andrews-MacBook-Pro-3:~/Software/datafusion-site/infrastructure-actions$ docker build -t df-site-build pelican
[+] Building 6.1s (27/27) FINISHED docker:desktop-linux
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 2.79kB 0.0s
=> WARN: FromAsCasing: 'as' and 'FROM' keywords' casing do not match (line 8) 0.0s
=> [internal] load metadata for docker.io/library/python:3.8.10-slim-buster 2.5s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load build context 0.0s
=> => transferring context: 1.47kB 0.0s
=> [pelican-asf 1/6] FROM docker.io/library/python:3.8.10-slim-buster@sha256:6c0b171f6e4cbd880a972a36b77f18ccb03c3f32a6385e3306 0.0s
=> CACHED [stage-1 2/17] RUN apt update && apt upgrade -y 0.0s
=> CACHED [stage-1 3/17] RUN apt install subversion wget -y 0.0s
=> CACHED [stage-1 4/17] RUN pip install pelican[markdown]==4.5.4 0.0s
=> CACHED [stage-1 5/17] WORKDIR /opt/pelican-asf/cmark-gfm-0.28.3.gfm.12/lib 0.0s
=> CACHED [pelican-asf 2/6] RUN apt update && apt upgrade -y 0.0s
=> CACHED [pelican-asf 3/6] RUN apt install curl cmake build-essential -y 0.0s
=> CACHED [pelican-asf 4/6] WORKDIR /opt/pelican-asf 0.0s
=> CACHED [pelican-asf 5/6] COPY build-cmark.sh bin/build-cmark.sh 0.0s
=> CACHED [pelican-asf 6/6] RUN bash bin/build-cmark.sh ${GFM_VERSION} 0.0s
=> CACHED [stage-1 6/17] COPY --from=pelican-asf /opt/pelican-asf/cmark-gfm-0.28.3.gfm.12/lib . 0.0s
=> CACHED [stage-1 7/17] WORKDIR /opt/pelican-asf 0.0s
=> [stage-1 8/17] COPY requirements.txt . 0.0s
=> [stage-1 9/17] RUN pip install -r requirements.txt --no-deps 2.3s
=> [stage-1 10/17] RUN pip install 'MarkupSafe<2.1.0' # needed for Pelican 4.5.4 0.6s
=> [stage-1 11/17] COPY plugin_paths.py . 0.0s
=> [stage-1 12/17] COPY plugins plugins 0.0s
=> [stage-1 13/17] RUN { find plugins -type d -name __pycache__ -exec rm -rf {} ; ; true; } 0.1s
=> [stage-1 14/17] RUN ln -s /site/.authtokens /root/.authtokens 0.1s
=> [stage-1 15/17] WORKDIR /site 0.0s
=> [stage-1 16/17] RUN { echo '#!/usr/bin/env bash' >/usr/local/bin/pelicanasf; chmod +x /usr/local/bin/pelicanasf;} 0.1s
=> [stage-1 17/17] RUN echo 'SRC=${1:?source}; shift; python3 -B -m pelican $SRC -b 0.0.0.0 -e "$(python3 $WORKDIR/plugin_paths 0.1s
=> exporting to image 0.1s
=> => exporting layers 0.1s
=> => writing image sha256:93f0f68d4b4559679e17509d2d6ab5716f778e88dd0dcf3ac32fe238cf1fe111 0.0s
=> => naming to docker.io/library/df-site-build
yea super weird. this is the offending commit apache/infrastructure-actions@0b75be2 for when I cant figure out why |
|
ok updated the PR to include the workaround. lets pin to use the working commit for |
alamb
reviewed
Jul 9, 2025
There was a problem hiding this comment.
I tried this locally and now it doesn't seem to build
andrewlamb@Andrews-MacBook-Pro-3:~/Software/datafusion-site$ make
infrastructure-actions already exists, skipping clone.
# Pinned to commit 8aee7a080268198548d8d1b4f1315a4fb94bffea due to https://github.com/apache/infrastructure-actions/issues/218
alamb
approved these changes
Jul 9, 2025
There was a problem hiding this comment.
Thank you @kevinjqliu -- when testing last night I only ran make rather than make build (ala RTFM)
I pushed a commit this morning to allow make to work and it works great now. Thanks again
|
This |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Run
make