Migrate CI to hosted Cirrus Runners #15
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Collaborator
Author
|
I guess it's fine to use this, since it falls back to (for now) free Github runners. |
Sjors
commented
Sep 3, 2025
| CI_FAILFAST_TEST_LEAVE_DANGLING: 1 # GHA does not care about dangling processes and setting this variable avoids killing the CI script itself on error | ||
| MAKEJOBS: '-j10' | ||
| CIRRUS_CACHE_HOST: http://127.0.0.1:12321/ # When using Cirrus Runners this host can be used by the docker `gha` build cache type. | ||
| REPO_USE_CIRRUS_RUNNERS: 'bitcoin/bitcoin' # Use cirrus runners and cache for this repo, instead of falling back to the slow GHA runners |
Collaborator
Author
There was a problem hiding this comment.
I'm just going to leave the incorrect repo name here, since the GHA runners are plenty fast for this project.
Merged
Sjors
force-pushed
the
2025/05/cirrus-runners
branch
from
d340972 to
b8661e5
Compare
Add "Restore" and "Save" caching actions. These actions reduce boilerplate in the main ci.yml configuration file. These actions are implemented so that caches will be saved on `push` only. When a pull request is opened it will cache hit on the caches from the lastest push, or in the case of depends will hit on any matching depends hash, falling back to partial matches. Depends caches are hashed using `$(git ls-tree HEAD depends "ci/test/$FILE_ENV" | sha256sum | cut -d' ' -f1)` and this hash is passed in as an input to the actions. This means we direct cache hit in cases where depends would not be re-built, otherwise falling back to a partial match. Previous releases cache is hashed similarly to depends, but using the test/get_previous_releases.py file. The cirruslabs cache action will fallback transparently to GitHub's cache in the case that the job is not being run on a Cirrus Runner, making these compatible with running on forks (on free GH hardware).
If set, Cirrus runners will be used on pushes to, and pull requests against, this repository. Forks can set this if they have their own cirrus runners.
Another action to reduce boilerplate in the main ci.yml file. This action will set up a docker builder compatible with caching build layers to a container registry using the `gha` build driver. It will then configure the docker build cache args.
Using buildx is required to properly load the correct driver, for use with registry caching. Neither build, nor BUILDKIT=1 currently do this properly. Use of `docker buildx build` is compatible with podman.
Reverts: e87429a2d0f23eb59526d335844fa5ff5b50b21f This was added in PR #31545 with the intention that self-hosted runners might use it to save build cache. As we are not using hosted runners with a registry build cache, the bulk of this commit can be reverted, simply using the value of $DOCKER_BUILD_CACHE_ARG in the script. link: bitcoin/bitcoin#31545
This sets the build dir at build time so that Apple SDK gets installed in the correct/expected location for the runtime to find it. Co-authored-by: Max Edwards <[email protected]>
Whilst the action cirruslabs/actions/cache will automatically set this host, the docker `gha` build cache backend will not be aware of it. Set the value here, which will later be used in the docker build args to enable docker build cache on the cirrus cache.
To remove multiple occurances of the respository name, against which we
compare `${{ github.repository }}` to check if we should use Cirrus
Runners, introduce a helper job which can check a single environment
variable and output this as an input to subsequent jobs.
Forks can maintain a trivial patch of their repo name against the
`REPO_USE_CIRRUS_RUNNERS` variable in ci.yml if they have Cirrus Runners
of their own, which will then enable cache actions and docker build
cache to use Cirrus Cache.
It's not possible to use `${{ env.USE_CIRRUS_RUNNERS }}` in the
`runs-on:` directive as the context is not supported by GitHub.
If it was, this job would no longer be necessary.
Co-authored-by: Max Edwards <[email protected]>
Co-authored-by: Max Edwards <[email protected]>
When using hosted runners in combination with cached docker images, there is the possibility that the host runner image is updated, rendering the linux-headers package (stored in the cached docker image) incompatible. Fix this by doing a re-install of the headers package in 03_test_script.sh. If the underlying runner kernel has not changed thie has no effect, but prevents the job from failing if it has.
Co-authored-by: Max Edwards <[email protected]>
Co-authored-by: Max Edwards <[email protected]>
Co-authored-by: Max Edwards <[email protected]>
Co-authored-by: Max Edwards <[email protected]>
Co-authored-by: Max Edwards <[email protected]>
Co-authored-by: Max Edwards <[email protected]>
Co-authored-by: Max Edwards <[email protected]>
Co-authored-by: Max Edwards <[email protected]>
Removed as unused.
Previously jobs were running on a large multi-core server where 10 jobs as default made sense (or may even have been on the low side). Using hosted runners with fixed (and lower) numbers of vCPUs we should adapt compilation to match the number of cpus we have dynamically. This is cross-platform compatible with macos and linux only.
Print the ccache hit-rate for the job using a GitHub annotation if it was below 75%.
Docker currently warns that we are missing a default value. Set this to scratch which will error if an appropriate image tag is not passed in to silence the warning.
ci/lint_run.sh: Only used in .cirrus.yml. Refer to test/lint/README.md on how to run locally. ci/lint_run_all.sh: Only used in .cirrus.yml for stale re-runs of old pull request tasks.
Rather than trying to match the apt installed clang version, which is prone to intermittent issues. i.e bitcoin/bitcoin#33345.
Sjors
force-pushed
the
2025/05/cirrus-runners
branch
from
b8661e5 to
48d32d4
Compare
Sjors
changed the title
Collaborator
Author
|
Going to merge this, so I can eventually stop maintaining my own runners. |
Sjors
added a commit
that referenced
this pull request
Oct 18, 2025
- mark the SHA256 digest returned from Sv2SignatureNoiseMessage::GetHash()
as initialized so MSan stops flagging the certificate hash
- clear the chain name literals and resulting base params object used by
Sv2BasicTestingSetup when selecting regtest
This still fails:
==198==WARNING: MemorySanitizer: use-of-uninitialized-value
#0 0x55b5ca320949 in basic_string /cxx_build/include/c++/v1/string:1003:9
#1 0x55b5ca320949 in CBaseChainParams /src/sv2-tp/build_fuzz/src/./chainparamsbase.h:29:55
#2 0x55b5ca320949 in std::__1::unique_ptr<CBaseChainParams, std::__1::default_delete<CBaseChainParams>> std::__1::make_unique[abi:de210103]<CBaseChainParams, char const (&) [8], int, int, 0>(char const (&) [8], int&&, int&&) /cxx_build/include/c++/v1/__memory/unique_ptr.h:759:30
#3 0x55b5ca31e66f in CreateBaseChainParams(ChainType) /src/sv2-tp/build_fuzz/src/./chainparamsbase.cpp:48:16
#4 0x55b5ca31dce5 in SelectBaseParams(ChainType) /src/sv2-tp/build_fuzz/src/./chainparamsbase.cpp:55:29
#5 0x55b5ca2e4cad in Sv2BasicTestingSetup::Sv2BasicTestingSetup() /src/sv2-tp/build_fuzz/src/test/fuzz/./test/sv2_test_setup.cpp:34:5
#6 0x55b5ca2c381a in std::__1::unique_ptr<Sv2BasicTestingSetup const, std::__1::default_delete<Sv2BasicTestingSetup const>> std::__1::make_unique[abi:de210103]<Sv2BasicTestingSetup const, 0>() /cxx_build/include/c++/v1/__memory/unique_ptr.h:759:30
#7 0x55b5ca2c339e in (anonymous namespace)::Initialize() /src/sv2-tp/build_fuzz/src/test/fuzz/./test/fuzz/sv2_noise.cpp:32:39
#8 0x55b5ca2ca01d in __invoke<void (*&)()> /cxx_build/include/c++/v1/__type_traits/invoke.h:87:27
#9 0x55b5ca2ca01d in __call<void (*&)()> /cxx_build/include/c++/v1/__type_traits/invoke.h:342:5
#10 0x55b5ca2ca01d in __invoke_r<void, void (*&)()> /cxx_build/include/c++/v1/__type_traits/invoke.h:348:10
#11 0x55b5ca2ca01d in std::__1::__function::__func<void (*)(), void ()>::operator()() /cxx_build/include/c++/v1/__functional/function.h:174:12
#12 0x55b5ca2b9622 in operator() /cxx_build/include/c++/v1/__functional/function.h:274:12
#13 0x55b5ca2b9622 in operator() /cxx_build/include/c++/v1/__functional/function.h:772:10
#14 0x55b5ca2b9622 in initialize /src/sv2-tp/build_fuzz/src/test/fuzz/./test/fuzz/fuzz.cpp:247:5
#15 0x55b5ca2b9622 in LLVMFuzzerInitialize /src/sv2-tp/build_fuzz/src/test/fuzz/./test/fuzz/fuzz.cpp:321:5
#16 0x55b5ca21df42 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:652:5
#17 0x55b5ca24cfa2 in main /llvm-project/compiler-rt/lib/fuzzer/FuzzerMain.cpp:20:10
#18 0x7f0f188e4082 in __libc_start_main (/tmp/not-out/tmp_tv4p4q7/libc.so.6+0x24082) (BuildId: 5792732f783158c66fb4f3756458ca24e46e827d)
#19 0x55b5ca17802d in _start (/tmp/not-out/tmp_tv4p4q7/sv2_noise_cipher_roundtrip+0x24202d) (BuildId: 4d0ffc8495228d8cab1595a0fea7fa76c40aa18b)
DEDUP_TOKEN: basic_string--CBaseChainParams--std::__1::unique_ptr<CBaseChainParams, std::__1::default_delete<CBaseChainParams>> std::__1::make_unique[abi:de210103]<CBaseChainParams, char const (&) [8], int, int, 0>(char const (&) [8], int&&, int&&)
Uninitialized value was created by an allocation of 'ref.tmp' in the stack frame
#0 0x55b5ca32023c in std::__1::unique_ptr<CBaseChainParams, std::__1::default_delete<CBaseChainParams>> std::__1::make_unique[abi:de210103]<CBaseChainParams, char const (&) [8], int, int, 0>(char const (&) [8], int&&, int&&) /cxx_build/include/c++/v1/__memory/unique_ptr.h:759:34
DEDUP_TOKEN: std::__1::unique_ptr<CBaseChainParams, std::__1::default_delete<CBaseChainParams>> std::__1::make_unique[abi:de210103]<CBaseChainParams, char const (&) [8], int, int, 0>(char const (&) [8], int&&, int&&)
SUMMARY: MemorySanitizer: use-of-uninitialized-value /src/sv2-tp/build_fuzz/src/./chainparamsbase.h:29:55 in CBaseChainParams
Assisted-by: GitHub Copilot
Assisted-by: OpenAI GPT-5-Codex
Sjors
added a commit
that referenced
this pull request
Oct 21, 2025
- mark the SHA256 digest returned from Sv2SignatureNoiseMessage::GetHash()
as initialized so MSan stops flagging the certificate hash
- clear the chain name literals and resulting base params object used by
Sv2BasicTestingSetup when selecting regtest
This still fails:
==198==WARNING: MemorySanitizer: use-of-uninitialized-value
#0 0x55b5ca320949 in basic_string /cxx_build/include/c++/v1/string:1003:9
#1 0x55b5ca320949 in CBaseChainParams /src/sv2-tp/build_fuzz/src/./chainparamsbase.h:29:55
#2 0x55b5ca320949 in std::__1::unique_ptr<CBaseChainParams, std::__1::default_delete<CBaseChainParams>> std::__1::make_unique[abi:de210103]<CBaseChainParams, char const (&) [8], int, int, 0>(char const (&) [8], int&&, int&&) /cxx_build/include/c++/v1/__memory/unique_ptr.h:759:30
#3 0x55b5ca31e66f in CreateBaseChainParams(ChainType) /src/sv2-tp/build_fuzz/src/./chainparamsbase.cpp:48:16
#4 0x55b5ca31dce5 in SelectBaseParams(ChainType) /src/sv2-tp/build_fuzz/src/./chainparamsbase.cpp:55:29
#5 0x55b5ca2e4cad in Sv2BasicTestingSetup::Sv2BasicTestingSetup() /src/sv2-tp/build_fuzz/src/test/fuzz/./test/sv2_test_setup.cpp:34:5
#6 0x55b5ca2c381a in std::__1::unique_ptr<Sv2BasicTestingSetup const, std::__1::default_delete<Sv2BasicTestingSetup const>> std::__1::make_unique[abi:de210103]<Sv2BasicTestingSetup const, 0>() /cxx_build/include/c++/v1/__memory/unique_ptr.h:759:30
#7 0x55b5ca2c339e in (anonymous namespace)::Initialize() /src/sv2-tp/build_fuzz/src/test/fuzz/./test/fuzz/sv2_noise.cpp:32:39
#8 0x55b5ca2ca01d in __invoke<void (*&)()> /cxx_build/include/c++/v1/__type_traits/invoke.h:87:27
#9 0x55b5ca2ca01d in __call<void (*&)()> /cxx_build/include/c++/v1/__type_traits/invoke.h:342:5
#10 0x55b5ca2ca01d in __invoke_r<void, void (*&)()> /cxx_build/include/c++/v1/__type_traits/invoke.h:348:10
#11 0x55b5ca2ca01d in std::__1::__function::__func<void (*)(), void ()>::operator()() /cxx_build/include/c++/v1/__functional/function.h:174:12
#12 0x55b5ca2b9622 in operator() /cxx_build/include/c++/v1/__functional/function.h:274:12
#13 0x55b5ca2b9622 in operator() /cxx_build/include/c++/v1/__functional/function.h:772:10
#14 0x55b5ca2b9622 in initialize /src/sv2-tp/build_fuzz/src/test/fuzz/./test/fuzz/fuzz.cpp:247:5
#15 0x55b5ca2b9622 in LLVMFuzzerInitialize /src/sv2-tp/build_fuzz/src/test/fuzz/./test/fuzz/fuzz.cpp:321:5
#16 0x55b5ca21df42 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:652:5
#17 0x55b5ca24cfa2 in main /llvm-project/compiler-rt/lib/fuzzer/FuzzerMain.cpp:20:10
#18 0x7f0f188e4082 in __libc_start_main (/tmp/not-out/tmp_tv4p4q7/libc.so.6+0x24082) (BuildId: 5792732f783158c66fb4f3756458ca24e46e827d)
#19 0x55b5ca17802d in _start (/tmp/not-out/tmp_tv4p4q7/sv2_noise_cipher_roundtrip+0x24202d) (BuildId: 4d0ffc8495228d8cab1595a0fea7fa76c40aa18b)
DEDUP_TOKEN: basic_string--CBaseChainParams--std::__1::unique_ptr<CBaseChainParams, std::__1::default_delete<CBaseChainParams>> std::__1::make_unique[abi:de210103]<CBaseChainParams, char const (&) [8], int, int, 0>(char const (&) [8], int&&, int&&)
Uninitialized value was created by an allocation of 'ref.tmp' in the stack frame
#0 0x55b5ca32023c in std::__1::unique_ptr<CBaseChainParams, std::__1::default_delete<CBaseChainParams>> std::__1::make_unique[abi:de210103]<CBaseChainParams, char const (&) [8], int, int, 0>(char const (&) [8], int&&, int&&) /cxx_build/include/c++/v1/__memory/unique_ptr.h:759:34
DEDUP_TOKEN: std::__1::unique_ptr<CBaseChainParams, std::__1::default_delete<CBaseChainParams>> std::__1::make_unique[abi:de210103]<CBaseChainParams, char const (&) [8], int, int, 0>(char const (&) [8], int&&, int&&)
SUMMARY: MemorySanitizer: use-of-uninitialized-value /src/sv2-tp/build_fuzz/src/./chainparamsbase.h:29:55 in CBaseChainParams
Assisted-by: GitHub Copilot
Assisted-by: OpenAI GPT-5-Codex
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Mirrors bitcoin/bitcoin#32989 but I'm not sure if it's a good idea for this repo.