Warning "Unhandled error cleaning up build images"

[BenJackGill]

[REQUIRED] Environment info

firebase-functions: 4.8.0
firebase-tools: 13.5.1
Ubuntu: Latest

[REQUIRED] Test case

Unable to provide minimal example since this is happening in Github Actions CICD.

But Firebase Functions deploy always has this warning:

functions: Unhandled error cleaning up build images. This could result in a small monthly bill if not corrected. You can attempt to delete these images by redeploying or you can delete them manually at https://console.cloud.google.com/gcr/images/development-c46c5/us/gcf

[REQUIRED] Steps to reproduce

1. Create and deploy a Gen 2 Firebase Function in Github Actions CICD pipeline using ubuntu-latest.
2. See that some build images are not cleaned up.

Note: only some function images are there. I cannot figure out a rhyme or reason to which ones it will be. But there is always some that are not cleaned up.

The debug logs show that it is trying to cleanup us.gcr.io but it is not found. In artifact registry it shows gcf-artifacts so maybe it's trying to clean up the wrong thing?

Someone else also posted this very similar issue here on Stackoverflow.

[REQUIRED] Expected behavior

It should clean up the gcf-artifacts artifact images.

[REQUIRED] Actual behavior

It always has some images that are not cleaned up.

Related Logs:

[2024-03-15T08:52:54.398Z] <<< [apiv2][status] DELETE https://artifactregistry.googleapis.com/v1beta2/projects/development-c46c5/locations/us-central1/repositories/gcf-artifacts/packages/firestoreserpgridondocumentcreated 404
[2024-03-15T08:52:54.398Z] <<< [apiv2][body] DELETE https://artifactregistry.googleapis.com/v1beta2/projects/development-c46c5/locations/us-central1/repositories/gcf-artifacts/packages/firestoreserpgridondocumentcreated {"error":{"code":404,"message":"Package \"projects/development-c46c5/locations/us-central1/repositories/gcf-artifacts/packages/firestoreserpgridondocumentcreated\" was not found.","status":"NOT_FOUND"}}
[2024-03-15T08:52:54.462Z] <<< [apiv2][status] GET https://us.gcr.io/v2/development-c46c5/gcf/us-central1/tags/list 404
[2024-03-15T08:52:54.462Z] <<< [apiv2][body] GET https://us.gcr.io/v2/development-c46c5/gcf/us-central1/tags/list {"errors":[{"code":"NAME_UNKNOWN","message":"Repository \"us.gcr.io\" not found"}]}
[2024-03-15T08:52:54.463Z] Failed docker command with error  HTTP Error: 404, Not Found {"name":"FirebaseError","children":[],"context":{"body":{"errors":[{"code":"NAME_UNKNOWN","message":"Repository \"us.gcr.io\" not found"}],"error":{"message":"Not Found"}},"response":{"statusCode":404}},"exit":1,"message":"HTTP Error: 404, Not Found","status":404}
[2024-03-15T08:52:54.563Z] >>> [apiv2][query] GET https://us.gcr.io/v2/development-c46c5/gcf/us-central1/tags/list [none]
[2024-03-15T08:52:54.678Z] <<< [apiv2][status] GET https://us.gcr.io/v2/development-c46c5/gcf/us-central1/tags/list 404
[2024-03-15T08:52:54.678Z] <<< [apiv2][body] GET https://us.gcr.io/v2/development-c46c5/gcf/us-central1/tags/list {"errors":[{"code":"NAME_UNKNOWN","message":"Repository \"us.gcr.io\" not found"}]}
[2024-03-15T08:52:54.678Z] Failed docker command with error  HTTP Error: 404, Not Found {"name":"FirebaseError","children":[],"context":{"body":{"errors":[{"code":"NAME_UNKNOWN","message":"Repository \"us.gcr.io\" not found"}],"error":{"message":"Not Found"}},"response":{"statusCode":404}},"exit":1,"message":"HTTP Error: 404, Not Found","status":404}
⚠  functions: Unhandled error cleaning up build images. This could result in a small monthly bill if not corrected. You can attempt to delete these images by redeploying or you can delete them manually at https://console.cloud.google.com/gcr/images/development-c46c5/us/gcf 

[RaphaelMaschinsen]

I have the exact same issue when manually deploying with firebase deploy --only functions. Right now I have to manually cleanup gcf-artifacts.

[inlined]

I'm able to reproduce a bug where we're attempting to delete a cache image on function deletion, which GCF does automatically now.

I'm not able to reproduce failures in function creation or updates. If you're running into the issue still after next week's release of the CLI can you reopen the bug with the debug statement and the region of your function (it's possible our map of region -> artifact registry API URL has a bug somewhere).

[inlined]

Oh, and to address a comment from OP, we still talk to Google Container Repository currently to help clean up functions containers from users who had deployed back when we used GCR (well over a year ago), but a 404 from GCR is expected and does not emit a warning.

[TroniPM]

Well, i still having this exactly same problem.

node 22.2.0
npm 10.7.0
firebase-admin: 12.1.1
firebase-functions: 5.0.1
firebase-tools: 13.10.2
Platform: windows 11 23H2

My workaraound: with GCP CLI use this command to get the full path of gcf-artifacts at variable 'name': gcloud artifacts repositories list --format=json. This variable 'name' should be something like "projects/PROJECT_NAME/locations/us-central?/repositories/gcf-artifacts". Than delete the repository manually with command: gcloud artifacts repositories delete VARIABLE_NAME_HERE --async.

You can also create a command like firebase deploy --only functions & gcloud artifacts repositories delete VARIABLE_NAME_HERE --async to make it easier/faster. Just putting this here to help somebody in the future.

[youngjun-k]

I'm also having the same issue.

[Hibrix-net]

Same here,
I believe this should be corrected somehow from the package itself since it seems to be related with the recent deprecation of the Container Registry in favour of the Artifact Registry, but who knows...