Specify user-agent for livecheck

[MTCoster]

Output of brew config

HOMEBREW_VERSION: 4.4.11-27-g9783ab0
ORIGIN: https://github.com/Homebrew/brew
HEAD: 9783ab062cfcdc5ce776bfe5d163be3ee5fc7840
Last commit: 9 hours ago
Branch: master
Core tap HEAD: 426314a4b4ac29c169aee613e2d269567a74e8f5
Core tap last commit: 54 minutes ago
Core tap JSON: 12 Dec 16:23 UTC
Core cask tap HEAD: 66cd4ed05836d4cb6788e64c72f48c1a1b932940
Core cask tap last commit: 42 minutes ago
Core cask tap branch: adobe-dng-converter-user-agent
Core cask tap JSON: 12 Dec 16:23 UTC
HOMEBREW_PREFIX: /opt/homebrew
HOMEBREW_BAT: set
HOMEBREW_BAT_CONFIG_PATH: /Users/matt/.config/bat/config
HOMEBREW_CASK_OPTS: []
HOMEBREW_DEVELOPER: set
HOMEBREW_EDITOR: nvim
HOMEBREW_MAKE_JOBS: 12
HOMEBREW_NO_AUTO_UPDATE: set
HOMEBREW_NO_INSTALL_CLEANUP: set
HOMEBREW_NO_INSTALL_FROM_API: set
HOMEBREW_SORBET_RUNTIME: set
Homebrew Ruby: 3.3.6 => /opt/homebrew/Library/Homebrew/vendor/portable-ruby/3.3.6/bin/ruby
CPU: dodeca-core 64-bit arm_blizzard_avalanche
Clang: 16.0.0 build 1600
Git: 2.47.1 => /opt/homebrew/bin/git
Curl: 8.7.1 => /usr/bin/curl
macOS: 15.1.1-arm64
CLT: 16.1.0.0.1.1729049160
Xcode: 16.2
Rosetta 2: false

Output of brew doctor

Warning: You have unlinked kegs in your Cellar.
Leaving kegs unlinked can lead to build-trouble and cause formulae that depend on
those kegs to fail to run properly once built. Run `brew link` on these:
  node

Warning: You have uncommitted modifications to Homebrew/homebrew-cask.
If this is a surprise to you, then you should stash these modifications.
Stashing returns Homebrew to a pristine state but can be undone
should you later need to do so for some reason.
  cd /opt/homebrew/Library/Taps/homebrew/homebrew-cask && git stash -u && git clean -d -f

Uncommitted files:
   M audit_exceptions/simple_user_agent_for_homepage.json

Warning: Some taps are not on the default git origin branch and may not receive
updates. If this is a surprise to you, check out the default branch with:
  git -C $(brew --repo homebrew/cask) checkout master

Description of issue

I'm trying to bump adobe-dng-converter, but the audit fails because the livecheck and homepage URLs don't like custom user agents. They work fine with the curl/<version> default, but any non-browser custom string (homebrew or not) results in a closed connection.

The homepage URL is easy because the cask can just be added to audit_exceptions/simple_user_agent_for_homepage.json (tested; works).

The problem is livecheck - I can't find a way to specify a user-agent for it. Weirdly though, something in homebrew is able to access the URL because the audit fails with both "livecheck not reachable" and "version does not match livecheck" (see log below).

Output of brew audit --online --debug adobe-dng-converter:

/opt/homebrew/Library/Homebrew/brew.rb (Formulary::NullLoader): loading adobe-dng-converter
/opt/homebrew/Library/Homebrew/brew.rb (Cask::CaskLoader::FromNameLoader): loading adobe-dng-converter
/opt/homebrew/Library/Homebrew/brew.rb (Formulary::NullLoader): loading adobe-dng-converter
/opt/homebrew/Library/Homebrew/brew.rb (Cask::CaskLoader::FromNameLoader): loading adobe-dng-converter
/usr/bin/env XDG_CACHE_HOME=/Users/matt/Library/Caches/Homebrew/style /opt/homebrew/Library/Homebrew/vendor/portable-ruby/3.3.6/bin/ruby -W1 --disable=gems,rubyopt -- /opt/homebrew/Library/Homebrew/utils/rubocop.rb --format json --force-exclusion --parallel --except FormulaAuditStrict --config /opt/homebrew/Library/.rubocop.yml /opt/homebrew/Library/Taps/homebrew/homebrew-cask/Casks/a/adobe-dng-converter.rb
==> Auditing Cask adobe-dng-converter on os sequoia and arch arm
/opt/homebrew/Library/Homebrew/brew.rb (Cask::CaskLoader::FromPathLoader): loading /opt/homebrew/Library/Taps/homebrew/homebrew-cask/Casks/a/adobe-dng-converter.rb
==> Auditing Rosetta 2 requirement
==> Downloading and extracting artifacts
==> Downloading https://download.adobe.com/pub/adobe/dng/mac/DNGConverter_17_0.dmg
/usr/bin/env /opt/homebrew/Library/Homebrew/shims/shared/curl --disable --cookie /dev/null --globoff --show-error --user-agent Homebrew/4.4.11-27-g9783ab0\ \(Macintosh\;\ arm64\ Mac\ OS\ X\ 15.1.1\)\ curl/8.7.1 --header Accept-Language:\ en --retry 3 -V
/usr/bin/env /opt/homebrew/Library/Homebrew/shims/shared/curl --disable --cookie /dev/null --globoff --show-error --user-agent Homebrew/4.4.11-27-g9783ab0\ \(Macintosh\;\ arm64\ Mac\ OS\ X\ 15.1.1\)\ curl/8.7.1 --header Accept-Language:\ en --retry 3 --fail --location --silent --head https://download.adobe.com/pub/adobe/dng/mac/DNGConverter_17_0.dmg
/usr/bin/env /opt/homebrew/Library/Homebrew/shims/shared/curl --disable --cookie /dev/null --globoff --show-error --user-agent Homebrew/4.4.11-27-g9783ab0\ \(Macintosh\;\ arm64\ Mac\ OS\ X\ 15.1.1\)\ curl/8.7.1 --header Accept-Language:\ en --retry 3 --fail --location --silent --head --request GET --http1.1 https://download.adobe.com/pub/adobe/dng/mac/DNGConverter_17_0.dmg
Already downloaded: /Users/matt/Library/Caches/Homebrew/downloads/767c3c2dbda00bb210078d9bb32cc63edac79a33be998bc7ac5c2aa7d5446f03--DNGConverter_17_0.dmg
==> Checking quarantine support
/usr/bin/env /usr/bin/xattr -h
/usr/bin/env /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swift -target arm64-apple-macosx15 /opt/homebrew/Library/Homebrew/cask/utils/quarantine.swift
==> Quarantine is available.
==> Verifying Gatekeeper status of /Users/matt/Library/Caches/Homebrew/downloads/767c3c2dbda00bb210078d9bb32cc63edac79a33be998bc7ac5c2aa7d5446f03--DNGConverter_17_0.dmg
/usr/bin/env /usr/bin/xattr -p com.apple.quarantine /Users/matt/Library/Caches/Homebrew/downloads/767c3c2dbda00bb210078d9bb32cc63edac79a33be998bc7ac5c2aa7d5446f03--DNGConverter_17_0.dmg
==> /Users/matt/Library/Caches/Homebrew/downloads/767c3c2dbda00bb210078d9bb32cc63edac79a33be998bc7ac5c2aa7d5446f03--DNGConverter_17_0.dmg is quarantined
/usr/bin/env hdiutil imageinfo -format /Users/matt/Library/Caches/Homebrew/downloads/767c3c2dbda00bb210078d9bb32cc63edac79a33be998bc7ac5c2aa7d5446f03--DNGConverter_17_0.dmg
/usr/bin/env hdiutil attach -plist -nobrowse -readonly -mountrandom /private/tmp/homebrew-dmg20241212-42292-q184mq /Users/matt/Library/Caches/Homebrew/downloads/767c3c2dbda00bb210078d9bb32cc63edac79a33be998bc7ac5c2aa7d5446f03--DNGConverter_17_0.dmg
/usr/bin/env find . -print0
/usr/bin/env mkbom -s -i /private/tmp/20241212-42292-6hrsb0.list -- /private/tmp/20241212-42292-vjtama.bom
/usr/bin/env ditto --bom /private/tmp/20241212-42292-vjtama.bom -- /private/tmp/homebrew-dmg20241212-42292-q184mq/dmg.tFYIV1 /private/tmp/homebrew-unpack-20241212-42292-3gejhf
/usr/bin/env diskutil info -plist /private/tmp/homebrew-dmg20241212-42292-q184mq/dmg.tFYIV1
/usr/bin/env diskutil eject /private/tmp/homebrew-dmg20241212-42292-q184mq/dmg.tFYIV1
/usr/bin/env hdiutil imageinfo -format /private/tmp/homebrew-unpack-20241212-42292-3gejhf/DNGConverter_17_0.pkg
==> Auditing GitHub prerelease
/usr/bin/env /opt/homebrew/Library/Homebrew/shims/shared/curl --disable --cookie /dev/null --globoff --show-error --user-agent curl --header Accept-Language:\ en --connect-timeout 15 --max-time 25 --retry 3 --retry-max-time 25 --dump-header - --output /private/tmp/20241212-42292-7y6lsu --location https://helpx.adobe.com/camera-raw/using/adobe-dng-converter.html
/usr/bin/env /opt/homebrew/Library/Homebrew/shims/shared/curl --disable --cookie /dev/null --globoff --show-error --user-agent Homebrew/4.4.11-27-g9783ab0\ \(Macintosh\;\ arm64\ Mac\ OS\ X\ 15.1.1\)\ curl/8.7.1 --header Accept-Language:\ en --connect-timeout 15 --max-time 25 --retry 3 --retry-max-time 25 --dump-header - --output /private/tmp/20241212-42292-24b9qz --location https://download.adobe.com/pub/adobe/dng/mac/DNGConverter_17_0.dmg
/usr/bin/env /opt/homebrew/Library/Homebrew/shims/shared/curl --disable --cookie /dev/null --globoff --show-error --user-agent Homebrew/4.4.11-27-g9783ab0\ \(Macintosh\;\ arm64\ Mac\ OS\ X\ 15.1.1\)\ curl/8.7.1 --header Accept-Language:\ en --connect-timeout 15 --max-time 25 --retry 3 --retry-max-time 25 --dump-header - --output /private/tmp/20241212-42292-u5gzeq --location https://helpx.adobe.com/photoshop/kb/uptodate.html
/usr/bin/env /opt/homebrew/Library/Homebrew/shims/shared/curl --disable --cookie /dev/null --globoff --show-error --user-agent Mozilla/5.0\ \(Macintosh\;\ Intel\ Mac\ OS\ X\ 10_15_7\)\ AppleWebKit/605.1.15\ \(KHTML,\ like\ Gecko\)\ Version/17.0\ Safari/605.1.15 --header Accept-Language:\ en --connect-timeout 15 --max-time 25 --retry 3 --retry-max-time 25 --dump-header - --output /private/tmp/20241212-42292-66qu7s --location https://helpx.adobe.com/photoshop/kb/uptodate.html
==> Downloading https://download.adobe.com/pub/adobe/dng/mac/DNGConverter_17_0.dmg
Already downloaded: /Users/matt/Library/Caches/Homebrew/downloads/767c3c2dbda00bb210078d9bb32cc63edac79a33be998bc7ac5c2aa7d5446f03--DNGConverter_17_0.dmg
==> Verifying Gatekeeper status of /Users/matt/Library/Caches/Homebrew/downloads/767c3c2dbda00bb210078d9bb32cc63edac79a33be998bc7ac5c2aa7d5446f03--DNGConverter_17_0.dmg
/usr/bin/env /usr/bin/xattr -p com.apple.quarantine /Users/matt/Library/Caches/Homebrew/downloads/767c3c2dbda00bb210078d9bb32cc63edac79a33be998bc7ac5c2aa7d5446f03--DNGConverter_17_0.dmg
==> /Users/matt/Library/Caches/Homebrew/downloads/767c3c2dbda00bb210078d9bb32cc63edac79a33be998bc7ac5c2aa7d5446f03--DNGConverter_17_0.dmg is quarantined
==> Auditing pkg stanza: allow_untrusted
==> Auditing stanzas which require an uninstall
==> Auditing preflight and postflight stanzas
==> Auditing single uninstall_* and zap stanzas
==> Auditing required stanzas
==> Auditing version :latest does not appear as a string ('latest')
==> Auditing sha256 :no_check with version :latest
==> Auditing sha256 string is a legal SHA-256 digest
==> Auditing sha256 is not a known invalid value
/usr/bin/env /opt/homebrew/Library/Homebrew/shims/shared/curl --disable --cookie /dev/null --globoff --show-error --user-agent Homebrew/4.4.11-27-g9783ab0\ \(Macintosh\;\ arm64\ Mac\ OS\ X\ 15.1.1\)\ curl/8.7.1 --header Accept-Language:\ en --retry 3 -V
==> Auditing URL format
audit for adobe-dng-converter: failed
 - The livecheck URL https://helpx.adobe.com/photoshop/kb/uptodate.html is not reachable
 - Version '17.0' differs from '17.1' retrieved by livecheck.
 - Version '17.0' differs from '17.1' retrieved by livecheck.
adobe-dng-converter
  * The livecheck URL https://helpx.adobe.com/photoshop/kb/uptodate.html is not reachable
  * Version '17.0' differs from '17.1' retrieved by livecheck.
Error: 2 problems in 1 cask detected.

Relevant casks

adobe-dng-converter

[p-linnane]

@samford We still don't have a way to fake user agents for livecheck right?

[samford]

That's the goal but no, we can't specify a user agent in a livecheck block yet (I have an implementation finished but I have to get through some blockers before I can create a PR for all that work). However, page_content and page_headers contain some logic to retry requests with the :browser user agent if they fail with :default. That's just a temporary workaround until proper user agent support is added but it allows some checks like this to work for now (albeit, at the expense of efficiency).

For some brief context: the "livecheck URL...is not reachable" check is from audit_livecheck_https_availability, whereas the "Version X differs from Y retrieved by livecheck" output is from audit_livecheck_version. The former uses curl_check_http_content and the latter uses Livecheck::latest_version. Both involve similar user agent retry logic (though the implementations are different) but for some reason audit_livecheck_https_availability is failing and audit_livecheck_version isn't. It's interesting that jt-bridge (which also has a livecheck block that falls back to the :browser user agent) works but adobe-dng-converter fails.

I've run out of time today but I'll try to dig into this some more when I have a moment to figure out why this isn't working how I would expect.