ReferenceGrant API version bump has caused Gateway API Conformance to fail on main

[sunjayBhatia]

See: kubernetes-sigs/gateway-api#1455

Failing daily build: https://github.com/projectcontour/contour/actions/runs/3410157745

Related:

• [dev-gwapi-0.6] drop support for ReferencePolicy #4830
• Gateway API: ReferencePolicy -> ReferenceGrant rename #4555

We could install Gateway API CRDs here using the GATEWAY_API_VERSION variable we use in the Makefile/workflows, or here before where we actually run the tests

In addition we need to move to v1beta1 RBAC etc. in Contour

[sunjayBhatia]

Discussed this just now and we're gonna drop the main conformance runs to focus on running conformance against main of GW API to a long-running branch, hoping to track the 0.6.0 release of GW API