NV12 support in libgbm

[jmondi]

Hello, opening an issue here to continue the discussion from
waydroid/android_hardware_waydroid#44

Let me recap the context:

I'm working on enabling camera support for Waydroid on a PinephonePro which use an RK3399 SoC.
The RK3399 camera subsystem produces images in NV12 format (only partially supports other formats, NV12 is strongly preferred).

As per Android design, the Camera HAL and the gralloc HAL implementation should agree on what format HAL_PIXEL_FORMAT_IMPLEMENTATION_DEFINED has to be mapped to (see the above mentioned merge request).

The gralloc_gbm backend does the allocation using libgbm, which is provided by mesa on Waydroid and which as of today doesn't support NV12.

I investigated with upstream #dri-devel last week about this, and what I've been told is that NV12 support is not in upstream mesa because nobody actually used that, and I've been reminded that the 'g' in 'gbm' stands for 'graphic' and not 'generic' so it is probably possible to add NV12 support to mesa, but it's not that anyone really cares as RGB formats are way more common.

So here's my question(s)

1. Is NV12/YCbCr use case supported by Waydroid ? I see this line in the gralloc implementation
   https://github.com/waydroid/android_hardware_waydroid/blob/lineage-18.1/gralloc/gralloc_gbm.cpp#L100
   which makes me think support for some kind of YUV format was needed and it has been implemented by mapping it on what's available on mesa today, with a few hacks here and there to have it working
   https://github.com/waydroid/android_hardware_waydroid/blob/lineage-18.1/gralloc/gralloc_gbm.cpp#L194

This commit actually puzzles me as well waydroid/android_hardware_waydroid@0876013
as I don't see any "mapping" there, as a value of "0" doesn't seem a valid GBM_FORMAT value.

2. ChromiumOS, which has several supported devices on RK3399, has adopted minigbm (which waydroid ships) as their gralloc_gbm backend implementation, which (afaiu) supports allocating NV12 buffers on the Rockchip using a driver-custom ioctl()
   https://github.com/waydroid/android_external_minigbm/blob/lineage-18.1/rockchip.c#L173
   Yes, it requires out-of-tree kernel support at the moment. The reasons why that ioctl is not in the rockchip upstream DRM driver are not fully clear to me yet.

In a case like this, where using minigbm seems more appropriate for the device at hand, would it be possible to instrument the Waydroid build to pick minigbm instead of mesa as libgbm provider ?

All in all, with the current gralloc+mesa implementation, camera support using NV12 format doesn't seem possible to me...

[rmader]

@fooishbar this is the issue regarding Waydroid and NV12 in Mesa we shortly talked about at FOSDEM - you said you might have something in mind?

[fooishbar]

@rmader thanks for the tag!

@jmondi you're right that it is indeed the Graphics Buffer Manager, not Generic. You can often use GBM as a proxy for gralloc, but not always. In particular, Mesa doesn't implement NV12 as a BO/surface type for alloc, because we don't (currently) have support for NV12 as a GPU render target. NV12 imports may work directly, or you may need to separately import each plane as R8/GR88 and do the CSC yourself.

I guess the three paths I can see are:

• just take the downstream kernel patches to add the Rockchip alloc ioctls, and use minigbm - least effort, worst result
• adapt the minigbm patches to use dma-heaps instead and use this - medium effort and still requires minigbm but does at least remove the dependency on downstream kernel uAPI
• allocate camera buffers via the camera rather than via gralloc - probably highest effort and highest divergence from Android but does at least match upstream