Optimize pixel blending with integer arithmetics

[antonfirsov]

Problem

The current float and Vector4 -based pixel blender API does not give us too much space for introducing the rasterization perf improvements needed for SixLabors/ImageSharp.Drawing#102. With our current bulk API, the maximum we can do is to process 2 pixels in one AVX batch, since we can fit only 8 float-s into one AVX register. This means that the expected speedup for blending is around or below 2x. With this we would keep lagging behind Skia and GDI significantly.

Idea

We should explore API-s and implementations working with UInt16-based fixed point arithmetics. This is technically very similar to approach taken by the libjpeg decoder SIMD pipelines which we eventually also want to adapt. In theory, UInt16-based bulk processing should reduce the time spent in pixel blenders by ~4x (or more) when AVX2 is present.

This will require API additions similar to the following:

public abstract class PixelBlender<TPixel> 
{
    public void Blend<TPixelSrc>(
            Configuration configuration,
            Span<TPixel> destination,
            ReadOnlySpan<TPixel> background,
            ReadOnlySpan<TPixelSrc> source,
            // 'amount' is scaled to 0-255. Could be byte, but with UInt16 we will avoid some unnecessary conversions
            ReadOnlySpan<UInt16> amount); 

    protected virtual BlendFunction(
            Configuration configuration,
            Span<Rgba32> destination,
            ReadOnlySpan<Rgba32> background,
            ReadOnlySpan<Rgba32> source,
            ReadOnlySpan<UInt16> amount);
}


public static class PorterDuffFunctions
{
    /*public*/ static Vector4 NormalSrcOver(Span<Rgba32> destination,
            ReadOnlySpan<Rgba32> background,
            ReadOnlySpan<Rgba32> source,
            ReadOnlySpan<UInt16> opacity);
}

Update:
In the first API variant there was a type ScaledUInt16Vector4, but after thinking it through, I realized it is unnecessary. We should work with Rgba32 to maximize perf.

[JimBobSquarePants]

Agreed. This seems to be the smartest approach while keeping work on the CPU. Other libraries defer to the GPU for perf

[JimBobSquarePants]

Pinning this as It's super important for ImageSharp.Drawing

[JimBobSquarePants]

@br3aker I know it's not jpeg but is this something you'd be interested in? Seems like something you could churn out whereas it would take me ages.

[antonfirsov]

This is important in long term, but improving Jpeg decoder performance has higher priority at the moment IMO, so if @br3aker is more interested in Jpeg he's doing the right thing :)

We need to close the 40% gap to vips and MagicScaler in thumbnail making. (See #1775).

[JimBobSquarePants]

Yep. Let's get that gap closed! We still need someone to look at this so hoping someone from the community can step up while I focus on the Fonts/Drawing libraries themselves.

[br3aker]

Oops, totally missed this conversation, I'm sorry.

We can work something out after jpeg decoder :)