add autoround._generate_recipe()

[xin3he]

layer_config, avg_bits = autoround._generate_recipe(
        # same data type config as before
        mp_dtype={
            "data_type": "mx_fp8",
            "act_data_type": "mx_fp8",
        },
        # special mix-precision configuration
        mp_config={
            "mp_ratio": 1/3,
            "loss_weight": 2.0,
            "numel_weight": 1.0,
        },
)
autoround.layer_config = layer_config
autoround.quantize()

This code is used for INC accuracy tuning, currently only mx_fp8 is supported for mixing with mx_fp4 and nv_fp4.

[wenhuach21]

Have you tested it on an MoE model? It might require some special handling.

[wenhuach21]

How about creating a new file in the experimental folder and moving these there?

[wenhuach21]

It would be better to wrap this new code into a function and call it as early as possible.

[wenhuach21]

This function cannot be used by AutoRound, since layers are converted to QuantizedLinear after quantization. If the function can correctly dump average bits in typical scenarios such as INT4, I’d prefer to keep it in the class. Otherwise, it would be better to move it elsewhere for now.