🐞 Cannot compress models while exporting to OpenVINO

[adrianboguszewski]

Describe the bug

As the API has changed in 2.0, compression like INT8_PTQ or INT8_ACQ no longer works. The problem is with the data delivered in the ImageBatch format.

To avoid that situation in the future, unit tests for different compression types should be implemented.

Dataset

N/A

Model

N/A

Steps to reproduce the behavior

from anomalib.deploy import ExportType, CompressionType
from anomalib.engine import Engine
from anomalib.models import EfficientAd
from anomalib.data import MVTecAD

datamodule = MVTecAD(
    category="carpet",  # MVTec category to use
    train_batch_size=1,  # Number of images per training batch
    eval_batch_size=1,  # Number of images per validation/test batch
)

model = EfficientAd()

engine = Engine(max_epochs=10)
engine.fit(datamodule=datamodule, model=model)

engine.export(model, export_root="ov_model", export_type=ExportType.OPENVINO, compression_type=CompressionType.INT8_PTQ, datamodule=datamodule)

OS information

OS information:

• OS: Ubuntu 24.04
• Python version: 3.12
• Anomalib version: main branch

Expected behavior

Model is exported and compressed.

Screenshots

No response

Pip/GitHub

GitHub

What version/branch did you use?

main

Configuration YAML

N/A

Logs

N/A

Code of Conduct

• I agree to follow this project's Code of Conduct

[github-actions]

Thanks for submitting this issue! It has been added to our triage queue. A maintainer will review it shortly.

[adrianboguszewski]

Fix for PTQ is rather easy. Use the following as the function:

    @staticmethod
    def _post_training_quantization_ov(
        model: "CompiledModel",
        datamodule: AnomalibDataModule | None = None,
    ) -> "CompiledModel":
        """Apply post-training quantization to OpenVINO model.

        Args:
            model (CompiledModel): OpenVINO model to quantize
            datamodule (AnomalibDataModule | None): DataModule for calibration.
                Must contain at least 300 images. Defaults to ``None``

        Returns:
            CompiledModel: Quantized OpenVINO model

        Raises:
            ValueError: If datamodule is not provided
        """
        import nncf

        if datamodule is None:
            msg = "Datamodule must be provided for OpenVINO INT8_PTQ compression"
            raise ValueError(msg)
        datamodule.setup("fit")

        model_input = model.input(0)
        # batch size must be the same as the input shape, but if the input shape is dynamic, use 1
        batch_size = model_input.shape[0] if model_input.partial_shape[0].is_static else 1

        dataloader = DataLoader(dataset=datamodule.val_data, shuffle=True, batch_size=batch_size, collate_fn=datamodule.val_data.collate_fn)
        if len(dataloader.dataset) < 300:
            logger.warning(
                f">300 images recommended for INT8 quantization, found only {len(dataloader.dataset)} images",
            )

        calibration_dataset = nncf.Dataset(dataloader, lambda batch: np.array([item.image for item in batch.items]))
        return nncf.quantize(model, calibration_dataset)

[adrianboguszewski]

ACQ requires more changes:

1. Deal with ImageBatch class
2. Deal with different types of provided metrics
3. Deal with different types of task type (classification vs. segmentation) - taking different output from the OV model