GPU Benchmark Flux DEV fp8 5090 4090 3090

[metamountain]

Lets do a benchmark for flux dev fp8.

Just use the workflow template. Post the time of second or third render , no loading ....

[metamountain]

Nvidia 3090: 26s

[simonlui]

Intel A770 on Fedora Linux. Using Pytorch Nightly.

100%|███████████████████████████████████████████████████████████████████████| 20/20 [00:52<00:00,  2.61s/it]
Prompt executed in 53.26 seconds

If I go back to Pytorch Extension for Pytorch v2.3.110+xpu

100%|███████████████████████████████████████████████████████████████████████| 20/20 [00:46<00:00,  2.33s/it]
Prompt executed in 47.13 seconds

Edit: ComfyUI options used was --fp8_e4m3fn-unet --bf16-vae --bf16-text-enc --lowvram --async-offload --mmap-torch-files --reserve-vram 0.2 with --async-offload omitted for testing Pytorch v2.3.110+xpu.

[eric-rolph]

5090 8s

[eric-rolph]

5090 5.46s with some ComfyUI optimization, but left the workflow template unchanged

[CaptailTyler]

Just wondering what do you mean by optimizations did you do in ComfyUI? I didn't know that you can do any optimizations outside of the workflow? Thanks

[Rogala]

Total VRAM 16311 MB, total RAM 32599 MB
pytorch version: 2.8.0+cu128
xformers version: 0.0.32+635a8ab2.d20250804
Enabled fp16 accumulation.
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA GeForce RTX 5060 Ti : cudaMallocAsync
Using sage attention (2.2.0)
Python version: 3.12.10 (tags/v3.12.10:0cc8128, Apr 8 2025, 12:21:36) [MSC v.1943 64 bit (AMD64)]
ComfyUI version: 0.3.48
ComfyUI frontend version: 1.23.4

*.bat --use-sage-attention --fast

loaded completely 13400.370919891357 11350.067443847656 True
100%|██████████████████████████████████████████████| 20/20 [00:23<00:00, 1.20s/it]
Requested to load AutoencodingEngine
loaded completely 176.43157958984375 159.87335777282715 True
Prompt executed in 72.58 seconds
loaded completely 13240.49756211853 11350.067443847656 True
100%|██████████████████████████████████████████████| 20/20 [00:23<00:00, 1.20s/it]
Prompt executed in 25.71 seconds
got prompt
loaded completely 13240.49756211853 11350.067443847656 True
100%|██████████████████████████████████████████████| 20/20 [00:23<00:00, 1.20s/it]
Prompt executed in 25.89 seconds

[CaptailTyler]

Inno3D RTX 5090 X3 OC

Distributor ID: Ubuntu
Description: Ubuntu 25.04
Release: 25.04
Codename: plucky

Getting 8.78s at 2.38it/s for 3 runs.