Skip to content

Update Habana Gaudi doc #3863

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jun 24, 2023
Merged

Update Habana Gaudi doc #3863

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
5337a6d
Update Habana Gaudi doc
regisss Jun 24, 2023
876a5d6
Fix typo
regisss Jun 24, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 8 additions & 8 deletions docs/source/en/optimization/habana.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,8 @@ specific language governing permissions and limitations under the License.

## Requirements

- Optimum Habana 1.5 or later, [here](https://huggingface.co/docs/optimum/habana/installation) is how to install it.
- SynapseAI 1.9.
- Optimum Habana 1.6 or later, [here](https://huggingface.co/docs/optimum/habana/installation) is how to install it.
- SynapseAI 1.10.


## Inference Pipeline
Expand All @@ -41,7 +41,7 @@ pipeline = GaudiStableDiffusionPipeline.from_pretrained(
scheduler=scheduler,
use_habana=True,
use_hpu_graphs=True,
gaudi_config="Habana/stable-diffusion",
gaudi_config="Habana/stable-diffusion-2",
)
```

Expand All @@ -62,18 +62,18 @@ For more information, check out Optimum Habana's [documentation](https://hugging

## Benchmark

Here are the latencies for Habana first-generation Gaudi and Gaudi2 with the [Habana/stable-diffusion](https://huggingface.co/Habana/stable-diffusion) Gaudi configuration (mixed precision bf16/fp32):
Here are the latencies for Habana first-generation Gaudi and Gaudi2 with the [Habana/stable-diffusion](https://huggingface.co/Habana/stable-diffusion) and [Habana/stable-diffusion-2](https://huggingface.co/Habana/stable-diffusion-2) Gaudi configurations (mixed precision bf16/fp32):

- [Stable Diffusion v1.5](https://huggingface.co/runwayml/stable-diffusion-v1-5) (512x512 resolution):

| | Latency (batch size = 1) | Throughput (batch size = 8) |
| ---------------------- |:------------------------:|:---------------------------:|
| first-generation Gaudi | 4.22s | 0.29 images/s |
| Gaudi2 | 1.70s | 0.925 images/s |
| first-generation Gaudi | 3.80s | 0.308 images/s |
| Gaudi2 | 1.33s | 1.081 images/s |

- [Stable Diffusion v2.1](https://huggingface.co/stabilityai/stable-diffusion-2-1) (768x768 resolution):

| | Latency (batch size = 1) | Throughput |
| ---------------------- |:------------------------:|:-------------------------------:|
| first-generation Gaudi | 23.3s | 0.045 images/s (batch size = 2) |
| Gaudi2 | 7.75s | 0.14 images/s (batch size = 5) |
| first-generation Gaudi | 10.2s | 0.108 images/s (batch size = 4) |
| Gaudi2 | 3.17s | 0.379 images/s (batch size = 8) |