We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
A fast inference library for running LLMs locally on modern consumer-class GPUs
Python 4.3k 318
Web UI for ExLlamaV2
JavaScript 508 48
An optimized quantization and inference library for running LLMs locally on modern consumer-class GPUs
Python 484 40
Loading…
There was an error while loading. Please reload this page.