Introduction of CUDA Graphs to LLama.cpp#6766

Merged

slaren merged 20 commits intoggml-org:masterfrom

agray3:ag_cuda_graphs

May 8, 2024

Commits on Apr 19, 2024

DRAFT: Introduction of CUDA Graphs to LLama.cpp
agray3
committed

Commits on Apr 22, 2024

Commits on Apr 23, 2024

Disable CUDA graphs for old GPU arch and with env var
agray3
committed

Commits on Apr 24, 2024

Commits on Apr 25, 2024

limit to GGML_ALLOW_CUDA_GRAPHS defined in llama.cpp cmake
agray3
committed

Commits on Apr 29, 2024

Merge branch 'ggerganov:master' into ag_cuda_graphs
agray3
authored

Commits on Apr 30, 2024

Commits on May 1, 2024

Fall back if graph capture fails and address other comments
agray3
committed

Commits on May 2, 2024

Merge branch 'ggerganov:master' into ag_cuda_graphs
agray3
authored

Commits on May 7, 2024

Commits on May 8, 2024