SOTA 2-bit quants - part 2#4856

Merged

ggerganov merged 11 commits intomasterfrom

Jan 11, 2024

Commits on Jan 9, 2024

iq2_xs: basics
Iwan Kawrakow
committed
iq2_xs: this should have been in the basics
Iwan Kawrakow
committed
iq2_xs: CUDA and scalar CPU works
Iwan Kawrakow
committed
iq2_xs: WIP Metal
Iwan Kawrakow
committed
iq2_xs: Metal now works
Iwan Kawrakow
committed
iq2_xs: working, but dog slow, ARM_NEON dot product
Iwan Kawrakow
committed
iq2_xs: better ARM_NEON dot product
Iwan Kawrakow
committed

Commits on Jan 10, 2024

iq2_xs: AVX2 dot product - 19.5 t/s
Iwan Kawrakow
committed
iq2_xs: faster AVX2 dit product
Iwan Kawrakow
committed
iq2_xs: had forgotten to delete iq2-data.h
Iwan Kawrakow
committed

Commits on Jan 11, 2024

Add llama enum for IQ2_XS
Iwan Kawrakow
committed