Skip to content

Conversation

@Chi-Chu319
Copy link

co-authors: @Chi-Chu319 @juuso-oskari

Added XCD remapping for flatmm moe

<style> </style>
batch Mixtral (tflops, wip_355) Mixtral-7B  (tflops, our branch) perf boost
64 865.424 995.455 15.0%
256 886.336 1020.96 15.2%
1024 890.808 1022.53 14.8%

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants