Second Major Open Release of the Day - MiniMax M3

It turned out that the model has 428 billion parameters, with 23B active, quite small compared to competitors. The main innovation of the model is yet another variant of sparse attention, MSA (MiniMax Sparse Attention), which is noticeably more efficient than GQA in large contexts.