The first large LLM trained on Chinese chips

Meituan has released LongCat 2.0, which was trained on 50,000 unnamed Chinese chips, similar to Huawei Ascend 910C. The large pre-trained model has 1.6 trillion parameters (almost like DeepSeek V4 Pro), of which 48 billion are active. It was trained on 35 trillion tokens, with several hundred billion tokens being data with a context length of about one million tokens. Such scales were previously only achieved on Nvidia GPUs and Google TPUs, making this a significant achievement. An interesting feature of the model is that inactive parameters are used not only for MoE layers but also for huge n-gram embeddings, which take up almost 10% of all model parameters (in LongCat Flash-Lite, which they tested on, this took almost half of the parameters). By the way, they also parallelized the embeddings, resulting in 6D parallelism. And of course, they couldn't resist and created their own version of Sparse Attention by modifying DSA beyond recognition. For the last two months, LongCat 2.0 has been tested on Openrouter under the code name Owl Alpha, where it did not achieve stellar results. The model is priced at $0.75/$3 per million tokens in the API, which is quite expensive for such a level of intelligence (although its resource consumption is still unclear). The model weights are promised to be released soon, as Meituan usually releases under Apache 2.0/MIT.