MiniMax traces "Ma Jiaqi" bug to fine-tuning
MiniMax says its M2 models stopped generating the name "Ma Jiaqi" because rare tokens were forgotten during supervised fine-tuning, a failure mode that matters for anyone post-training on narrow data.
MiniMax has published an internal investigation into why its M2 series models could not output the name "Ma Jiaqi" (马嘉祺), a quirk that drew wide discussion on Xiaohongshu and Zhihu. The company says it has fixed the issue in later model updates.
The model still knew facts about Ma Jiaqi, including his group affiliation and debut date, but would not produce the token "嘉祺" (id 190467). MiniMax reproduced the behaviour with other low-frequency tokens, including "Wang Dan" (王郸).
The team first suspected that pretraining and serving used different tokenizers. It ruled that out. The token's embedding norm fell within the normal range, and its nearest neighbours were related name tokens such as "千玺" and "亚轩". In a few-shot test, the pretrained base model also generated the name without trouble, while the post-trained model avoided it.
That pointed to post-training. MiniMax found fewer than five samples containing "嘉祺" in its post-training data. It concluded that the model gradually "forgot" how to generate the token during supervised fine-tuning (SFT). Under top-p 0.95 sampling, a token whose probability falls this low can be cut out entirely.
The post goes beyond adding more data and looks at how the model's parameters changed. According to the published summary, it links drift in the lm_head output layer on sparse tokens to this bug and to a separate, long-running problem where Japanese responses slipped into Russian. MiniMax reports that its fixes cut that confusion rate from 47% to 1%. The excerpt available here does not include the full detail of those fixes.