Evidence at a glance
A Checkpoint for Further Training
ERNIE 5.5 Flash Base is Baidu’s text-only sparse Mixture-of-Experts (MoE) model. It has 280B total parameters and activates 14B per token. Baidu released it as a checkpoint at the end of pre-training, with no mid- or post-training. Teams can use that shared starting point for domain adaptation, instruction tuning, or reinforcement learning; the checkpoint is not an instruction-aligned model ready for chat.
The model has 43 layers, a hidden size of 4096, and 64 attention heads. Baidu says it was pre-trained on 33.5T tokens using the Muon optimizer and FP8 mixed precision, with a 128K training context. The released weights come in FP8 and BF16 versions. Baidu lists support for up to 1M tokens in fine-tuning and inference; that is separate from the 128K pre-training context.
Hybrid Attention and Multi-Router MoE
Of the model’s 43 layers, 37 use Heavily Compressed Attention (HCA), which compresses every 128 tokens into one cache entry. These layers use 512-dimensional heads. The remaining 6 use Multi-head Latent Attention with sparse attention (MLA + DSA), with 256-dimensional heads. Baidu describes the combination as a way to manage KV-cache and compute costs at long context lengths; the MLA + DSA layers retain token-level context access.
Each MoE layer has 512 routed experts, selects the top 10 for each token, and includes 1 shared expert. Instead of a single router, the model uses several independent sigmoid routers. The experts operate in a 2048-dimensional latent space, half the hidden size. Baidu says expert FLOPs and communication volume remain unchanged.
ECHO, Inference, and the Training Kernel
ECHO reuses the backbone’s final layer to predict future tokens, with that same layer also serving as a draft model for speculative decoding. It adds 0.36B parameters, which are excluded from the headline parameter count. The available materials do not give a measured speedup for ECHO against other prediction methods.
Baidu documents deployment with SGLang and vLLM. Its SGLang example uses `--tp 4`; the vLLM path requires vLLM 0.32.0 and the `vLLM-ERNIE` plugin. As a base model, it is called through the completions endpoint rather than a chat endpoint. The weights are available on [Hugging Face](https://huggingface.co/baidu/ERNIE-5.5-Flash-Base).
Training used FlashMask-4, a kernel for SM90 and SM100 GPUs. Baidu reports that on SM100 it delivers up to 175% faster forward passes and 74% faster backward passes than FlashAttention-4 MaskMod. These figures are reported by Baidu.
Benchmark Results and License Terms
In Baidu’s unified evaluation pipeline, ERNIE 5.5 Flash Base scored best on 24 of 30 benchmarks. Baidu reports an MMLU-Pro score of 78.4, compared with 69.2 for DeepSeek-V4-Flash-Base in the same pipeline. Its other reported scores include 82.3 on HumanEval+, 81.5 on MBPP+, 91.7 on RepoQA, and 48.7 on LongBench-v2. It does not lead every task: on HELMET-Rerank it scored 56.2, below DeepSeek-V4-Flash-Base’s 60.0. Baidu reports these results using the same prompts and few-shot settings for the models; the available materials do not mention independent replication or a third-party audit.
The license also restricts use. The source says use is free for entities with revenue below US$50M. The terms exclude the EU, the UK, and South Korea, ban military use, and prohibit using the model’s outputs to improve other AI models. Hong Kong law governs the license. Downloadable weights do not mean the license is unrestricted.
Further information: [Baidu’s technical introduction](https://ernie.baidu.com/release/ernie-5-5-flash-base/), [Baidu’s model card](https://huggingface.co/baidu/ERNIE-5.5-Flash-Base-BF16), and the [original report](https://www.marktechpost.com/2026/11/05/baidu-ernie-5-5-flash-base-280b-moe/).