Researchers Alex M Tseng and Luca Zancato published HOPE, short for Higher-Order Pruning of Experts, in a Hugging Face article dated November 9, 2026. The method targets Mixture-of-Experts language models, where expert networks take up most of the memory footprint. Unlike earlier approaches that score each expert on its own, HOPE accounts for pairwise interactions between experts, which the authors describe as cooperative groups that are often activated together.
At 50% pruning, HOPE reached an average rank of 1.58, against 2.42 for REAP, the next-best method. The authors evaluated it on 3 models, Qwen3.5-122B, Qwen3.5-35B and GLM-4.5-Air, and compared it with 4 first-order baselines. On agentic coding tasks the gains reached up to +6.1% over the next-best pruning method. On Qwen3.5-35B-A3B at 50% pruning, HOPE scored 42.3 on SWE-Bench Pro, matching the unpruned model's 41.8.
HOPE runs in three steps: calibrate, solve and prune, through a basic package the authors created. The calibration pass is the same one used by other expert-pruning methods, and solving the quadratic program takes 1-2 seconds each layer. The package has been tested on Qwen3, Qwen3-Next, Qwen3.5 and GLM-4.5. The advantage shrinks at lower pruning rates of 10-20%, where REAP and MAN are sometimes competitive.
Key facts
- HOPE achieves an average rank of 1.58 at 50% pruning, compared with 2.42 for REAP.
- Gains over the next-best pruning method reach up to +6.1% on agentic coding tasks.
- On SWE-Bench Pro, the pruned Qwen3.5-35B-A3B model at 50% pruning scored 42.3, against 36.2 for REAP.
- Solving the quadratic program takes 1-2 seconds each layer, on top of a standard calibration pass.
- The package was tested on Qwen3, Qwen3-Next, Qwen3.5 and GLM-4.5 models.
Why it matters
The pruned output is a standard HuggingFace checkpoint, smaller on disk and in memory, and needs no changes to vLLM or SGLang model definitions. The authors call 50% pruning the most deployment-relevant setting, where memory savings can accommodate longer contexts, deeper reasoning and more tool calls. They also say pruning avoids the latency issues that make dynamic loading of experts impractical.




