📝 Selected Publications

Advancing Block Diffusion Language Models for Test-Time Scaling

Yi Lu, Deyang Kong, Jianing Wang, Linsen Guo, Xue Wang, Qi Guo, Tao Gui, Xuanjing Huang, Wei Ye, Shikun Zhang, Wei Wang

[Paper] | [Model] | [Code] |

• Best long CoT block diffusion language models at 8B scale 🚀
• Proposes a “Think Coarse, Critic Fine” paradigm for fast exploration and fine-grained reflection in diffusion reasoning models 🧠

R-Horizon: How Far Can Your Large Reasoning Model Really Go in Breadth and Depth?

Yi Lu, Jianing Wang, Linsen Guo, Wei He, Hongyin Tang, Tao Gui, Xuanjing Huang, Xuezhi Cao, Wei Wang, Xunliang Cai

[Paper] | [Project] | [Code] | [Datasets] | [Models] |

• A benchmark for evaluating long-horizon reasoning 📏
• Proposes a long-horizon data synthesis pipeline and improves long-horizon reasoning with reinforcement learning 🔧

A Controlled Study on Long Context Extension and Generalization in LLMs

Yi Lu, Jing Nathan Yan, Songlin Yang, Justin T. Chiu, Siyu Ren, Fei Yuan, Wenting Zhao, Zhiyong Wu, Alexander M. Rush

[Paper] | [Video] | [Code] | [Models and Datasets] |

• Tutorial: Create Long-Context LLM Extension tutorial with Nathan and Sasha Rush [Video] 🎥
• The first controlled study that systematically evaluates long-context extension methods 🧪

Effective Length Extrapolation via Dimension-Wise Positional Embeddings Manipulation

Yi Lu, Wanxu Zhao, Xin Zhou, Chenxin An, Chenglong Wang, Shuo Li, Yuming Yang, Jun Zhao, Tao Ji, Tao Gui, Qi Zhang, Xuanjing Huang

[Paper] | [Code] |

• Achieves stronger extrapolation performance than YaRN without additional training ⚡
• The first training-free extrapolation method that modifies RoPE at the dimension level 🧩

LongHeads: Multi-Head Attention is Secretly a Long Context Processor

Yi Lu, Xin Zhou, Wei He, Jun Zhao, Tao Ji, Tao Gui, Qi Zhang, Xuanjing Huang

[Paper] | [Code] |

• The first block-selection method for multi-head attention 🌟
• Closely related to Kimi MoBA and DeepSeek NSA 🔗
• Cited by Kimi’s MoBA 📌