Published October 21, 2026 | Version v1

Beyond Fixed Canvases: A Survey of Dynamic-Length, Fully Parallel, and Bidirectional Diffusion Language Models

  • 1. ROR icon University of Bergamo

Description

Diffusion large language models (dLLMs), particularly masked-diffusion models with full attention, generate text through iterative denoising over multiple positions at once, offering a structurally different alternative to the token-by-token, causal decoding of autoregressive (AR) language models. This parallelism is constrained by a basic feature of the standard sampler: the output canvas is fixed before ordinary content denoising begins, forcing a trade-off between oversized canvases, which waste computation on padding and can degrade output quality, and truncation of undersized generations. Block diffusion addresses this problem by generating sequentially dependent blocks, but thereby reintroduces a causal dependency structure across blocks, with reported costs to reasoning coherence and long-range context. This survey organizes length-adaptive methods compositionally by intervention point: external length prediction, masked-canvas probing, content-time canvas intervention, and native training-time length control. We then examine the relationship between length adaptation and sparse attention, KV-caching, step-skipping, speculative decoding, and discrete flow matching, and close with two sets of open problems for dynamic-length dLLMs and for fully parallel, bidirectional, and efficient dLLMs more broadly.

Files

main.pdf

Files (249.9 kB)

Name Size
md5:f18c48a7c6223e5398bf91e32dacc336
249.9 kB Preview Download