Adaptive Block Forcing: Hybrid Diffusion and Autoregression at Semantic Level

Foundation models are increasingly used in scientific AI workflows, but their adoption is limited by the high computational cost, energy consumption, and low transparency of standard token-level autoregressive generation. Purely sequential decoding is slow and expensive on shared HPC infrastructure, while one-pass generation also makes model behaviour harder to interpret and control. This project proposes Adaptive Block Forcing (ABF), a hybrid generation framework that combines semantic-level autoregression with diffusion-based refinement within each semantic block. Instead of generating strictly token by token, ABF advances block by block, while iteratively inserting, deleting, and refining tokens inside each block until semantic completion is reached. This reduces sequential generation steps, improves GPU efficiency, and supports lower-cost, lower-carbon training and inference at scale. ABF is also designed for transparent and human-centred AI. By separating semantic progression from local refinement and making block completion explicit, the framework provides interpretable intermediate states, better controllability, and more auditable generation. These properties are important for reliable AI-enabled scientific workflows, where traceability and alignment with user intent are essential. Using EuroHPC GPU resources, we will develop and evaluate ABF through large-scale training and ablation studies, with a focus on speed–quality–energy trade-offs, controllability, and reproducibility. The project will deliver open research artifacts (code, models, and evaluation protocols) to support efficient, trustworthy, and sustainable foundation-model research in Europe.