NeuroDev: Computational Dynamics of Learned Connectivity in Large-Scale Neural Architectures

Why do neural networks develop the connectivity they do, and what does this reveal about the computational principles underlying intelligence? This project investigates learned structural connectivity as a defining property of intelligent cognitive systems and as a guiding principle for efficient artificial neural network design through Hybrid Auto-Compressing Networks (H-ACNs): architectures that grow and reorganize their inter-layer connectivity during training, mirroring developmental processes in biological systems. 

The preliminary results demonstrate substantial efficiency gains when networks are allowed to learn their own connectivity, alongside striking cognitive signatures: vision tasks converge to local, columnar structures resembling visual cortex organization, while language tasks develop modular, hierarchical patterns analogous to frontal–temporal networks.

 Building on NeurIPS 2025 Oral paper introducing ACNs, the project looks into: do these signatures scale? Are they robust across random seeds or task-specific? And how does learned structural connectivity interact with dynamic routing in Mixture-of-Experts architectures? 

This project addresses these questions through large-scale pretraining at the 1B-parameter scale, a cross-modal cognitive task suite, and the first H-ACN-MoE models at 7B total parameters, aiming to uncover general principles linking learned connectivity, initialization, efficiency, and continual adaptation. Specifically, the team tests whether modular architectures that can grow, reuse, and reorganize their components in a task-dependent manner can match or surpass static dense models at significantly lower compute cost, and whether such adaptive functional connectivity is a prerequisite for scalable, long-horizon, continually learning foundation systems. 

By establishing structural connectivity as a unifying lens on both efficient AI and the computational principles of cognition, this project seeks to lay a principled foundation for the next generation of robust, modular, and continually adapting cognitively inspired large-scale models.