Music Content Understanding and Creation of Music Visualisers

Conventional music visualisation systems rely on handcrafted ad hoc transformations of shapes and colours that offer only limited expressiveness. As a part of the ongoing, EU-funded project Mixed Reality Environment for Immersive Experience of Art and Culture (XTREME), this project aims to advance AI-powered music visualisation and explore the possibilities of generative AI models to handle sound as an input modality for video generation. The goal is to develop an open-source system that can produce video clips that fit any piece of music from an aesthetical standpoint.