AURA: Unified Multimodal Framework for
Conversational Music Editing
1 Aalto University · 2 Technical University of Denmark · 3 University of South Dakota · 4 OpenRB Lab
1 Aalto University · 2 Technical University of Denmark · 3 University of South Dakota · 4 OpenRB Lab
Adding an instrument while preserving the existing mix.
“Music piece. Instruct: Add bass.”
“Music piece. Instruct: Add guitar.”
“Music piece. Instruct: Add piano.”
“Add a distorted bass guitar to the mix to thicken the low end for this rock track.”
“add some subtle fingerpicking guitar to the singer songwriter track”
“add piano to the mix”
Removing a target instrument, keeping the rest intact.
“Music piece. Instruct: No bass.”
“Music piece. Instruct: No bass.”
“Music piece. Instruct: No piano.”
“remove the bass from the rock mix”
“Can you mute the bassline in this rap track?”
“please remove the piano from this pop track”
Isolating a single instrument from the mix.
“Music piece. Instruct: Only drums.”
“Music piece. Instruct: Only drums.”
“Keep only the drums and mute the guitar, bass, and vocals.”
“Keep only the guitar and mute everything else.”
Where AURA’s extraction breaks down — shown for transparency.
“Music piece. Instruct: Only piano.”
Extraction fails: the isolated stem is dominated by bleed from other instruments and/or loses the target timbre.
“Music piece. Instruct: Only guitar.”
Extraction fails: the isolated stem is dominated by bleed from other instruments and/or loses the target timbre.
“Music piece. Instruct: Only bass.”
Extraction fails: the isolated stem is dominated by bleed from other instruments and/or loses the target timbre.
“Keep only the guitar and mute everything else in this rock track.”
Extraction fails: the isolated stem is dominated by bleed from other instruments and/or loses the target timbre.
“Mute all instruments except the piano.”
Extraction fails: the isolated stem is dominated by bleed from other instruments and/or loses the target timbre.
“Isolate the guitar track and mute all other instruments.”
Extraction fails: the isolated stem is dominated by bleed from other instruments and/or loses the target timbre.
A dependent 3-turn conversation (remove drums → also bass → also guitar); each turn edits the previous result.
Original mix, then a 3-turn conversation:
Turn 1: “Remove the drums.”
Turn 2: “Now also remove the bass.”
Turn 3: “And remove the guitar as well.”
Original mix, then a 3-turn conversation:
Turn 1: “Remove the drums.”
Turn 2: “Now also remove the bass.”
Turn 3: “And remove the guitar as well.”
Original mix, then a 3-turn conversation:
Turn 1: “Remove the drums.”
Turn 2: “Now also remove the bass.”
Turn 3: “And remove the guitar as well.”
Original mix, then a 3-turn conversation:
Turn 1: “Remove the drums.”
Turn 2: “Now also remove the bass.”
Turn 3: “And remove the guitar as well.”
Original mix, then a 3-turn conversation:
Turn 1: “Remove the drums.”
Turn 2: “Now also remove the bass.”
Turn 3: “And remove the guitar as well.”
@article{trinh2026aura,
title = {AURA: Unified Multimodal Framework for Conversational Music Editing},
author = {Trinh, Quoc-Huy and Nguyen, Minh-Van and Jha, Debesh},
journal = {arXiv preprint arXiv:2609.14344},
year = {2026}
}