Audio morphing using matrix decomposition and optimal transport

Gerard Roma, Owen Green, Pierre Alexandre Tremblay
CeReNem, University of Huddersfield

This paper presents a system for morphing between audio recordings in a continuous parameter space. The proposed approach combines matrix decompositions used for audio source separation with displacement interpolation enabled by 1D optimal transport. Using NMF decompositions, the system allows varying the timbre of a sound in real time, while reproducing the temporal activations of its main components. Using HPSS as a pre-processing step, the system affords more detailed control of the interpolation in perceptually meaningful dimensions.

This page contains example code and audio for the proposed approach.

python code
max code

Example 1: Drum loop and bells

Algorithm Source λ = 0 λ = 0.25 λ = 0.5 λ = 0.75 λ = 0.95 λ = 1 Target
NMF-OT
HPSS-NMF-OT

Example 2: Polyphonic beat and guitar strums

Algorithm Source λ = 0 λ = 0.25 λ = 0.5 λ = 0.75 λ = 0.95 λ = 1 Target
NMF-OT
HPSS-NMF-OT

Example 3: Electric glitch and piano chord

Algorithm Source λ = 0 λ = 0.25 λ = 0.5 λ = 0.75 λ = 0.95 λ = 1 Target
NMF-OT
HPSS-NMF-OT

Example 4: Chaotic analog synth descent and digital legato chords

Algorithm Source λ = 0 λ = 0.25 λ = 0.5 λ = 0.75 λ = 0.95 λ = 1 Target
NMF-OT
HPSS-NMF-OT