Jointist Demo¶

This page contains the audio samples for the paper Jointist: Simultaneous Improvement of Multi-instrument Transcription and Music Source Separation via Joint Training.

Subjective evaluaiton: https://forms.gle/bXEazqjNwAgKfGch9

Source code: To be released upon acceptance

Table of contents¶

Jointist v.s. MT3
Jointist Transcription Demo
Jointist Source Separation Demo

Jointist v.s. MT3¶

The full audio and midi files are avaliable at: https://drive.google.com/file/d/1-F2wAALel9UUwMWZdHYZhQb3kIAfAQqL/view?usp=sharing

In general, Jointist is more robust to unseen musical instruments.

Audio Name	Mozart Symphony No.40	Beatles Let_It_Be	JayChou chaorenbuhuifei	MichaelJackson BlackOrWhite	Queen IWantToBreakFree	Radiohead Karma_Police	RWC RM-P083s
Audio
MT3
Jointist

Jointist Transcription Examples¶

Here, we test the music transcription feature of Jointist on real audio clips in a variaty of music genres. $f_{IR}$ determines the musical instruments appear in an audio clip, and then $f_{T}$ uses instrument condition to perform transcription.

A. Mozart Symphony No. 40 in G minor K. 550 (Classical Music)¶

Mozart Symphony

Input	Output

B. In Bloom - Nirvana (Rock)¶

Input	Output

C. 突然好想你 - Mayday (Chinese pop)¶

Input	Output

D. Psycho - Red Velvet (K-pop)¶

Input	Output

E. Lemon - Yonezu Kenshi (J-pop)¶

Input	Output

F. 夜に駆ける - YOASOBI (J-pop)¶

Input	Output

Back to TOC

Jointst Source Separation Examples¶

Here, we test the source separtion feature of Jointist on real audio clips in a variaty of genres. $f_{IR}$ determines the musical instruments appear in an audio clip, and then $f_{MSS}$ uses both the transcription output and the instrument condition to separate the sources. And hence, Jointist is able to deal with various number of musical instruments.

Since Slakh2100 has no vocal track (voice track in this dataset is simply a sound effect using voice), our Jointist is very weak in separating vocals. Most of the time, Jointist treats it as Synth.

We believe that with a better defined instrument taxonomy and more training dataset, the performance for Jointist would be even better.