[1909.01218] Translating Visual Art into Music

The Synesthetic Variational Autoencoder (SynVAE) introduced in this research is able to learn a consistent mapping between visual and auditive sensory modalities in the absence of paired datasets. A quantitative evaluation on MNIST as well as the Behance Artistic Media dataset (BAM) shows that SynVAE is capable of retaining sufficient information content during the translation while maintaining cross-modal latent space consistency. In a qualitative evaluation trial, human evaluators were furthermore able to match musical samples with the images which generated them with accuracies of up to 73%.

1 mentions: @hardmaru
Date: 2019/09/10 11:17

Referring Tweets

@hardmaru Translating Visual Art into Music Synesthetic Variational Autoencoder purposed to learn a mapping between visual and auditive sensory modalities without paired datasets. Synesthesia is a condition in which hearing is simultaneously perceived as sight. t.co/EYU2C5enJq t.co/MztcbHifUa

Related Entries

Read more Deep Learningを用いた教師なし画像検査の論文調査 GAN/SVM/Autoencoderとか .pdf
0 users, 0 mentions 2018/10/09 06:53
Read more Variational Autoencoder in Tensorflow - facial expression low dimensional embedding - Machine learni...
0 users, 0 mentions 2018/04/22 03:40
Read more 時系列データでVariational AutoEncoder keras - 機械学習を学習する天然ニューラルネットワーク
0 users, 0 mentions 2018/09/23 06:23
Read more 猫でも分かるVariational AutoEncoder
0 users, 0 mentions 2018/06/15 04:30
Read more Intuitively Understanding Variational Autoencoders – Towards Data Science
0 users, 0 mentions 2018/07/08 12:23