CDHVAE

Paper Information

Kei Akuzawa, Kotaro Onishi, Keisuke Takiguchi, Kohki Mametani, Koichiro Mori. Conditional Deep Hierarchical Variational Autoencoder for Voice Conversion. APSIPA2021. Paper

Abstract

Variational autoencoder-based voice conversion (VAE-VC) has the advantage of requiring only pairs of speeches and speaker labels for training. Unlike the majority of the research in VAE-VC which focuses on utilizing auxiliary losses or discretizing latent variables, this paper investigates how an increasing model expressiveness has benefits and impacts on the VAE-VC. Specifically, we first analyze VAE-VC from a rate-distortion perspective, and point out that model expressiveness is significant for VAE-VC because rate and distortion reflect similarity and naturalness of converted speeches. Based on the analysis, we propose a novel VC method using a deep hierarchical VAE, which has high model expressiveness as well as having fast conversion speed thanks to its non-autoregressive decoder. Also, our analysis reveals another problem that similarity can be degraded when the latent variable of VAEs has redundant information. We address the problem by controlling the information contained in the latent variable using beta-VAE objective. In the experiment using VCTK corpus, the proposed method achieved mean opinion scores higher than 3.5 on both naturalness and similarity in inter-gender settings, which are higher than the scores of existing autoencoder-based VC methods.

Audio Demos

Below are a few demo audios. As described in our paper,

NOTE: You can compare these audios to those of other autoencoder-based methods in the demo page of AutoVC (Qian et. al. 2019).

Source Speaker / Speech Target Speaker / Speech Conversion
p270 (male) p256 (male) beta=1
beta=10
beta=50
p228 (female) beta=1
beta=10
beta=50
p225 (female) p256 (male) beta=1
beta=10
beta=50
p228 (female) beta=1
beta=10
beta=50

Other speakers and utterances:

Source Speaker / Speech Target Speaker / Speech Conversion
p226 (male) p259 (male) beta=1
beta=10
beta=50
p231 (female) beta=1
beta=10
beta=50
p225 (female) p254 (male) beta=1
beta=10
beta=50
p236 (female) beta=1
beta=10
beta=50