babysor/MockingBird

What it is

MockingBird is a PyTorch-based voice cloning project forked from Real-Time-Voice-Cloning, which only supports English. It adds Mandarin support by reusing a pretrained encoder and vocoder with a newly trained synthesizer. It can be used through a web server, a desktop toolbox or a command-line script. The author states they no longer actively update the repo.

Who it's for

• Developers who want to experiment with Mandarin voice cloning and speech synthesis
• Researchers training their own synthesizer or vocoder on Chinese datasets such as aidatatang_200zh, magicdata, aishell3 and data_aishell
• Users who want to serve voice synthesis results via a web server for remote calling

Requirements

Setup

1. Install Python dependencies

   Install PyTorch and ffmpeg first, then install the remaining packages from requirements.txt. Install webrtcvad-wheels only if needed.

   bashCopy

   pip install -r requirements.txt
   pip install webrtcvad-wheels

2. Alternative: conda/mamba environment

   Create a virtual environment from env.yml, then activate it. env.yml covers only the necessary dependencies and does not include monotonic-align.

   bashCopy

   conda env create -n env_name -f env.yml
   conda activate env_name

3. Preprocess synthesizer dataset

   Download and unzip a dataset, then preprocess the audio and mel spectrograms. The --dataset parameter selects the dataset; the default is aidatatang_200zh.

   bashCopy

   python pre.py <datasets_root>

4. Train the synthesizer

   Train until the attention line appears and the loss meets your needs. Models are saved in synthesizer/saved_models/.

   bashCopy

   python train.py --type=synth mandarin <datasets_root>/SV2TTS/synthesizer

Examples

python web.py

python demo_toolbox.py -d <datasets_root>

python gen_voice.py <text_file.txt> your_wav_file.wav

python vocoder_preprocess.py <datasets_root> -m <synthesizer_model_path>
python vocoder_train.py mandarin <datasets_root> hifigan

Pros & cons

Images