Retrieval-based Voice Conversion

Retrieval-based Voice ConversionDeveloper(s)| RVC-Project team

|

Initial release| 2024 (2024)
Repository| Github
Written in| Python
Operating system| Windows, Linux, macOS
Available in| English, Simplified Chinese, Japanese, Korean, French, Turkish, Portuguese
Type| Voice conversion software
License| MIT License

Retrieval-based Voice Conversion (RVC) is an open source voice conversion AI algorithm that enables realistic speech-to-speech transformations, accurately preserving the intonation and audio characteristics of the original speaker.[1]

## Overview[edit]

In contrast to text-to-speech systems such as ElevenLabs, RVC differs by providing speech-to-speech outputs instead. It maintains the modulation, timbre and vocal attributes of the original speaker, making it suitable for applications where emotional tone is crucial.

The algorithm enables both pre-processed and real-time voice conversion with low latency. This real-time capability marks a significant advancement over previous AI voice conversion technologies, such as So-vits SVC. Its speed and accuracy have led many to note that its generated voices sound near-indistinguishable from "real life", provided that sufficient computational specifications and resources (e.g., a powerful GPU and ample RAM) are available when running it locally and that a high-quality voice model is used. [2]

## Applications and concerns[edit]

The technology enables voice changing and mimicry, allowing users to create accurate models of others using only a negligible amount of minutes of clear audio samples. These voice models can be saved as .pth (PyTorch) files and shared on platforms like Weights.gg. While this capability facilitates numerous creative applications, it has also raised concerns about potential misuse as deepfake software for identity theft and malicious impersonation through voice calls.

### In pop culture[edit]

RVC inference has been used to create realistic depictions of song covers, such as replacing original vocals with characters like Twilight Sparkle and Mordecai to have them sing duets of popular music like "Airplanes" and "Somebody That I Used to Know." These AI-generated covers, which can sound strikingly similar to the voice imitated, have gained popularity on platforms like YouTube as humorous memes.[3]

## References[edit]

1. ^ Cochard, David (January 7, 2024). "RVC: An AI-Powered Voice Changer". Medium.
2. ^ "What's RVC". AI Hub. Retrieved 2024-05-27.
3. ^ "RVC WebUI How To – Make AI Song Covers in Minutes! (Voice Conversion Guide) - Tech Tactician". Tech Tactician. 2023-07-06. Retrieved 2024-05-27.

## External links[edit]

* Retrieval-based-Voice-Conversion-WebUI on GitHub

* Free software portal

* v
* t
* e

Speech synthesis

Free software|

| Speaking|

* eSpeak/eSpeakNG
* Gnopernicus
* Gnuspeech
* Orca
* Festival Speech Synthesis System/Flite
* FreeTTS
* Automatik Text Reader

|

Singing|

* eCantorix
* Lyricos / Flinger
* Sinsy

Proprietary
software|

| Speaking|

* Amazon Polly
* DECtalk
* Software Automatic Mouth
* Talk It!
* Microsoft Agent
* Microsoft Speech API
* Microsoft text-to-speech voices
* Readspeaker
* Voice browser
* CoolSpeech
* IVONA
* CereProc
* CeVIO Creative Studio
* Voiceroid
* LaLaVoice
* 15.ai

|

Singing|

* Alter/Ego
* Cantor
* CeVIO Creative Studio
* Chipspeech
* NIAONiao Virtual Singer
* PPG Phonem
* Realivox
* Symphonic Choirs
* UTAU
* Vocalina
* Vocaloid
* Xiaoice

Machine|

* Echo II
* Mockingboard
* Pattern playback
* Phasor
* RIAS
* Texas Instruments LPC Speech Chips
* General Instrument SP0256
* TuVox

Applications|

* AOLbyPhone
* DialogOS
* Dr. Sbaitso
* MBROLA
* Windows Narrator
* Microsoft Speech Server
* PlainTalk
* Voice font

Protocols|

* Speech Synthesis Markup Language
* SABLE
* VoiceXML

Developers/
Researchers|

* Alan W. Black
* Catherine Browman
* Franklin Seaney Cooper
* Gunnar Fant
* Haskins Laboratories
* Wolfgang von Kempelen
* Ignatius Mattingly
* Philip Rubin
* Yamaha

Process|

* Articulatory synthesis
* Concatenative synthesis
* Currah
* Inverse filter
* PSOLA
* Phase vocoder
* Self-voicing
* Voice cloning

*[v]: View this template
*[t]: Discuss this template
*[e]: Edit this template