Synthetic Media

AI Voice Actor

Replica is a company that using AI that can replicate human voice and produce natural-sounding text-to-speech. Their core technology takes a few minutes of speech and creates a Replica Voice which can be given a script to say anything. This AI voice model could be used in games, animation, and those low-budget documentaries instead of hiring voiceovers. For now, their model is compatible with Unreal Engine, Unity, iClone, Roblox and use as Replica API.

I consider this as synthetic media since it turns clips of the human voices as a base, and produces text-to-speech voices. The AI model learns how to perform by copying the real voice actors' unique speech patterns, pronunciation, and emotional range. And the end result is an AI voice actor that could be used in games or any audio projects.

For the data set they use to train the AL, I look up their term of use and found out that they use datasets from the LibriTTS corpus which is available at https://research.google/tools/ datasets/libri-tts/ .

The authors of the LibriTTS Datasets are Heiga Zen, Viet Dang, Rob Clark, Yu Zhang, Ron Weiss, Ye Jia, Zhifeng Chen and Yonghui Wu.

The LibriTTS Datasets are licensed pursuant to a Creative Commons Attribution 4.0 International licence, the terms of which which are available at https://creativecommons.org/licenses/by/4.0/legalcode

About the ethical ramifications, there is a lot of limitations in their Term of Service and Ethics and Security. But a thing I found mentioned in their term of service that I feel is worth a head up is:

You grant us the right to apply the Service to the text you submit and to create Synthesized Audio of it. We grant you a perpetual, worldwide royalty-free licence to use the Synthesized Audio for any purpose, including commercial purposes, subject to the terms set out in these Terms Of Use, including the restrictions in clause 1.3.

You grant us a, perpetual, worldwide, royalty-free right to store and use the text you provide:

(a) for our own internal research and development purposes;

(b) for our own internal purposes in improving the Service and our other products and services; and

(c) to monitor compliance with these Terms Of Use (which we may do using both automated and manual means).

There could a potential authorship rights between the users and people who developed these AI for voice generation.