AI: the next frontier in dubbing?

Jacques Barreau, “Dean of Dubbing” y Vice President – Media & Interactive Entertainment de TransPerfect Media, reflects in this Tribune about artificial intelligence and its possible applications in the dubbing industry, which should not be negative.

IA: these 2 letters are starting to frighten to many of the people who are part of the dubbing community. Will machine learning and AI be a threat or support for voice-over professionals?

In this forum, we will talk about what AI is currently capable of achieving in terms of reproduction of voices and nuances, and we will see what this means not only for the voice actors, but for the entire dubbing industry.

What is AI?

Currently, the European Union is trying to define what AI is. Without going into too much detail, and for the sake of simplicity, we can divide the AI ​​applied to dubbing into 4 subgroups:

• Voice morphing, technology that transforms one voice into another voice.
• Text to speech, tools that allow you to convert written text into audio.
• Synthetic voices, which are used in audio description, audiobooks or e-learning programs.
• Facial transformation, a solution with which the original actor's mouth follows the sound of the translated dialogue.

Can some of these AI subgroups already be applied in dubbing? How? Will they replace the actors? What place will they occupy in the future in the dubbing industry?

Knowing that AI can already generate beautiful paintings imitating the style of different painters, we can imagine that it will also be able to generate dialogues in different languages ​​while maintaining the style of the most famous actors. Of course, a well-done dubbing with good actors will continue to provide a level of nuance in interpretation that AI cannot currently provide. Will it be just a matter of time?

Could AI favor the creation of a new group of super-actors?

The nuances in interpretation are infinite and the technology of text to speech is currently not capable of expressing all the ways in which the same phrase or even a single word.

The danger this implies is that we could witness a simplification of performance generated by AI if some companies rely entirely on AI to duplicate human emotions. Just as we witness a simplification of philosophical or political thought, we could well witness a simplification of the “art” of acting in dubbing due to the ability of AI to reproduce just a few basic emotions.

He voice morphing It could lead to fewer actors dubbing more characters, since their voices could be modified once their work is completed. Will this impact the dubbing community? Of course it does, but not necessarily because the talent pool is shrinking, since while all this is happening, demand is growing so fast that we are looking for new voice actors all over the world. Voice actors will need to increase their skills and be able to convey a greater variety of attitudes when dubbing. In the case of animation, the voice morphing could allowing them to focus more on acting, letting the AI ​​handle the very specific variety of voices that animated characters bring with them.

The resemblance to the original actors

The voice, like the image, is an integral part of an actor's image and using it without your consent is illegal. Some actors give up their voice to AI, just as some of us give up our bodies to science. But, for the actor, this could mean less work, since AI alone could generate content with his voice. Therefore, actors are only giving up the right to use their voices in exchange for a large amount of money.

Just like the algorithms can create new faces in a smartphone, are also emerging new voices… and that changes everything.

On the other hand, does the facial transformation Could this be a solution for dubbing? In this case, the original actor's face will change slightly as the mouth follows the dubbed dialogue. Will the original actors accept that their face moves slightly or changes expression depending on the language? Could it be considered a violation of your own identity? In this case, regulation will also be necessary. And, although dubbing rules are different in each country, the same cannot happen with AI, since it will cross borders.

Recently, algorithms morphing have become ready to create synthetic voices. Just as algorithms can create new faces on a smartphone, new voices are also emerging… and this changes everything. An actor could give their voice to the AI ​​and never hear it again anywhere, as it will be used to create a new synthetic voice. In this case, how can actors control the use of their image if we cannot recognize their voice? This is a different case of image use and a real challenge for legal departments.

Taking on a greater workload: the real challenge for actors

This is the real challenge of the current dubbing industry: the existence of a greater volume of content to dub in shorter deadlines. In this case, AI is not a threat to voice actors, but a necessity.

The IA is not a threat to voice actors, but rather a need.

In the very near future, a small group of good actors, and this number is not scalable, will be able to dub films that require a high level of interpretive nuance, while a hybrid process will use synthetic voices for minor roles or background settings. In reality, we already see this hybrid model in some areas such as translation, where machine translation does the first pass and humans do a second to refine it. Machine translation does not reduce the work of the translator, it only does part of it, which is why the need for translators will continue to grow around the world.

Towards new horizons

In the same way that the automatic translation has helped the translation business, AI could be a great learning tool for new groups of actors in countries that historically have not opted for dubbing. Although OTT platforms have started broadcasting in many emerging markets (we are dubbing in Kenya, Nigeria, Vietnam, etc.), where dubbing is not as developed as in traditional dubbing countries, the need to create groups of actors is essential .

In these situations, in order to satisfy the demand of an audience that wants to see more content dubbed in your language, AI can be a great accelerator to attract younger generations to the dubbing business and develop society's awareness about dubbing.

And now the scripts?

How to finish this tribune without mentioning the American screenwriters' strike. One of the complaints that accompanies this conflict is the unfair use of AI to generate scripts for talk shows, which is why they demand that AI not be used to generate new scripts from existing ones.

The IA can contribute some positive things to the dubbing process and the acting community will have to learn to work with her, not against her.

This is an important point, since AI only exists because it is fed by thousands of voices, audios or documents. Once again, we urgently need regulations because, in this case, if AI generates scripts by imitating screenwriters, who will take the credit?

Conclusion

Who knows what applications will be discovered using AI? It is unlikely that AI will disappear from our lives at this point and dubbing will be no exception.

We can't imagine what AI will be able to achieve, but one thing is for sure: AI can bring some positive things to the dubbing process and the acting community will have to learn to work with it, not against it.

Jacques Barreau

“Dean of Dubbing” y Vice President – Media & Interactive Entertainment de TransPerfect Media

Other articles on TransPerfect Media