AI: the next frontier in dubbing?

Jacques Barreau, “Dean of Dubbing” y Vice President – Media & Interactive Entertainment de TransPerfect Media, reflects in this Tribune about artificial intelligence and its possible applications in the dubbing industry, which should not be negative.

IA: these 2 letters are starting to scare to many of the people who are part of the dubbing community. Will machine learning and AI be a threat or support for voice-over professionals?

In this forum, we will talk about what AI is currently capable of achieving in terms of reproduction of voices and nuances, and we will see what this means not only for the voice actors, but for the entire dubbing industry.

What is AI?

Currently, the European Union is trying to define what AI is. Without going into too much detail, and for the sake of simplicity, we can divide the AI applied to dubbing into 4 subgroups:

Voice morphing, technology that transforms one voice into another voice.
Text to speech, tools that allow you to convert written text into audio.
Synthetic voices, which are used in audio description, audiobooks or e-learning programs.
Facial transformation, a solution with which the original actor's mouth follows the sound of the translated dialogue.

Can some of these AI subgroups already be applied in dubbing? How? Will they replace the actors? What place will they occupy in the future in the dubbing industry?

Knowing that AI can already generate beautiful paintings imitating the style of different painters, we can imagine that it will also be able to generate dialogues in different languages while maintaining the style of the most famous actors. Of course, a well-done dubbing with good actors will continue to provide a level of nuance in interpretation that AI cannot currently provide. Will it be just a matter of time?

Could AI favor the creation of a new group of super-actors?

The nuances in interpretation are infinite and the technology of text to speech is currently not capable of expressing all the ways in which the same phrase or even a single word.

The danger this implies is that we could witness a simplification of performance generated by AI if some companies rely entirely on AI to duplicate human emotions. Just as we witness a simplification of philosophical or political thought, we could well witness a simplification of the “art” of acting in dubbing due to the ability of AI to reproduce just a few basic emotions.

He voice morphing podría provocar que menos actores doblen a más personajes, ya que sus voces podrían modificarse una vez finalizado su trabajo. Will this impact the dubbing community? Of course it does, but not necessarily because the talent pool is shrinking, since while all this is happening, demand is growing so fast that we are looking for new voice actors all over the world. Los actores de doblaje tendrán que aumentar sus habilidades y ser capaces de transmitir una mayor variedad de actitudes al doblar. In the case of animation, the voice morphing could allowing them to focus more on acting, letting the AI handle the very specific variety of voices that animated characters bring with them.

The resemblance to the original actors

The voice, like the image, is an integral part of an actor's image and using it without your consent is illegal. Some actors give up their voice to AI, just as some of us give up our bodies to science. But, for the actor, this could mean less work, since AI alone could generate content with his voice. Therefore, actors are only giving up the right to use their voices in exchange for a large amount of money.

Just like the algorithms can create new faces in a smartphone, are also emerging new voices…and this changes everything.

On the other hand, does the facial transformation Could this be a solution for dubbing? En este caso, la cara del actor original cambiará ligeramente cuando la boca siga el diálogo doblado. ¿Aceptarán los actores originales que su cara se mueva ligeramente o cambie de expresión en función del idioma? ¿Podría considerarse una violación de su propia identidad? En este caso, también será necesaria una regulación. Y es que, si bien las normas de doblaje son diferentes en cada país, no puede ocurrir lo mismo con la IA, ya que traspasará fronteras.

Recently, algorithms morphing have become ready to create synthetic voices. Just as algorithms can create new faces on a smartphone, new voices are also emerging… and this changes everything. An actor could give their voice to the AI and never hear it again anywhere, as it will be used to create a new synthetic voice. In this case, how can actors control the use of their image if we cannot recognize their voice? This is a different case of image use and a real challenge for legal departments.

Taking on a greater workload: the real challenge for actors

This is the real challenge of the current dubbing industry: the existence of a greater volume of content to dub in shorter deadlines. In this case, AI is not a threat to voice actors, but a necessity.

The IA is not a threat to voice actors, but rather a need.

In the very near future, a small group of good actors, and this number is not scalable, will be able to dub films that require a high level of interpretive nuance, while a hybrid process will use synthetic voices for minor roles or background settings. In reality, we already see this hybrid model in some areas such as translation, where machine translation does the first pass and humans do a second to refine it. Machine translation does not reduce the work of the translator, it only does part of it, which is why the need for translators will continue to grow around the world.

Towards new horizons

In the same way that the machine translation has helped the translation business, AI could be a great learning tool for new groups of actors in countries that historically have not opted for dubbing. Although OTT platforms have started broadcasting in many emerging markets (we are dubbing in Kenya, Nigeria, Vietnam, etc.), where dubbing is not as developed as in traditional dubbing countries, the need to create groups of actors is essential.

In these situations, in order to satisfy the demand of an audience that wants to see more content dubbed in your language, AI can be a great accelerator to attract younger generations to the dubbing business and develop society's awareness about dubbing.

And now the scripts?

How to finish this tribune without mentioning the American screenwriters' strike. One of the complaints that accompanies this conflict is the unfair use of AI to generate scripts for talk shows, which is why they demand that AI not be used to generate new scripts from existing ones.

The IA can contribute some positive things to the dubbing process and the acting community will have to learn to work with her, not against her.

This is an important point, since AI only exists because it is fed by thousands of voices, audios or documents. Once again, we urgently need regulations because, in this case, if AI generates scripts by imitating screenwriters, who will take the credit?

Conclusion

Who knows what applications will be discovered using AI? It is unlikely that AI will disappear from our lives at this point and dubbing will be no exception.

We can't imagine what AI will be able to achieve, but one thing is for sure: AI can bring some positive things to the dubbing process and the acting community will have to learn to work with it, not against it.