|
AI may be about to change all that.
Start-up Flawless AI, co-founded by film director Scott Mann,
has a tool that it says can accurately recreate lip sync in
dubbing without altering the performance of the actors.
The tool studies how actors move their mouths and swaps the
movements out according to the dubbed words in different
languages, making it seem like Tom Hanks can speak Japanese or
Jack Nicholson is fluent in French.
Mann was inspired to come up with the tool when he saw how
dubbing affected the narrative cohesion of his 2015 film
"Heist", starring Robert De Niro.
"I hate dubbing as it stands," he told Reuters. "You have to
change so many things to try and catch sync. You're changing
words filmmakers and performers have thought about so deeply.
They're thrown out to find a different word that fits, but it
never really does."
Mann decided to do something about this conspicuous mismatch.
After some research, he discovered a white paper by Christian
Theobalt of the Max Planck Institute for Informatics, laying out
a new approach that uses artificial intelligence to recreate
photo-real human visual effects.
The tool worked to translate a De Niro scene in "Heist".
"It's able to essentially take an 'ooh' sound from De Niro 20
minutes earlier and place that in a different moment in the
film," explained Mann. "It measures at the same time and blends
it so that the performance is the same but it's a different
mouth movement."
Flawless AI is now working with producers and studios to
integrate the technology into post-production. The first films
using this tool are around a year away.
Mann believes this technology will be a game-changer, helping to
eradicate foreign-language categories at award shows and
creating a more inclusive film industry with a real global
stage.
"From a filmmaking point of view, you're going to see the rise
of a much more diverse range of stars," he said.
There might be a dissonance and a focus on the AI technology in
the beginning when it is introduced, but eventually Mann hopes
its use will be seamless.
"When it's a common thing, we're not going to think about it
anymore and we'll just enjoy the movies," he said.
(Reporting by Rollo Ross; Writing by Karishma Singh; Editing by
Lisa Shumaker)
[© 2021 Thomson Reuters. All rights
reserved.] Copyright 2021 Reuters. All rights reserved. This material may not be published,
broadcast, rewritten or redistributed.
Thompson Reuters is solely responsible for this content
|
|
