AI Tool Could Change Language in Films

A start-up company has a tool that it says can recreate lip sync - matching a speaker’s lip movements with pre-recorded audio - in films.

The artificial intelligence, or AI, tool may change how film producers replace recorded speech in a movie with speech recorded in another language. This process is known as dubbing.

Start-up Flawless AI, co-founded by film director Scott Mann, has a tool that it says can recreate lip sync without changing the performance of the actors.

The tool studies how actors move their mouths. It then changes the movements to sync with the replaced, or dubbed, words in different languages. The result could be that Tom Hanks looks like he can speak Japanese or Jack Nicholson looks like he can speak French.

"I hate dubbing as it stands," Mann told Reuters. He went on to describe the problems that dubbing can create. "You have to change so many things to try and catch sync. You're changing words filmmakers and performers have thought about so deeply. They're thrown out to find a different word that fits, but it never really does."

After some research, he discovered a paper written by Christian Theobalt of the Max Planck Institute for Informatics in Germany.

Leave a Comment