When Swedish UFO film “Watch the Skies” hits U.S. cinemas this May, audiences won’t be able to tell that it wasn’t made in English.
The film is the first full theatrical release to showcase “visual dubbing” technology from AI firm Flawless, which enables actors’ performances to be digitally lip-synced with foreign-language dubs.
“Watch the Skies” follows rebellious teenager Denise, who teams up with a club of UFO watchers to solve the mystery of her missing father. The film was shot in Swedish, and released there under the title “UFO Sweden.” However, for its U.S. release, the original cast have re-recorded their performances in English, with Flawless using its TrueSync machine learning technology to digitally alter their lip movements so that they sync up with the English dialogue.
“Flawless and their technology gives us the opportunity to release the film for a much larger audience,” said writer-director Victor Danell in a making-of featurette.
The filmmakers stress that the use of the technique has “full endorsement from SAG,” the actors’ union, which went on strike in 2023 amid concerns over the “threat” posed by AI to the profession.
“A lot of filmmakers and a lot of actors will be afraid of this technology at first, but we have the creative control, and to act out the film in English was a real exciting experience,” said Danell, adding that, “It’s still our movie, it’s still the actors’ performance, and that’s the key part.”
What is TrueSync?
In a 2023 presentation, Flawless co-founder Scott Mann explained that the company’s TrueSync technology uses deep learning to create a volumetric 3D representation of the actors’ faces throughout a film, which can then be altered to match the dubbed dialogue.
TrueSync was previously used on the 2022 film “Fall” to remove swearing for a PG-13 edit, while another of the company’s products, DeepEditor, enables an actor’s performance to be extracted from one scene and applied to another scene, without the need for reshoots.
The company has partnered with distributor XYZ Films to release a slate of features localized using TrueSync, including “Run Lola Run” director Tom Tykwer’s upcoming “The Light,” horror feature “Vincent Must Die,” and South Korean film “Smugglers.”
Edited by Andrew Hayward
Generally Intelligent Newsletter
A weekly AI journey narrated by Gen, a generative AI model.
Source: https://decrypt.co/311114/swedish-film-watch-skies-ai-visual-dubbing
