Photo credit: YouTube
YouTube is testing a new AI-powered tool to make syncing videos in non-English languages even easier — and it’s free.
YouTube has partnered with synchronization service Aloud, which in turn is part of Google’s in-house incubator Area 120. With the addition of multilingual audio tracks to YouTube videos earlier this year, an AI-powered syncing solution makes sense. Previously, YouTube creators had to work with a third-party sync service to provide audio tracks for different languages. But syncing with Aloud allows YouTubers to expand into multilingual markets at no additional cost.
The service works by providing a transcription for the creator before translation. Creators can review and edit this transcript before creating the sync, saving a lot of unnecessary work. YouTube is testing the tool with hundreds of creators, including the popular ones Mr. Beast. With enough dubbing hours under its belt, YouTube will make the feature available to all YouTubers. Aloud is currently available in English, Spanish and Portuguese.
Amjad Hanif, YouTube’s VP of Creator Products, says the company is working on technology to make the dub sound like the YouTuber’s voice “with more expression and lip sync.” The generative AI in Aloud would give creators access to better emotional transmission and lip reanimation to give video announcers the impression of speaking the dubbing natively.
“In the past, synchronization required weeks of effort and a large budget. But with ALoud, you only need a few minutes,” reads a blog post announcing the new technology. “We leverage advances in audio separation, machine translation, and speech synthesis to reduce time-consuming and costly steps like translation, video editing, and audio production. You don’t even have to know a different language than the one you already speak.”
“We want to ensure that synthetically dubbed content is never confused with the original or with content dubbed by voice actors. To ensure transparency, we require our creators to clearly state any dubs produced by Aloud in the video description or in the video Post Credits Screen that these are synthetic dubs compared to the original.”