YouTube launches AI lip sync feature for users

YouTube is making significant advancements in its AI-powered video tools. After launching AI dubbing in December and expanding Multi-Language Audio tracks last month, the platform is now testing a new feature: AI lip sync for videos in different languages.

YouTube demonstrated this feature with a short 15-second video. It showcased an Italian clip that was automatically translated into English, with the speaker’s lip movements perfectly synchronized with the dubbed voice.

The caption read, “With the upcoming lip sync technology, translated videos will visually match the speaker’s lips to the newly dubbed language, making your content more accessible and engaging for global audiences.”

According to Buddhika Kottahachchi, YouTube’s Head of Product for Dubbing, the platform uses a custom technology stack that alters the pixels on the screen to align with the translated speech. This means that YouTube’s AI system adjusts the visuals of the speaker’s mouth to match the dubbed audio, creating a smoother and more realistic experience for viewers.

Currently, the lip sync feature is being tested with a small group of creators. Kottahachchi mentioned that the company is still studying how the system performs before rolling it out more widely.

“We are not ready to make any broad statements about availability,” he said, “but we do want to make it accessible to more creators while understanding the compute constraints and quality,” he added.

The pilot program supports five languages: English, Spanish, German, Portuguese, and French. YouTube plans to eventually expand this program to include all the languages supported by its autodubbing feature.

Read More: China’s Generative AI users surge to 515 million in six months

This initiative responds to the growing demand for AI-driven lip sync technology. Modern AI tools can deliver studio-quality results by syncing lip movements with translated voices while accurately capturing details such as facial hair, tooth movements, and high-resolution visuals.

With this innovation, YouTube aims to make it easier for creators to reach global audiences without language barriers. Viewers will be able to enjoy videos that look and sound more natural, no matter what language they speak.

If the newly introduced initiative is fully launched, YouTube’s automated lip sync could transform how creators share content worldwide, bringing them one step closer to a truly universal video experience.