Multilingual video dubbing—a technology review and current challenges

Abstract

Read online

The proliferation of multi-lingual content on today’s streaming services has created a need for automated multi-lingual dubbing tools. In this article, current state-of-the-art approaches are discussed with reference to recent works in automatic dubbing and the closely related field of talking head generation. A taxonomy of papers within both fields is presented, and the main challenges of both speech-driven automatic dubbing, and talking head generation are discussed and outlined, together with proposals for future research to tackle these issues.

Keywords

• talking head generation
• dubbing
• deep fakes
• deep learning
• artificial intelligence
• video synthesis