Vlogger is the latest AI tool that may generate videos. While there are plenty of tools around, Vlogger seems special. It can take a single input image of a person and generates a video out of it.
Apart from generating the video, Vlogger may also create audio. This works well in conjunction with an existing video file. Say, you have a video of someone talking in English. Vlogger can take that video, translate the video into another language, and manipulate the video so that the person’s lip activity and face areas are consistent with the translated content.
This technology allows video producers to create a single video and make it available to a worldwide audience in all supported languages.
Vlogger is short for “Video Blogger”, and a VLOG is a “Video Blog”.
How Vlogger works
Google notes on GitHub that Vlogger does not require specific training, or face detection and cropping. This would make the tool ideal for use on YouTube, but also for other purposes.
On YouTube, Google could offer the tool to publishers. They could use the AI to create videos in different languages using a single source video. Vlogger is also capable of changing the expressions of a person in the video. It could therefore also be used for video editing specific parts of a video, e.g., to close or open the mouth or eyes of a person.
The ultimate goal of the team behind Vlogger is to “generate a photorealistic video of variable length depicting a target human talking, including head and gestures”.
This sounds relatively static, and it seems to be the case at this stage. Not all video bloggers use portrait mode videos. Numerous creators walk around with camera in hand, and switch between different views.
Whether Vlogger will also be able to use its generative capabilities for these types of videos remains to be seen. Eventually, it might even be capable of synchronizing an entire movie that you feed it.
Closing Words
You can check out sample videos on GitHub. There you also find additional technical information about the AI.
Sounds cool, or scary? The next couple of years will certainly see many breakthrough technologies in the AI field, but also misuse of the technology. (via Günter Born)