back Back

StyleTalker takes one photo of you and produces a video of you talking

Mar. 18, 2024.
1 min. read 14 Interactions

Korean researchers recently presented a new AI model that takes one photograph as input, and outputs a video of somebody saying any arbitrary speech, with realistic lip-movement

About the writer

conor o'higgins

67.73054 MPXR

Conor O'Higgins is the editor of Mindplex Magazine

Scientists from KAIST, the South Korean AI Institute, have described a new model called StyleTalker, which takes a single image of a person as an input, and produces a video of them talking “with accurately audio-synced lip shapes, realistic head poses, and eye blinks”.

StyleTalker combines AI techniques for “audio-driven generation” (generating realistic lip-movements from audio) with “motion-controllable” generation, that can do things like take the head-movements and gestures from one video, and use them in a new video with a new face.

The work builds on a recent boom in neural lip-synced video generation, a research-field that aims to “transforming the lip region of the person in the target video,generating new videos with the lip shapes that match the input audio.” (This could be used, for example, when movies are dubbed from one language into another.)

StyleTalker “can generate more natural and robust talking head videos compared to other models” previously described. It is a step towards more realistic fake videos, but is that a good thing? Let us know in the comments how you think this technology could be used for good and for bad.

Citation: Dongchan Min, Minyoung Song, Eunji Ko, Sung Ju Hwang. StyleTalker: One-shot Style-based Audio-driven Talking Head Video Generation. eprint (2024). https://arxiv.org/abs/2208.10922 (open access)

Let us know your thoughts! Sign up for a Mindplex account now, join our Telegram, or follow us on Twitter

Comment on this content

4 Comments

4 thoughts on “StyleTalker takes one photo of you and produces a video of you talking

  1. beautiful drawing

    Like
    Dislike
    Share
    Reply
  2. Learned a lot and the drawing is also beautiful thanks

    Like
    Dislike
    Share
    Reply
  3. good info👍🏻

    Like
    Dislike
    Share
    Reply
  4. good one

    1 Like
    Dislike
    Share
    Reply

7

Like

Dislike

1

Share

4

Comments
Reactions
💯 💘 😍 🎉 👏
🟨 😴 😡 🤮 💩

Here is where you pick your favorite article of the month. An article that collected the highest number of picks is dubbed "People's Choice". Our editors have their pick, and so do you. Read some of our other articles before you decide and click this button; you can only select one article every month.

People's Choice
Bookmarks