[2102.03141] CharacterGAN: Few-Shot Keypoint Character Animation and Reposing

We introduce CharacterGAN, a generative model that can be trained on only a few samples (8 - 15) of a given character. Our model generates novel poses based on keypoint locations, which can be modified in real time while providing interactive feedback, allowing for intuitive reposing and animation. Since we only have very limited training samples, one of the key challenges lies in how to address (dis)occlusions, e.g. when a hand moves behind or in front of a body. To address this, we introduce a novel layering approach which explicitly splits the input keypoints into different layers which are processed independently. These layers represent different parts of the character and provide a strong implicit bias that helps to obtain realistic results even with strong (dis)occlusions. To combine the features of individual layers we use an adaptive scaling approach conditioned on all keypoints. Finally, we introduce a mask connectivity constraint to reduce distortion artifacts that occur with

2 mentions: @AkiraTOSEI@AkiraTOSEI
Keywords: gan
Date: 2021/02/20 17:21

Referring Tweets

@AkiraTOSEI t.co/kbIhWe04k0 数枚の画像からアニメーションを作成する研究。部分ごとに3つのレイヤに分けて生成させることで前景と背景を混同しない画像生成をさせ、同時にマスクも予測させることで各部分の接続部が離れないような学習をさせている t.co/rNmtRwZpoW
@AkiraTOSEI t.co/kbIhWe04k0 This is a study of creating animation from several images. By dividing each part of the image into three layers, they can generate images that do not confuse the foreground and background. t.co/JhkkGmzDed

Related Entries

Read more [2004.12411] Disentangled Image Generation Through Structured Noise Injectionopen searchopen navigat...
0 users, 2 mentions 2020/06/01 13:01
Read more [1901.07884] Rank-consistent Ordinal Regression for Neural Networksopen searchopen navigation menuco...
0 users, 2 mentions 2020/07/28 12:53
Read more [2003.10780] Rethinking Class-Balanced Methods for Long-Tailed Visual Recognition from a Domain Adap...
0 users, 2 mentions 2020/08/07 11:21
Read more [2007.15963] Disentangling Human Error from the Ground Truth in Segmentation of Medical Images
0 users, 2 mentions 2021/01/06 20:21
Read more [2101.09865] ECOL-R: Encouraging Copying in Novel Object Captioning with Reinforcement Learning
0 users, 2 mentions 2021/02/11 11:21
Read more [2010.04456] Augmenting Physical Models with Deep Networks for Complex Dynamics Forecasting
0 users, 2 mentions 2021/02/12 11:21