[2102.06529] Improving Object Detection in Art Images Using Only Style Transfer

Despite recent advances in object detection using deep learning neural networks, these neural networks still struggle to identify objects in art images such as paintings and drawings. This challenge is known as the cross depiction problem and it stems in part from the tendency of neural networks to prioritize identification of an object's texture over its shape. In this paper we propose and evaluate a process for training neural networks to localize objects - specifically people - in art images. We generate a large dataset for training and validation by modifying the images in the COCO dataset using AdaIn style transfer. This dataset is used to fine-tune a Faster R-CNN object detection network, which is then tested on the existing People-Art testing dataset. The result is a significant improvement on the state of the art and a new way forward for creating datasets to train neural networks to process art images.

3 mentions: @ak92501@AkiraTOSEI@AkiraTOSEI
Date: 2021/02/19 11:21

Referring Tweets

@AkiraTOSEI t.co/kN8dZrXH9e MS COCOを芸術作品風にスタイル変換したデータでfine-tuningを行うことにより、芸術作品中の人物検出を行う研究。原理上人物以外の検出にも利用でき、ある特定の物体を写した芸術作品を収集する場合に活用できる可能性があるとのこと。 t.co/MH9WD0Za0i
@AkiraTOSEI t.co/kN8dZrXH9e This is a study on detecting people in artworks by fine-tuning with the modified MS COCO that is style transferred to art. In principle, it can be used for the detection of non-people, and it may be used for collecting artworks that depict a certain object t.co/3ucRiREOzM

Related Entries

Read more [2002.04724] Improved Consistency Regularization for GANscontact arXivarXiv Twitter
0 users, 7 mentions 2020/02/13 20:21
Read more [2008.00951] Encoding in Style: a StyleGAN Encoder for Image-to-Image Translationopen searchopen nav...
0 users, 3 mentions 2020/08/11 14:21
Read more [2009.13829] TinyGAN: Distilling BigGAN for Conditional Image Generationopen searchopen navigation m...
0 users, 4 mentions 2020/10/07 11:21
Read more [2011.04181] Two-Stream Appearance Transfer Network for Person Image Generationopen searchopen navig...
0 users, 3 mentions 2020/11/18 11:22
Read more [2011.15126] One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencingopen searchopen ...
0 users, 4 mentions 2020/12/01 03:51