[1911.01655] High Fidelity Video Prediction with Large Stochastic Recurrent Neural Networks

Predicting future video frames is extremely challenging, as there are many factors of variation that make up the dynamics of how frames change through time. Previously proposed solutions require complex inductive biases inside network architectures with highly specialized computation, including segmentation masks, optical flow, and foreground and background separation. In this work, we question if such handcrafted architectures are necessary and instead propose a different approach: finding minimal inductive bias for video prediction while maximizing network capacity. We investigate this question by performing the first large-scale empirical study and demonstrate state-of-the-art performance by learning large models on three different datasets: one for modeling object interactions, one for modeling human motion, and one for modeling car driving.

9 mentions: @RubenEVillegas@hillbig@roadrunning01@honglaklee@hardmaru@jaguring1@hillbig@arxiv_cs_cv_pr
Date: 2019/11/07 00:51

Referring Tweets

@hillbig 動画予測はセグメンテーションマスクやオプティカルフローなどの帰納バイアスを使わずとも、単にネットワークサイズを数倍大きくするだけで予測精度を大きく改善できる。RNN、確率的遷移の利用も有効 t.co/i690hvGpLp t.co/UNl6ATsIj9
@hardmaru High Fidelity Video Prediction with Large Stochastic RNNs They investigate inductive bias of video prediction networks via a large scale empirical study. Their learned large models get SOTA on datasets for object interaction, human motion and car driving. t.co/CMUUkvxEsV t.co/1yDjlom3h0
@honglaklee In our #NeurIPS2019 work (t.co/f8YP6Gnhuo), we show that large video generation models with a modest inductive bias can achieve compelling results on some benchmarks (e.g., human motions and robot/object interaction). More videos available at: t.co/tgFw8pWq61 t.co/uaXBQBSa6m t.co/BRHafFp3fA
@jaguring1 高精度なビデオ予測で成果。人間は行動をする時、近未来をある程度は予想してる。これは将来のAIにとっても重要な能力 High Fidelity Video Prediction with Large Stochastic Recurrent Neural Networks t.co/z6UM7gSKNX サイト(結果) t.co/vvMvVsyxtH t.co/ns707Qwp3t
@hillbig High fidelity video prediction can be achieved just by using a larger model (~10x) without other inductive biases such as optical flow or segmentation masks. Also, recurrent connections and stochastic transitions are effective. t.co/i690hvGpLp t.co/UNl6ATsIj9
@RubenEVillegas Check out our study on the effects of inductive bias and model capacity in video prediction models. This is work with @arkanathpathak @harinidkannan @doomie @quocleix @honglaklee Paper: t.co/zBP3XgVwzb Website: t.co/CjVJaMhvFE t.co/vzbseeYMr9
@roadrunning01 High Fidelity Video Prediction with Large Stochastic Recurrent Neural Networks pdf: t.co/xmkENCBC2g abs: t.co/w9PQwmsKRR project page: t.co/ekOWvBZ9eW t.co/P3S4yzzI6Q

Related Entries

Read more Neural networks for Graph Data NeurIPS2018読み会@PFN
25 users, 9 mentions 2019/01/26 09:46
Read more GitHub - thunlp/GNNPapers: Must-read papers on graph neural networks (GNN)
1 users, 0 mentions 2019/08/18 08:16
Read more Deep Forest :Deep Neural Networkの代替へ向けて - QiitaQiita
0 users, 0 mentions 2018/04/25 17:22
Read more GitHub - NervanaSystems/nlp-architect: A model library for exploring state-of-the-art deep learning ...
3 users, 2 mentions 2019/11/03 14:20
Read more GitHub - nnzhan/Awesome-Graph-Neural-Networks: Paper Lists for Graph Neural Networks
0 users, 2 mentions 2019/05/21 15:48