Skip to content

Latest commit

 

History

History
307 lines (228 loc) · 22.4 KB

NOVEL-VIEW-SYNTHESIS-FOR-OBJECTS-AND-SCENES.md

File metadata and controls

307 lines (228 loc) · 22.4 KB

Novel-View Synthesis for Objects and Scenes

Novel-View Synthesis

Based on Diffusion Models

iNVS: Repurposing Diffusion Inpainters for Novel View Synthesis.
Yash Kant, Aliaksandr Siarohin, Michael Vasilkovsky, Riza Alp Guler, Jian Ren, Sergey Tulyakov, Igor Gilitschenski.
SIGGRAPH Asia 2023. [PDF] [Project]

Based on Implicit Neural Representations

Vision Transformer for NeRF-Based View Synthesis from a Single Input Image.
Kai-En Lin, Lin Yen-Chen, Wei-Sheng Lai, Tsung-Yi Lin, Yi-Chang Shih, Ravi Ramamoorthi.
WACV 2023. [PDF] [Project] [Code]

Seeing 3D Objects in a Single Image via Self-Supervised Static-Dynamic Disentanglement.
Prafull Sharma, Ayush Tewari, Yilun Du, Sergey Zakharov, Rares Ambrus, Adrien Gaidon, William T. Freeman, Fredo Durand, Joshua B. Tenenbaum, Vincent Sitzmann.
ICLR 2023. [PDF] [Project]

PVSeRF: Joint Pixel-, Voxel- and Surface-Aligned Radiance Field for Single-Image Novel View Synthesis.
Xianggang Yu, Jiapeng Tang, Yipeng Qin, Chenghong Li, Linchao Bao, Xiaoguang Han, Shuguang Cui.
ACM MM 2022. [PDF]

Unsupervised Learning of Efficient Geometry-Aware Neural Articulated Representations.
Atsuhiro Noguchi, Xiao Sun, Stephen Lin, Tatsuya Harada.
ECCV 2022. [PDF] [Project] [Code]

3D-Aware Semantic-Guided Generative Model for Human Synthesis.
Jichao Zhang, Enver Sangineto, Hao Tang, Aliaksandr Siarohin, Zhun Zhong, Nicu Sebe, Wei Wang.
ECCV 2022. [PDF] [Code]

DyNeRF: Neural 3D Video Synthesis.
Tianye Li, Mira Slavcheva, Michael Zollhoefer, Simon Green, Christoph Lassner, Changil Kim, Tanner Schmidt, Steven Lovegrove, Michael Goesele, Zhaoyang Lv.
CVPR 2022 (oral). [PDF] [Project]

Neural Rays for Occlusion-aware Image-based Rendering.
Yuan Liu, Sida Peng, Lingjie Liu, Qianqian Wang, Peng Wang, Christian Theobalt, Xiaowei Zhou, Wenping Wang.
CVPR 2022. [PDF] [Code] [Code]

Novel-View Synthesis of Human Tourist Photos.
Jonathan Freer, Kwang Moo Yi, Wei Jiang, Jongwon Choi, Hyung Jin Chang.
WACV 2022. [PDF]

Fast and Explicit Neural View Synthesis.
Pengsheng Guo, Miguel Angel Bautista, Alex Colburn, Liang Yang, Daniel Ulbricht, Joshua M. Susskind, Qi Shan.
WACV 2022. [PDF]

Seeing Implicit Neural Representations as Fourier Series.
Nuri Benbarka, Timon Höfer, Hamd ul-moqeet Riaz, Andreas Zell.
WACV 2022. [PDF]

NeuLF: Efficient Novel View Synthesis with Neural 4D Light Field.
Zhong Li, Liangchen Song, Celong Liu, Junsong Yuan, Yi Xu.
EGSR 2022. [PDF] [Project] [Code]

LENS: Localization enhanced by NeRF synthesis.
Arthur Moreau, Nathan Piasco, Dzmitry Tsishkou, Bogdan Stanciulescu, Arnaud de La Fortelle.
CoRL 2021. [PDF]

Dynamic View Synthesis from Dynamic Monocular Video.
Chen Gao, Ayush Saraf, Johannes Kopf, Jia-Bin Huang.
ICCV 2021. [PDF] [Project]

MINE: Towards Continuous Depth MPI with NeRF for Novel View Synthesis.
Jiaxin Li, Zijian Feng, Qi She, Henghui Ding, Changhu Wang, Gim Hee Lee.
ICCV 2021. [PDF] [Code]

NeRFlow: Neural Radiance Flow for 4D View Synthesis and Video Processing.
Yilun Du, Yinan Zhang, Hong-Xing Yu, Joshua B. Tenenbaum, Jiajun Wu.
ICCV 2021. [PDF] [Project]

Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synthesis of a Dynamic Scene From Monocular Video.
Edgar Tretschk, Ayush Tewari, Vladislav Golyanik, Michael Zollhofer, Christoph Lassner, Christian Theobalt.
ICCV 2021 (oral). [PDF]

Baking Neural Radiance Fields for Real-Time View Synthesis.
Peter Hedman, Pratul P. Srinivasan, Ben Mildenhall, Jonathan T. Barron, Paul Debevec.
ICCV 2021 (oral). [PDF] [Project]

Animatable Neural Radiance Fields for Human Body Modeling.
Sida Peng, Junting Dong, Qianqian Wang, Shangzhan Zhang, Qing Shuai, Hujun Bao, Xiaowei Zhou.
ICCV 2021. [PDF] [Project] [Code]

3D Shape Generation with Grid-based Implicit Functions.
Moritz Ibing, Isaak Lim, Leif Kobbelt.
CVPR 2021. [PDF]

Stereo Radiance Fields (SRF): Learning View Synthesis for Sparse Views of Novel Scenes.
Julian Chibane, Aayush Bansal, Verica Lazova, Gerard Pons-Moll.
CVPR 2021. [PDF]

ID-Unet: Iterative Soft and Hard Deformation for View Synthesis.
Mingyu Yin, Li Sun, Qingli Li.
CVPR 2021. [PDF]

NSFF: Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes.
Zhengqi Li, Simon Niklaus, Noah Snavely, Oliver Wang.
CVPR 2021. [PDF] [Project] [Code]

Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humans.
Sida Peng, Yuanqing Zhang, Yinghao Xu, Qianqian Wang, Qing Shuai, Hujun Bao, Xiaowei Zhou.
CVPR 2021 (Best paper candidate). [PDF] [Project] [Code]

NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections.
Ricardo Martin-Brualla, Noha Radwan, Mehdi S. M. Sajjadi, Jonathan T. Barron, Alexey Dosovitskiy, Daniel Duckworth.
CVPR 2021 (oral). [PDF] [Code]

pixelNeRF: Neural Radiance Fields from One or Few Images.
Alex Yu, Vickie Ye, Matthew Tancik, Angjoo Kanazawa.
CVPR 2021. [PDF] [Project] [Code]

X-Fields: Implicit Neural View-, Light- and Time-Image Interpolation.
MOJTABA BEMANA, KAROL MYSZKOWSKI, HANS-PETER SEIDEL, TOBIAS RITSCHEL.
TOG 2020. [PDF]

GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis.
Katja Schwarz, Yiyi Liao, Michael Niemeyer, Andreas Geiger.
NeurIPS 2020. [PDF] [Project] [Code]

NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis.
Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, Ren Ng.
ECCV 2020. [PDF] [Project] [Gtihub-Tensorflow] [krrish94-PyTorch] [yenchenlin-PyTorch]

NeRF++: Analyzing and Improving Neural Radiance Fields.
Kai Zhang, Gernot Riegler, Noah Snavely, Vladlen Koltun.
arxiv 2020. [PDF] [Code]

From Single Image

Infinite Nature: Perpetual View Generation of Natural Scenes from a Single Image.
Andrew Liu, Richard Tucker, Varun Jampani, Ameesh Makadia, Noah Snavely, Angjoo Kanazawa.
ICCV 2021. [PDF] [Project] [Gtihub]

Worldsheet: Wrapping the World in a 3D Sheet for View Synthesis from a Single Image.
Ronghang Hu, Deepak Pathak.
ICCV 2021. [PDF] [Project]

Layout-Guided Novel View Synthesis from a Single Indoor Panorama.
Jiale Xu, Jia Zheng, Yanyu Xu, Rui Tang, Shenghua Gao.
CVPR 2021. [PDF]

pixelNeRF: Neural Radiance Fields from One or Few Images.
Alex Yu, Vickie Ye, Matthew Tancik, Angjoo Kanazawa.
CVPR 2021. [PDF] [Project]

ShaRF: Shape-conditioned Radiance Fields from a Single View.
Konstantinos Rematas, Ricardo Martin-Brualla, Vittorio Ferrari.
ICML 2021. [PDF] [Project]

Generative View Synthesis: From Single-view Semantics to Novel-view Images.
Tewodros Habtegebrial, Varun Jampani, Orazio Gallo, Didier Stricker.
NeurIPS 2020. [PDF] [Project]

Others

A Portable Multiscopic Camera for Novel View and Time Synthesis in Dynamic Scenes.
Tianjia Zhang, Yuen-Fui Lau, Qifeng Chen.
IROS 2022. [PDF]

CompNVS: Novel View Synthesis with Scene Completion.
Zuoyue Li, Tianxing Fan, Zhenqiang Li, Zhaopeng Cui, Yoichi Sato, Marc Pollefeys, Martin R. Oswald.
ECCV 2022. [PDF]

NeurMiPs: Neural Mixture of Planar Experts for View Synthesis.
Zhi-Hao Lin, Wei-Chiu Ma, Hao-Yu Hsu, Yu-Chiang Frank Wang, Shenlong Wang.
CVPR 2022. [PDF] [Project] [Code]

RGBD-Net: Predicting Color and Depth Images for Novel Views Synthesis.
Phong Nguyen-Ha, Animesh Karnewar, Lam Huynh, Esa Rahtu, Jiri Matas, Janne Heikkila.
3DV 2021. [PDF] [Code]

SofGAN: A Portrait Image Generator with Dynamic Styling.
Anpei Chen, Ruiyang Liu, Ling Xie, Jingyi Yu.
TOG 2021. [PDF] [Code]

Geometry-Free View Synthesis: Transformers and No 3D Priors.
Robin Rombach, Patrick Esser, Björn Ommer.
ICCV 2021. [PDF] [Code]

Dynamic View Synthesis From Dynamic Monocular Video.
Chen Gao, Ayush Saraf, Johannes Kopf, Jia-Bin Huang.
ICCV 2021. [PDF]

Embedding Novel Views in a Single JPEG Image.
Yue Wu, Guotao Meng, Qifeng Chen.
ICCV 2021. [PDF]

Learning To Stylize Novel Views.
Hsin-Ping Huang, Hung-Yu Tseng, Saurabh Saini, Maneesh Singh, Ming-Hsuan Yang.
ICCV 2021. [PDF]

Deep 3D Mask Volume for View Synthesis of Dynamic Scenes.
Kai-En Lin, Lei Xiao, Feng Liu, Guowei Yang, Ravi Ramamoorthi.
ICCV 2021. [PDF] [Project]

PixelSynth: Generating a 3D-Consistent Experience from a Single Image.
Chris Rockwell, David F. Fouhey, Justin Johnson.
ICCV 2021. [PDF] [Project]

Self-Supervised Visibility Learning for Novel View Synthesis.
Yujiao Shi, Hongdong Li, Xin Yu.
CVPR 2021. [PDF]

Nex: Real-time View Synthesis with Neural Basis Expansion.
S. Wizadwongsa, P. Phongthawee, J. Yenphraphai, S. Suwajanakorn.
CVPR 2021 (oral). [PDF] [Project]

ID-Unet: Iterative Soft and Hard Deformation for View Synthesis.
Mingyu Yin, Li Sun, Qingli Li.
CVPR 2021. [PDF] [Code]

IBRNet: Learning Multi-View Image-Based Rendering.
Qianqian Wang, Zhicheng Wang, Kyle Genova, Pratul Srinivasan, Howard Zhou, Jonathan T. Barron, Ricardo Martin-Brualla, Noah Snavely, Thomas Funkhouser.
CVPR 2021. [PDF] [Project]

SVS: Stable View Synthesis.
Gernot Riegler, Vladlen Koltun.
CVPR 2021. [PDF] [Code] [Video]

Bowtie Networks: Generative Modeling for Joint Few-Shot Recognition and Novel-View Synthesis.
Zhipeng Bao, Yu-Xiong Wang, Martial Hebert.
ICLR 2021. [PDF]

Novel View Synthesis via Depth-guided Skip Connections.
Yuxin Hou, Arno Solin, Juho Kannala.
WACV 2021. [PDF] [Code]

Continuous Object Representation Networks: Novel View Synthesis without Target View Supervision.
Nicolai Häni, Selim Engin, Jun-Jee Chao, Volkan Isler.
NeurIPS 2020. [PDF] [Project]

FVS: Free View Synthesis.
Gernot Riegler, Vladlen Koltun.
ECCV 2020. [PDF] [Code&Data]

Rotationally-Temporally Consistent Novel-View Synthesis of Human Performance Video.
Youngjoong Kwon, Stefano Petrangeli, Dahun Kim, Haoliang Wang, Eunbyung Park, Viswanathan Swaminathan, Henry Fuchs.
ECCV 2020. [pdf] [Supplement]

A Recurrent Transformer Network for Novel View Action Synthesis.
Kara Marie Schatz, Erik Quintanilla, Shruti Vyas, Yogesh S Rawat.
ECCV 2020. [pdf] [Supplement]

MatryODShka: Real-time 6DoF Video View Synthesis using Multi-Sphere Images.
Benjamin Attal, Selena Ling, Aaron Gokaslan, Christian Richardt, James Tompkin.
ECCV 2020. [pdf] [Supplement]

Novel View Synthesis on Unpaired Data by Conditional Deformable Variational Auto-Encoder.
Mingyu Yin, Li Sun, Qingli Li.
ECCV 2020. [pdf] [Supplement]

Deep Multi Depth Panoramas for View Synthesis.
Kai-En Lin, Zexiang Xu, Ben Mildenhall, Pratul P. Srinivasan, Yannick Hold-Geoffroy, Stephen DiVerdi, Qi Sun, Kalyan Sunkavalli, Ravi Ramamoorthi.
ECCV 2020. [pdf] [Supplement]

Semantic View Synthesis.
Hsin-Ping Huang, Hung-Yu Tseng, Hsin-Ying Lee, Jia-Bin Huang.
ECCV 2020. [PDF] [Project]

Deep Multi Depth Panoramas for View Synthesis.
Kai-En Lin, Zexiang Xu, Ben Mildenhall, Pratul P. Srinivasan, Yannick Hold-Geoffroy, Stephen DiVerdi, Qi Sun, Kalyan Sunkavalli, Ravi Ramamoorthi.
ECCV 2020. [PDF]

AUTO3D: Novel View Synthesis Through Unsupervisely Learned Variational Viewpoint and Global 3D Representation.
Xiaofeng Liu, Tong Che, Yiqun Lu, Chao Yang, Site Li, Jane You.
ECCV 2020. [PDF]

Novel View Synthesis of Dynamic Scenes with Globally Coherent Depths from a Monocular Camera.
Jae Shin Yoon, Kihwan Kim, Orazio Gallo, Hyun Soo Park, Jan Kautz.
CVPR 2020. [PDF]

Self-Supervised 3D Human Pose Estimation via Part Guided Novel Image Synthesis.
Jogendra Nath Kundu, Siddharth Seth, Varun Jampani, Mugalodi Rakesh, R. Venkatesh Babu, Anirban Chakraborty.
CVPR 2020. [PDF]

IGNOR: Image-guided Neural Object Rendering.
Justus Thies, Michael Zollhöfer, Christian Theobalt, Marc Stamminger, Matthias Nießner.
ICLR 2020. [PDF] [Project]

Monocular Neural Image Based Rendering with Continuous View Control.
Xu Chen, Jie Song, Otmar Hilliges.
ICCV 2019. [PDF]

Extreme View Synthesis.
Inchang Choi, Orazio Gallo, Alejandro Troccoli, Min H. Kim, Jan Kautz.
ICCV 2019. [PDF]

Transformable Bottleneck Networks.
Kyle Olszewski, Sergey Tulyakov, Oliver Woodford, Hao Li, Linjie Luo.
ICCV 2019. [PDF]

View Independent Generative Adversarial Network for Novel View Synthesis.
Xiaogang Xu, Ying-Cong Chen, Jiaya Jia.
ICCV 2019. [PDF]

PVA: Pixel-Aligned Volumetric Avatars.
Amit Raj, Michael Zollhöfer, Tomas Simon, Jason Saragih, Shunsuke Saito, James Hays, Stephen Lombardi.
CVPR 2021 [PDF][Project]

LLFF: Local Light Field Fusion: Practical View Synthesis with Prescriptive Sampling Guidelines.
Ben Mildenhall, Pratul Srinivasan, Rodrigo Ortiz-Cayon, Nima Khademi Kalantari, Ravi Ramamoorthi, Ren Ng, Abhishek Kar.
SIGGRAPH 2019. [PDF] [Project] [Code]

Neural Volumes: Learning Dynamic Renderable Volumes from Images.
Stephen Lombardi, Tomas Simon, Jason Saragih, Gabriel Schwartz, Andreas Lehrmann, Yaser Sheikh.
SIGGRAPH 2019. [PDF] [Code]