Adversarial Generation of Continuous Images
CVPR 2021
Abstract
In most existing learning systems, images are typically viewed as 2D pixel arrays. However, in another paradigm gaining popularity, a 2D image is represented as an implicit neural representation (INR) — an MLP that predicts an RGB pixel value given its \((x,y)\) coordinate. In this paper, we propose two novel architectural techniques for building INR-based image decoders: factorized multiplicative modulation and multi-scale INRs, and use them to build a state-of-the-art continuous image GAN. Previous attempts to adapt INRs for image generation were limited to MNIST-like datasets and do not scale to complex real-world data. Our proposed INR-GAN architecture improves the performance of continuous image generators by several times, greatly reducing the gap between continuous image GANs and pixel-based ones. Apart from that, we explore several exciting properties of the INR-based decoders, like out-of-the-box superresolution, meaningful image-space interpolation, accelerated inference of low-resolution images, an ability to extrapolate outside of image boundaries, and strong geometric prior.
Main idea
Properties
The key feature of the INR-based decoders lies in its properties. In our paper, we explore several of them: image extrapolation, superresolution, meaningful interpolation, strong geometric prior and others.
Related work
CIPS is a contemporary work which also builds a large-scale INR-based GAN for image generation.
BibTeX
@InProceedings{inr-gan, author = {Skorokhodov, Ivan and Ignatyev, Savva and Elhoseiny, Mohamed}, title = {Adversarial Generation of Continuous Images}, booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)}, month = {June}, year = {2021}, pages = {10753-10764} } @inproceedings{cips, title={Image generators with conditionally-independent pixel synthesis}, author={Anokhin, Ivan and Demochkin, Kirill and Khakhulin, Taras and Sterkin, Gleb and Lempitsky, Victor and Korzhenkov, Denis}, booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition}, pages={14278--14287}, year={2021} }