FEW-SHOT DIGITAL IMAGE GENERATION USING GAN-TO-GAN TRANSLATION

Utkarsh Ojha; Yijun Li; Richard Zhang; Jingwan Lu; Elya Shechtman; Alexei A. Efros

Patent

Atlas literature

Patent

US 11,763,495 B2

FEW-SHOT DIGITAL IMAGE GENERATION USING GAN-TO-GAN TRANSLATION

Utkarsh Ojha, Yijun Li, Richard Zhang, Jingwan Lu et al.

Adobe Inc., San Jose, CA (US)·Sep. 19, 2023·US

Drawings

Patent drawings and their descriptions. Click a drawing to enlarge it.

FIG. 1

FIG. 1 illustrates an example system environment in which a GAN translation system operates in accordance with one or more implementations;

FIG. 2

FIG. 2 illustrates an overview of utilizing GAN transla- tion to adapt a source generative adversarial neural network to a target domain in accordance with one …

FIG. 3

FIG. 3 illustrates a schematic diagram illustrating a pro- cess for GAN translation that preserves relative feature distances in accordance with one or more …

FIG. 4

FIG. 4 illustrates a process for enforcing relaxed realism during GAN translation in accordance with one or more implementations;

FIG. 5

FIGS. 5A-5B illustrate corresponding digital images gen- erated utilizing a target generative adversarial neural net- work and a source generative adversarial …

FIG. 6

FIG. 6 illustrates example digital images generated uti- lizing target generative adversarial neural networks gener- ated utilizing GAN translation with …

FIG. 7

FIG. 7 illustrates a table of performance metrics compar- ing the GAN translation system with other systems in accordance with one or more implementations;

FIG. 8

FIG. 8 illustrates example digital images generated by a target generative adversarial neural network compared to digital images generated by other systems in …

FIG. 9

FIG. 9 illustrates a schematic diagram of a GAN trans- lation system in accordance with one or more implementa- tions;

FIG. 10

FIGS. 10A-10B illustrate ﬂowcharts of a series of acts for modifying a source generative adversarial neural network using few-shot adaptation to generate a …

FIG. 11

FIG. 11 illustrates a block diagram of an example com- puting device in accordance with one or more implementa- tions.

Claims

Claims define the patent's legal scope. Independent claims stand alone; dependent claims (nested) narrow them. Click a claim to expand its dependents.

3 independent · 17 dependent

1
Independent
A non-transitory computer readable medium compris-ing instructions that, when executed by at least one proces-sor, cause a computing device to: generate a set of digital images belonging to a source domain utilizing a ﬁrst generative adversarial neural network comprising parameters learned from the source domain; determine relative pairwise distances in a feature space among pairs of digital images from the set of digital images belonging to the source domain; and learn parameters for a second generative adversarial neu-ral network by updating the parameters from the ﬁrst generative adversarial neural network while forcing the second generative adversarial neural network to preserve the relative pairwise distances in generating digi-tal images in a target domain.
1. 2
  Dependent← claim 1
  The non-transitory computer readable medium of claim 1, further comprising instructions that, when executed by the 5 at least one processor, cause the computing device to gen-erate a digital image within a target domain utilizing the second generative adversarial neural network.
2. 3
  Dependent← claim 1
  The non-transitory computer readable medium of claim 1, further comprising instructions that, when executed by the 10 at least one processor, cause the computing device to gen-erate the set of digital images belonging to the source domain by: sampling a batch of noise vectors; and utilizing the ﬁrst generative adversarial neural network to generate the set of digital images belonging to the source domain from the batch of noise vectors.
3. 4
  Dependent← claim 1
  The non-transitory computer readable medium of claim 1, further comprising instructions that, when executed by the 20 at least one processor, cause the computing device to deter-mine the relative pairwise distances by determining dis-tances within a feature space between digital image feature vectors corresponding to digital images belonging to the source domain.
4. 5
  Dependent← claim 1
  The non-transitory computer readable medium of claim 1, further comprising instructions that, when executed by the at least one processor, cause the computing device to force the second generative adversarial neural network to preserve the relative pairwise distances by: determining a relative order of similarity among the pairs of digital images from the set of digital images belong-ing to the source domain; and enforcing the relative order of similarity for pairs of digital images generated by the second generative adversarial neural network.
5. 6
  Dependent← claim 1
  The non-transitory computer readable medium of claim 1, further comprising instructions that, when executed by the at least one processor, cause the computing device to des-ignate anchor regions comprising subspaces within a latent space.
9
Independent
A system comprising: one or more memory devices storing a source generative adversarial neural network comprising parameters learned from a set of digital images belonging to a source domain; one or more processors conﬁgured to cause the system to generate a target generative adversarial neural network from the source generative adversarial neural network utilizing example digital images from a target domain by: generating, utilizing the target generative adversarial neural network, a ﬁrst set of digital images from a ﬁrst set of latent vectors sampled from an anchor region comprising a region of a latent space for enforcing image-level realism; generating, utilizing the target generative adversarial neural network, a second set of digital images from a second set of latent vectors sampled from a nonanchor region comprising a region of the latent space for enforcing patch-level realism; and updating parameters of the target generative adversarial neural network by: enforcing, for the ﬁrst set of digital images of the anchor region, a measure of image-level realism compared to the example digital images from the target domain; and enforcing, for the second set of digital images of the non-anchor region, a measure of patch-level real-ism compared to the example digital images from the target domain.
1. 10
  Dependent← claim 9
  The system of claim 9, wherein the one or more processors are further conﬁgured to cause the system to perform operations comprising splitting the latent space into the anchor region and the non-anchor region by deﬁning a number of sub-regions in the latent space corresponding to a number of the example digital images from the target domain, wherein the anchor region comprises the subregions and the non-anchor region comprises a remainder of the latent space.
2. 11
  Dependent← claim 9
  The system of claim 9, wherein: enforcing the measure of image-level realism for the ﬁrst set of digital images comprises utilizing an image-level realism measure; and enforcing the measure of patch-level realism for the second set of digital images comprises utilizing a partial image-level realism measure.
3. 13
  Dependent← claim 9
  The system of claim 9, wherein the one or more processors are further conﬁgured to cause the system to perform operations comprising: determining relative feature distances within a feature space between source feature vectors generated from pairs of latent vectors generated utilizing the source generative adversarial neural network; generating target feature vectors from the pairs of latent vectors utilizing the target generative adversarial neural network; and updating parameters of the target generative adversarial neural network by enforcing a cross-domain distance consistency between the relative feature distances and distances in the feature space between the target feature vectors.
16
Independent
A computer-implemented method for preserving diversity and realism in target digital images utilizing few-shot adaptation for generative adversarial neural networks, the computer-implemented method comprising: generating a set of digital images belonging to a source domain utilizing a ﬁrst generative adversarial neural network comprising parameters learned from the source domain; determining relative pairwise distances in a feature space among pairs of digital images from the set of digital images belonging to the source domain; and learning parameters for a second generative adversarial neural network by updating the parameters from the B₂ ﬁrst generative adversarial neural network while forc-ing the second generative adversarial neural network to preserve the relative pairwise distances in generating digital images in a target domain.
1. 17
  Dependent← claim 16
  The computer-implemented method of claim 16, fur-ther comprising generating a digital image within a target domain utilizing the second generative adversarial neural network.
2. 18
  Dependent← claim 16
  The computer-implemented method of claim 16, fur-ther comprising generating the set of digital images belong-ing to the source domain by: sampling a batch of noise vectors; and utilizing the ﬁrst generative adversarial neural network to generate the set of digital images belonging to the source domain from the batch of noise vectors.
3. 19
  Dependent← claim 16
  The computer-implemented method of claim 16, wherein determining the relative pairwise distances com-prises determining distances within a feature space between digital image feature vectors corresponding to digital images belonging to the source domain.
4. 20
  Dependent← claim 16
  The computer-implemented method of claim 16, fur-ther comprising designating anchor regions comprising subspaces within a latent space. ∗ ∗ ∗ ∗ ∗

Cited prior art

Patents and literature cited by this patent (applicant and examiner references).

Cited patents · 7

US 10,176,405 B110,176,405 B1 * 1/2019 Zhou....................... G06T 7/246examiner
US 2018/0314716 A12018/0314716 A1 * 11/2018 Kim.......................... G06T 1/20examiner
US 2019/0295302 A12019/0295302 A1 * 9/2019 Fu.......................... G06V 10/82examiner
US 2019/0332850 A12019/0332850 A1 * 10/2019 Sharma..................... G06T 5/20

FEW-SHOT DIGITAL IMAGE GENERATION USING GAN-TO-GAN TRANSLATION | Matter42 Literature

Cited non-patent literature · 4

Edge-Texture Feature-Based Image Forgery Detec- tion with Cross-Dataset Evaluation. Asghar et al., “Edge-Texture Feature-Based Image Forgery Detec- tion with Cross-Dataset Evaluation” (Year: 2019).
Triangle Generative Adversarial Networks. Gan et al., “Triangle Generative Adversarial Networks” (Year: 2017).
Image Super-Resolution using Progressive Gen- erative Adversarial Networks for Medical Image Analysis. Mahapatra et al., “Image Super-Resolution using Progressive Gen- erative Adversarial Networks for Medical Image Analysis” (Year: 2019).* Rameen Abdal, Yipeng Qin, and Peter Wonka. Image2stylegan: How to embed images into the stylegan latent space? In Int. Conf. Comput. Vis., 2019. Sagie Benaim and Lior Wolf. One-sided unsupervised domain mapping. In Adv. Neural Inform. Process. Syst., 2017. Andrew Brock, Jeff Donahue, and Karen Simonyan. Large scale gan training for high ﬁdelity natural image synthesis. arXiv preprint arXiv:1809.11096, 2018. Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. arXiv preprint arXiv:2002.05709, 2020. Alexey Dosovitskiy and Thomas Brox. Inverting visual represen- tations with convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recog- nition, pp. 4829-4837, 2016. Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In ICML, 2017. Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learn- ing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9729-9738, 2020. Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. In Adv. Neural Inform. Process. Syst., 2017. Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. Image-to-image translation with conditional adversarial networks. In IEEE Conf. Comput. Vis. Pattern Recog., 2017. Justin Johnson, Alexandre Alahi, and Li Fei-Fei. Perceptual losses for real-time style transfer and super-resolution. In European con- ference on computer vision, pp. 694-711. Springer, 2016. Tero Karras, Miika Aittala, Janne Hellsten, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Training generative adversarial networks with limited data. In Adv. Neural Inform. Process. Syst., 2020. Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In IEEE Conf. Comput. Vis. Pattern Recog., 2019. Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Analyzing and improving the image quality of stylegan. arXiv preprint arXiv:1912.04958, 2019. James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13):3521-3526, 2017. Brenden M Lake, Ruslan Salakhutdinov, and Joshua B Tenenbaum. Human-level concept learning through probabilistic program induc- tion. Science, 350(6266):1332-1338, 2015. Yijun Li, Richard Zhang, Jingwan Lu, and Eli Shechtman. Few-shot image generation with elastic weight consolidation. InAdvances in Neural Information Processing Systems, 2020. Ming-Yu Liu, Xun Huang, Arun Mallya, Tero Karras, Timo Aila, Jaakko Lehtinen, and Jan Kautz. Few-shot unsupervised image-to- image translation. In Int. Conf. Comput. Vis., 2019. Shaohui Liu, Xiao Zhang, Jianqiao Wangni, and Jianbo Shi. Nor- malized diversiﬁcation. In IEEE Conf. Comput. Vis. Pattern Recog., 2019. Qi Mao, Hsin-Ying Lee, Hung-Yu Tseng, Siwei Ma, and Ming- Hsuan Yang. Mode seeking generative adversarial networks for diverse image synthesis. In IEEE Conf. Comput. Vis. Pattern Recog., 2019. Sangwoo Mo, Minsu Cho, and Jinwoo Shin. Freeze discriminator: A simple baseline for ﬁne-tuning gans.arXiv preprint arXiv:2002. 10964, 2020. Alex Nichol, Joshua Achiam, and John Schulman. On ﬁrst-order meta-learning algorithms.arXiv preprint arXiv:1803.02999, 2018. Atsuhiro Noguchi and Tatsuya Harada. Image generation from small datasets via batch statistics adaptation. In Int. Conf. Comput. Vis., 2019. Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding.arXiv preprint arXiv:1807. 03748, 2018. Kuniaki Saito, Kate Saenko, and Ming-Yu Liu. Coco-funit: Few- shot unsupervised image translation with a content conditioned style encoder.arXiv preprint arXiv:2007.07431, 2020. Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical net- works for few-shot learning. InAdv. Neural Inform. Process. Syst., 2017. Ngoc-Trung Tran, Tuan-Anh Bui, and Ngai-Man Cheung. Dist-gan: An improved gan using distance constraints. In Proceedings of the European Conference on Computer Vision (ECCV), 2018. Dmitry Ulyanov, Vadim Lebedev, Andrea Vedaldi, and Victor S Lempitsky. Texture networks Feed-forward synthesis of textures and stylized images. InICML, vol. 1, p. 4, 2016. Arash Vahdat and Jan Kautz. Nvae: A deep hierarchical variational autoencoder. In Neural Information ProcessingSystems (NeurIPS), 2020. Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Daan Wierstra, et al. Matching networks for one shot learning. In Adv. Neural Inform. Process. Syst., 2016. Xiaogang Wang and Xiaoou Tang. Face photo-sketch synthesis and recognition. IEEE Trans. Pattern Anal. Mach. Intell., 31(11):1055- 1967, 2009. Yaxing Wang, Abel Gonzalez-Garcia, David Berga, Luis Herranz, Fahad Shahbaz Khan, and Joost van de Weijer. Minegan: effective knowledge transfer from gans to target domains with few images. In IEEE Conf. Comput. Vis. Pattern Recog., 2020. Yaxing Wang, Salman Khan, Abel Gonzalez-Garcia, Joost van de Weijer, and Fahad Shahbaz Khan. Semi-supervised learning for few-shot image-to-image translation. In IEEE Conf. Comput. Vis. Pattern Recog., 2020. Yaxing Wang, Chenshen Wu, Luis Herranz, Joost van de Weijer, Abel Gonzalez-Garcia, and Bogdan Raducanu. Transferring gans: generating images from limited data. In Eur. Conf. Comput. Vis., 2018. Dingdong Yang, Seunghoon Hong, Yunseok Jang, Tianchen Zhao, and Honglak Lee. Diversity-sensitive conditional generative adversarial networks. arXiv preprint arXiv:1901.09024, 2019. Jordan Yaniv, Yael Newman, and Ariel Shamir. The face of art: landmark detection and geometric style in portraits. ACM Trans- actions on Graphics (TOG), 38(4):1-15, 2019. Fisher Yu, Yinda Zhang, Shuran Song, Ari Seff, and Jianxiong Xiao. Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv:1506.03365, 2015. Shengyu Zhao, Zhijian Liu, Ji Lin, Jun-Yan Zhu, and Song Han. Differentiable augmentation for data-efficient gan training. arXiv preprint arXiv:2006.10738, 2020. Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Int. Conf. Comput. Vis., 2017. Jun-Yan Zhu, Richard Zhang, Deepak Pathak, Trevor Darrell,
AlexeiAEfros, Oliver Wang, and Eli Shechtman. Toward multimodal image-to-image translation. In Adv. Neural Inform. Process. Syst., 2017. L. A. Gatys, A. S. Ecker, and M. Bethge. Image style transfer using convolutional neural networks. In CVPR, 2016. I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde- Farley, S. Ozair, A. Courville, and Y. Bengio. Generative adversarial nets. In NIPS, 2014.

Cited non-patent literature · 4

Edge-Texture Feature-Based Image Forgery Detec- tion with Cross-Dataset Evaluation. Asghar et al., “Edge-Texture Feature-Based Image Forgery Detec- tion with Cross-Dataset Evaluation” (Year: 2019).
Triangle Generative Adversarial Networks. Gan et al., “Triangle Generative Adversarial Networks” (Year: 2017).
Image Super-Resolution using Progressive Gen- erative Adversarial Networks for Medical Image Analysis. Mahapatra et al., “Image Super-Resolution using Progressive Gen- erative Adversarial Networks for Medical Image Analysis” (Year: 2019).* Rameen Abdal, Yipeng Qin, and Peter Wonka. Image2stylegan: How to embed images into the stylegan latent space? In Int. Conf. Comput. Vis., 2019. Sagie Benaim and Lior Wolf. One-sided unsupervised domain mapping. In Adv. Neural Inform. Process. Syst., 2017. Andrew Brock, Jeff Donahue, and Karen Simonyan. Large scale gan training for high ﬁdelity natural image synthesis. arXiv preprint arXiv:1809.11096, 2018. Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. arXiv preprint arXiv:2002.05709, 2020. Alexey Dosovitskiy and Thomas Brox. Inverting visual represen- tations with convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recog- nition, pp. 4829-4837, 2016. Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In ICML, 2017. Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learn- ing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9729-9738, 2020. Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. In Adv. Neural Inform. Process. Syst., 2017. Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. Image-to-image translation with conditional adversarial networks. In IEEE Conf. Comput. Vis. Pattern Recog., 2017. Justin Johnson, Alexandre Alahi, and Li Fei-Fei. Perceptual losses for real-time style transfer and super-resolution. In European con- ference on computer vision, pp. 694-711. Springer, 2016. Tero Karras, Miika Aittala, Janne Hellsten, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Training generative adversarial networks with limited data. In Adv. Neural Inform. Process. Syst., 2020. Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In IEEE Conf. Comput. Vis. Pattern Recog., 2019. Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Analyzing and improving the image quality of stylegan. arXiv preprint arXiv:1912.04958, 2019. James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13):3521-3526, 2017. Brenden M Lake, Ruslan Salakhutdinov, and Joshua B Tenenbaum. Human-level concept learning through probabilistic program induc- tion. Science, 350(6266):1332-1338, 2015. Yijun Li, Richard Zhang, Jingwan Lu, and Eli Shechtman. Few-shot image generation with elastic weight consolidation. InAdvances in Neural Information Processing Systems, 2020. Ming-Yu Liu, Xun Huang, Arun Mallya, Tero Karras, Timo Aila, Jaakko Lehtinen, and Jan Kautz. Few-shot unsupervised image-to- image translation. In Int. Conf. Comput. Vis., 2019. Shaohui Liu, Xiao Zhang, Jianqiao Wangni, and Jianbo Shi. Nor- malized diversiﬁcation. In IEEE Conf. Comput. Vis. Pattern Recog., 2019. Qi Mao, Hsin-Ying Lee, Hung-Yu Tseng, Siwei Ma, and Ming- Hsuan Yang. Mode seeking generative adversarial networks for diverse image synthesis. In IEEE Conf. Comput. Vis. Pattern Recog., 2019. Sangwoo Mo, Minsu Cho, and Jinwoo Shin. Freeze discriminator: A simple baseline for ﬁne-tuning gans.arXiv preprint arXiv:2002. 10964, 2020. Alex Nichol, Joshua Achiam, and John Schulman. On ﬁrst-order meta-learning algorithms.arXiv preprint arXiv:1803.02999, 2018. Atsuhiro Noguchi and Tatsuya Harada. Image generation from small datasets via batch statistics adaptation. In Int. Conf. Comput. Vis., 2019. Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding.arXiv preprint arXiv:1807. 03748, 2018. Kuniaki Saito, Kate Saenko, and Ming-Yu Liu. Coco-funit: Few- shot unsupervised image translation with a content conditioned style encoder.arXiv preprint arXiv:2007.07431, 2020. Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical net- works for few-shot learning. InAdv. Neural Inform. Process. Syst., 2017. Ngoc-Trung Tran, Tuan-Anh Bui, and Ngai-Man Cheung. Dist-gan: An improved gan using distance constraints. In Proceedings of the European Conference on Computer Vision (ECCV), 2018. Dmitry Ulyanov, Vadim Lebedev, Andrea Vedaldi, and Victor S Lempitsky. Texture networks Feed-forward synthesis of textures and stylized images. InICML, vol. 1, p. 4, 2016. Arash Vahdat and Jan Kautz. Nvae: A deep hierarchical variational autoencoder. In Neural Information ProcessingSystems (NeurIPS), 2020. Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Daan Wierstra, et al. Matching networks for one shot learning. In Adv. Neural Inform. Process. Syst., 2016. Xiaogang Wang and Xiaoou Tang. Face photo-sketch synthesis and recognition. IEEE Trans. Pattern Anal. Mach. Intell., 31(11):1055- 1967, 2009. Yaxing Wang, Abel Gonzalez-Garcia, David Berga, Luis Herranz, Fahad Shahbaz Khan, and Joost van de Weijer. Minegan: effective knowledge transfer from gans to target domains with few images. In IEEE Conf. Comput. Vis. Pattern Recog., 2020. Yaxing Wang, Salman Khan, Abel Gonzalez-Garcia, Joost van de Weijer, and Fahad Shahbaz Khan. Semi-supervised learning for few-shot image-to-image translation. In IEEE Conf. Comput. Vis. Pattern Recog., 2020. Yaxing Wang, Chenshen Wu, Luis Herranz, Joost van de Weijer, Abel Gonzalez-Garcia, and Bogdan Raducanu. Transferring gans: generating images from limited data. In Eur. Conf. Comput. Vis., 2018. Dingdong Yang, Seunghoon Hong, Yunseok Jang, Tianchen Zhao, and Honglak Lee. Diversity-sensitive conditional generative adversarial networks. arXiv preprint arXiv:1901.09024, 2019. Jordan Yaniv, Yael Newman, and Ariel Shamir. The face of art: landmark detection and geometric style in portraits. ACM Trans- actions on Graphics (TOG), 38(4):1-15, 2019. Fisher Yu, Yinda Zhang, Shuran Song, Ari Seff, and Jianxiong Xiao. Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv:1506.03365, 2015. Shengyu Zhao, Zhijian Liu, Ji Lin, Jun-Yan Zhu, and Song Han. Differentiable augmentation for data-efficient gan training. arXiv preprint arXiv:2006.10738, 2020. Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Int. Conf. Comput. Vis., 2017. Jun-Yan Zhu, Richard Zhang, Deepak Pathak, Trevor Darrell,
AlexeiAEfros, Oliver Wang, and Eli Shechtman. Toward multimodal image-to-image translation. In Adv. Neural Inform. Process. Syst., 2017. L. A. Gatys, A. S. Ecker, and M. Bethge. Image style transfer using convolutional neural networks. In CVPR, 2016. I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde- Farley, S. Ozair, A. Courville, and Y. Bengio. Generative adversarial nets. In NIPS, 2014.

Cited non-patent literature · 4

Edge-Texture Feature-Based Image Forgery Detec- tion with Cross-Dataset Evaluation. Asghar et al., “Edge-Texture Feature-Based Image Forgery Detec- tion with Cross-Dataset Evaluation” (Year: 2019).
Triangle Generative Adversarial Networks. Gan et al., “Triangle Generative Adversarial Networks” (Year: 2017).
Image Super-Resolution using Progressive Gen- erative Adversarial Networks for Medical Image Analysis. Mahapatra et al., “Image Super-Resolution using Progressive Gen- erative Adversarial Networks for Medical Image Analysis” (Year: 2019).* Rameen Abdal, Yipeng Qin, and Peter Wonka. Image2stylegan: How to embed images into the stylegan latent space? In Int. Conf. Comput. Vis., 2019. Sagie Benaim and Lior Wolf. One-sided unsupervised domain mapping. In Adv. Neural Inform. Process. Syst., 2017. Andrew Brock, Jeff Donahue, and Karen Simonyan. Large scale gan training for high ﬁdelity natural image synthesis. arXiv preprint arXiv:1809.11096, 2018. Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. arXiv preprint arXiv:2002.05709, 2020. Alexey Dosovitskiy and Thomas Brox. Inverting visual represen- tations with convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recog- nition, pp. 4829-4837, 2016. Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In ICML, 2017. Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learn- ing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9729-9738, 2020. Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. In Adv. Neural Inform. Process. Syst., 2017. Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. Image-to-image translation with conditional adversarial networks. In IEEE Conf. Comput. Vis. Pattern Recog., 2017. Justin Johnson, Alexandre Alahi, and Li Fei-Fei. Perceptual losses for real-time style transfer and super-resolution. In European con- ference on computer vision, pp. 694-711. Springer, 2016. Tero Karras, Miika Aittala, Janne Hellsten, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Training generative adversarial networks with limited data. In Adv. Neural Inform. Process. Syst., 2020. Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In IEEE Conf. Comput. Vis. Pattern Recog., 2019. Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Analyzing and improving the image quality of stylegan. arXiv preprint arXiv:1912.04958, 2019. James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13):3521-3526, 2017. Brenden M Lake, Ruslan Salakhutdinov, and Joshua B Tenenbaum. Human-level concept learning through probabilistic program induc- tion. Science, 350(6266):1332-1338, 2015. Yijun Li, Richard Zhang, Jingwan Lu, and Eli Shechtman. Few-shot image generation with elastic weight consolidation. InAdvances in Neural Information Processing Systems, 2020. Ming-Yu Liu, Xun Huang, Arun Mallya, Tero Karras, Timo Aila, Jaakko Lehtinen, and Jan Kautz. Few-shot unsupervised image-to- image translation. In Int. Conf. Comput. Vis., 2019. Shaohui Liu, Xiao Zhang, Jianqiao Wangni, and Jianbo Shi. Nor- malized diversiﬁcation. In IEEE Conf. Comput. Vis. Pattern Recog., 2019. Qi Mao, Hsin-Ying Lee, Hung-Yu Tseng, Siwei Ma, and Ming- Hsuan Yang. Mode seeking generative adversarial networks for diverse image synthesis. In IEEE Conf. Comput. Vis. Pattern Recog., 2019. Sangwoo Mo, Minsu Cho, and Jinwoo Shin. Freeze discriminator: A simple baseline for ﬁne-tuning gans.arXiv preprint arXiv:2002. 10964, 2020. Alex Nichol, Joshua Achiam, and John Schulman. On ﬁrst-order meta-learning algorithms.arXiv preprint arXiv:1803.02999, 2018. Atsuhiro Noguchi and Tatsuya Harada. Image generation from small datasets via batch statistics adaptation. In Int. Conf. Comput. Vis., 2019. Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding.arXiv preprint arXiv:1807. 03748, 2018. Kuniaki Saito, Kate Saenko, and Ming-Yu Liu. Coco-funit: Few- shot unsupervised image translation with a content conditioned style encoder.arXiv preprint arXiv:2007.07431, 2020. Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical net- works for few-shot learning. InAdv. Neural Inform. Process. Syst., 2017. Ngoc-Trung Tran, Tuan-Anh Bui, and Ngai-Man Cheung. Dist-gan: An improved gan using distance constraints. In Proceedings of the European Conference on Computer Vision (ECCV), 2018. Dmitry Ulyanov, Vadim Lebedev, Andrea Vedaldi, and Victor S Lempitsky. Texture networks Feed-forward synthesis of textures and stylized images. InICML, vol. 1, p. 4, 2016. Arash Vahdat and Jan Kautz. Nvae: A deep hierarchical variational autoencoder. In Neural Information ProcessingSystems (NeurIPS), 2020. Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Daan Wierstra, et al. Matching networks for one shot learning. In Adv. Neural Inform. Process. Syst., 2016. Xiaogang Wang and Xiaoou Tang. Face photo-sketch synthesis and recognition. IEEE Trans. Pattern Anal. Mach. Intell., 31(11):1055- 1967, 2009. Yaxing Wang, Abel Gonzalez-Garcia, David Berga, Luis Herranz, Fahad Shahbaz Khan, and Joost van de Weijer. Minegan: effective knowledge transfer from gans to target domains with few images. In IEEE Conf. Comput. Vis. Pattern Recog., 2020. Yaxing Wang, Salman Khan, Abel Gonzalez-Garcia, Joost van de Weijer, and Fahad Shahbaz Khan. Semi-supervised learning for few-shot image-to-image translation. In IEEE Conf. Comput. Vis. Pattern Recog., 2020. Yaxing Wang, Chenshen Wu, Luis Herranz, Joost van de Weijer, Abel Gonzalez-Garcia, and Bogdan Raducanu. Transferring gans: generating images from limited data. In Eur. Conf. Comput. Vis., 2018. Dingdong Yang, Seunghoon Hong, Yunseok Jang, Tianchen Zhao, and Honglak Lee. Diversity-sensitive conditional generative adversarial networks. arXiv preprint arXiv:1901.09024, 2019. Jordan Yaniv, Yael Newman, and Ariel Shamir. The face of art: landmark detection and geometric style in portraits. ACM Trans- actions on Graphics (TOG), 38(4):1-15, 2019. Fisher Yu, Yinda Zhang, Shuran Song, Ari Seff, and Jianxiong Xiao. Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv:1506.03365, 2015. Shengyu Zhao, Zhijian Liu, Ji Lin, Jun-Yan Zhu, and Song Han. Differentiable augmentation for data-efficient gan training. arXiv preprint arXiv:2006.10738, 2020. Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Int. Conf. Comput. Vis., 2017. Jun-Yan Zhu, Richard Zhang, Deepak Pathak, Trevor Darrell,
AlexeiAEfros, Oliver Wang, and Eli Shechtman. Toward multimodal image-to-image translation. In Adv. Neural Inform. Process. Syst., 2017. L. A. Gatys, A. S. Ecker, and M. Bethge. Image style transfer using convolutional neural networks. In CVPR, 2016. I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde- Farley, S. Ozair, A. Courville, and Y. Bengio. Generative adversarial nets. In NIPS, 2014.

Cited non-patent literature · 4

Edge-Texture Feature-Based Image Forgery Detec- tion with Cross-Dataset Evaluation. Asghar et al., “Edge-Texture Feature-Based Image Forgery Detec- tion with Cross-Dataset Evaluation” (Year: 2019).
Triangle Generative Adversarial Networks. Gan et al., “Triangle Generative Adversarial Networks” (Year: 2017).
Image Super-Resolution using Progressive Gen- erative Adversarial Networks for Medical Image Analysis. Mahapatra et al., “Image Super-Resolution using Progressive Gen- erative Adversarial Networks for Medical Image Analysis” (Year: 2019).* Rameen Abdal, Yipeng Qin, and Peter Wonka. Image2stylegan: How to embed images into the stylegan latent space? In Int. Conf. Comput. Vis., 2019. Sagie Benaim and Lior Wolf. One-sided unsupervised domain mapping. In Adv. Neural Inform. Process. Syst., 2017. Andrew Brock, Jeff Donahue, and Karen Simonyan. Large scale gan training for high ﬁdelity natural image synthesis. arXiv preprint arXiv:1809.11096, 2018. Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. arXiv preprint arXiv:2002.05709, 2020. Alexey Dosovitskiy and Thomas Brox. Inverting visual represen- tations with convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recog- nition, pp. 4829-4837, 2016. Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In ICML, 2017. Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learn- ing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9729-9738, 2020. Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. In Adv. Neural Inform. Process. Syst., 2017. Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. Image-to-image translation with conditional adversarial networks. In IEEE Conf. Comput. Vis. Pattern Recog., 2017. Justin Johnson, Alexandre Alahi, and Li Fei-Fei. Perceptual losses for real-time style transfer and super-resolution. In European con- ference on computer vision, pp. 694-711. Springer, 2016. Tero Karras, Miika Aittala, Janne Hellsten, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Training generative adversarial networks with limited data. In Adv. Neural Inform. Process. Syst., 2020. Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In IEEE Conf. Comput. Vis. Pattern Recog., 2019. Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Analyzing and improving the image quality of stylegan. arXiv preprint arXiv:1912.04958, 2019. James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13):3521-3526, 2017. Brenden M Lake, Ruslan Salakhutdinov, and Joshua B Tenenbaum. Human-level concept learning through probabilistic program induc- tion. Science, 350(6266):1332-1338, 2015. Yijun Li, Richard Zhang, Jingwan Lu, and Eli Shechtman. Few-shot image generation with elastic weight consolidation. InAdvances in Neural Information Processing Systems, 2020. Ming-Yu Liu, Xun Huang, Arun Mallya, Tero Karras, Timo Aila, Jaakko Lehtinen, and Jan Kautz. Few-shot unsupervised image-to- image translation. In Int. Conf. Comput. Vis., 2019. Shaohui Liu, Xiao Zhang, Jianqiao Wangni, and Jianbo Shi. Nor- malized diversiﬁcation. In IEEE Conf. Comput. Vis. Pattern Recog., 2019. Qi Mao, Hsin-Ying Lee, Hung-Yu Tseng, Siwei Ma, and Ming- Hsuan Yang. Mode seeking generative adversarial networks for diverse image synthesis. In IEEE Conf. Comput. Vis. Pattern Recog., 2019. Sangwoo Mo, Minsu Cho, and Jinwoo Shin. Freeze discriminator: A simple baseline for ﬁne-tuning gans.arXiv preprint arXiv:2002. 10964, 2020. Alex Nichol, Joshua Achiam, and John Schulman. On ﬁrst-order meta-learning algorithms.arXiv preprint arXiv:1803.02999, 2018. Atsuhiro Noguchi and Tatsuya Harada. Image generation from small datasets via batch statistics adaptation. In Int. Conf. Comput. Vis., 2019. Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding.arXiv preprint arXiv:1807. 03748, 2018. Kuniaki Saito, Kate Saenko, and Ming-Yu Liu. Coco-funit: Few- shot unsupervised image translation with a content conditioned style encoder.arXiv preprint arXiv:2007.07431, 2020. Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical net- works for few-shot learning. InAdv. Neural Inform. Process. Syst., 2017. Ngoc-Trung Tran, Tuan-Anh Bui, and Ngai-Man Cheung. Dist-gan: An improved gan using distance constraints. In Proceedings of the European Conference on Computer Vision (ECCV), 2018. Dmitry Ulyanov, Vadim Lebedev, Andrea Vedaldi, and Victor S Lempitsky. Texture networks Feed-forward synthesis of textures and stylized images. InICML, vol. 1, p. 4, 2016. Arash Vahdat and Jan Kautz. Nvae: A deep hierarchical variational autoencoder. In Neural Information ProcessingSystems (NeurIPS), 2020. Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Daan Wierstra, et al. Matching networks for one shot learning. In Adv. Neural Inform. Process. Syst., 2016. Xiaogang Wang and Xiaoou Tang. Face photo-sketch synthesis and recognition. IEEE Trans. Pattern Anal. Mach. Intell., 31(11):1055- 1967, 2009. Yaxing Wang, Abel Gonzalez-Garcia, David Berga, Luis Herranz, Fahad Shahbaz Khan, and Joost van de Weijer. Minegan: effective knowledge transfer from gans to target domains with few images. In IEEE Conf. Comput. Vis. Pattern Recog., 2020. Yaxing Wang, Salman Khan, Abel Gonzalez-Garcia, Joost van de Weijer, and Fahad Shahbaz Khan. Semi-supervised learning for few-shot image-to-image translation. In IEEE Conf. Comput. Vis. Pattern Recog., 2020. Yaxing Wang, Chenshen Wu, Luis Herranz, Joost van de Weijer, Abel Gonzalez-Garcia, and Bogdan Raducanu. Transferring gans: generating images from limited data. In Eur. Conf. Comput. Vis., 2018. Dingdong Yang, Seunghoon Hong, Yunseok Jang, Tianchen Zhao, and Honglak Lee. Diversity-sensitive conditional generative adversarial networks. arXiv preprint arXiv:1901.09024, 2019. Jordan Yaniv, Yael Newman, and Ariel Shamir. The face of art: landmark detection and geometric style in portraits. ACM Trans- actions on Graphics (TOG), 38(4):1-15, 2019. Fisher Yu, Yinda Zhang, Shuran Song, Ari Seff, and Jianxiong Xiao. Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv:1506.03365, 2015. Shengyu Zhao, Zhijian Liu, Ji Lin, Jun-Yan Zhu, and Song Han. Differentiable augmentation for data-efficient gan training. arXiv preprint arXiv:2006.10738, 2020. Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Int. Conf. Comput. Vis., 2017. Jun-Yan Zhu, Richard Zhang, Deepak Pathak, Trevor Darrell,
AlexeiAEfros, Oliver Wang, and Eli Shechtman. Toward multimodal image-to-image translation. In Adv. Neural Inform. Process. Syst., 2017. L. A. Gatys, A. S. Ecker, and M. Bethge. Image style transfer using convolutional neural networks. In CVPR, 2016. I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde- Farley, S. Ozair, A. Courville, and Y. Bengio. Generative adversarial nets. In NIPS, 2014.