Henry AI Labs
This video explores a new GAN model for generating images by conditioning them on features from pre-trained image classifiers! This is really interesting for visualizing what is contained in pre-trained image classifiers as well as controllable image editing. The authors also show that this can be used for semantic image composition such as copying a tree and pasting it into a snow landscape or image relabeling by changing the embedded logit from the pre-trained classifier to produce an image of a new class while retaining as much of the original image as possible.
Thanks for watching! Please Subscribe!
Paper Links:
Semantic Pyramid for Image Generation: https://arxiv.org/abs/2003.06221
Corresponding Github Page: https://semantic-pyramid.github.io/supmat.html
Neural Style Transfer: https://arxiv.org/pdf/1508.06576.pdf
Zoom In: An Introduction to Circuits: https://distill.pub/2020/circuits/zoom-in/
EfficientDet: https://arxiv.org/pdf/1911.09070.pdf
Generative Teaching Networks: https://arxiv.org/pdf/1912.07768.pdf
DermGAN: https://arxiv.org/pdf/1911.08716.pdf
Classification Accuracy Score for Conditional Generative Models: https://arxiv.org/pdf/1905.10887.pdf
GauGAN: https://arxiv.org/pdf/1903.07291.pdf
SinGAN: https://arxiv.org/pdf/1905.01164.pdf
StyleGAN2 Distillation: https://arxiv.org/pdf/2003.03581.pdf
Semi-Supervised StyleGAN for Disentanglement Learning: https://arxiv.org/pdf/2003.03461.pdf
Thanks for watching! Please Subscribe!
1:18 Intro
1:45 Model Architecture
4:02 Animated Example of Pre-Trained Features
6:15 Applications of the Semantic Pyramid
6:42 Image Re-Painting
7:30 Inputs outside of Natural Image Distribution
8:25 Semantic Image Composition
9:08 Image Re-Labeling
9:52 Optimization Loss
11:00 Hierarchical Image Features
11:28 Feature Visualization with Realism Loss
12:50 Data Augmentation with GAN-Generated Data
It's mind-boggling how fast research is moving here. Thanks for sharing this paper! Really appreciate your dedication for all these videos
Thank you for your sharing! I have one question about the method: do you have an idea how the spatial varying masks are made during training? I believe spatial masks need some constraints like: "it should not be sparse" or "it's shape should not be too complicated". But i couldn't find implemenation details about spatial masks in the paper.
Thank you! I really appreciate the effort made in making theses videos!
keep it up 🙂
Thank you! Very Interesting paper!
What is Realism Loss" 11:28? How formula of loss looks like