On the distribution of penultimate activations of classification networks
- Title
- On the distribution of penultimate activations of classification networks
- Authors
- 곽수하; 서민교; 이윤호
- Date Issued
- 2021-07-28
- Publisher
- The Association for Uncertainty in Artificial Intelligence
- Abstract
- This paper studies probability distributions of penultimate activations of classification networks. We show that, when a classification network is trained with the cross-entropy loss, its final classification layer forms a Generative-Discriminative pair with a generative classifier based on a specific distribution of penultimate activations. More importantly, the distribution is parameterized by the weights of the final fully-connected layer, and can be considered as a generative model that synthesizes the penultimate activations without feeding input data. We empirically demonstrate that this generative model enables stable knowledge distillation in the presence of domain shift, and can transfer knowledge from a classifier to variational autoencoders and generative adversarial networks for class-conditional image generation.
- URI
- https://oasis.postech.ac.kr/handle/2014.oak/109903
- Article Type
- Conference
- Citation
- The Conference on Uncertainty in Artificial Intelligence, page. 1141 - 1151, 2021-07-28
- Files in This Item:
- There are no files associated with this item.
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.