Knowledge Distillation

Self-Distillation Bridges Distribution Gap in Language Model Fine-Tuning

SHOT-VAE: Semi-supervised Deep Generative Models With Label-aware ELBO Approximations

KD3A: Unsupervised Multi-Source Decentralized Domain Adaptation via Knowledge Distillation