knowledge distillation
Technical term in machine learning where a smaller model (student) is trained to mimic a larger model (teacher).
Knowledge distillation is a technique to transfer the knowledge of a large model to a smaller one.
In this paper, they used knowledge distillation to make the model lighter.
蒸留 alone means 'distillation' in a chemical sense; 知識の蒸留 is a specific machine learning term.
モデル圧縮 (model compression) is a broader category that includes knowledge distillation as one method.
Compound of 知識 (knowledge) + の (possessive particle) + 蒸留 (distillation). Calque of English 'knowledge distillation', a term coined in machine learning literature.