Artificial Intelligence
277 views
Model Distillation
Quick Definition
Creating smaller models that replicate larger model behavior
Full Definition
Creating smaller models that replicate the behavior of larger teacher models for efficient deployment.
Examples
TinyBERT, mobile models, edge deployment
Related Terms
pruning
quantization
model-compression
More Artificial Intelligence Terms
Prompt Injection
Attack manipulating AI behavior by overriding instructions
Self-Attention
Mechanism allowing sequence positions to attend to all others
Variational Autoencoder
Generative model using latent space encoding and decoding
AI Safety
Research ensuring AI is developed safely without harm
Reinforcement Learning
Learning through trial and error with reward feedback
Reward Modeling
Training models to predict human preferences for RLHF