Large Language Models
28 views
Speculative Decoding
Quick Definition
Using draft model candidates verified by large model
Full Definition
Using a small draft model to generate candidates verified in parallel by the large target model.
Examples
inference acceleration, LLM serving, throughput optimization
Related Terms
inference-optimization
kv-cache
More Large Language Models Terms
Nucleus Sampling
Generation using cumulative probability threshold for token selection
Autoregressive Model
Generative model predicting next token from previous tokens
Function Calling
Structured mechanism for LLMs to generate API function calls
Constitutional AI
Training AI to follow explicit principles for consistency
Tokenization
Breaking text into tokens for language model processing
Adapter Layer
Small trainable modules in frozen transformer layers