Large Language Models
338 views
Speculative Decoding
Quick Definition
Using draft model candidates verified by large model
Full Definition
Using a small draft model to generate candidates verified in parallel by the large target model.
Examples
inference acceleration, LLM serving, throughput optimization
Related Terms
inference-optimization
kv-cache
More Large Language Models Terms
Attention Head
Individual attention mechanism learning specific patterns
Instruction Tuning
Fine-tuning on instruction-response pairs for better following
Tree-of-Thought
Framework exploring multiple reasoning paths for optimal solutions
Quantization LLM
Reducing LLM weight precision for efficient inference
Grounded Generation
Generating outputs faithful to provided source documents
GPT
OpenAI's family of autoregressive large language models