Large Language Models
141 views
KV Cache
Quick Definition
Memory optimization storing key-values for faster generation
Full Definition
A memory optimization storing computed key-value pairs to avoid redundant attention computation.
Examples
inference optimization, generation speedup, memory management
Related Terms
attention-mechanism
inference-optimization
More Large Language Models Terms
Nucleus Sampling
Generation using cumulative probability threshold for token selection
Reward Model
Model scoring LLM outputs based on human preferences
GPT
OpenAI's family of autoregressive large language models
Function Calling
Structured mechanism for LLMs to generate API function calls
BPE
Subword tokenization algorithm merging frequent character pairs
Adapter Layer
Small trainable modules in frozen transformer layers