Large Language Models
30 views
KV Cache
Quick Definition
Memory optimization storing key-values for faster generation
Full Definition
A memory optimization storing computed key-value pairs to avoid redundant attention computation.
Examples
inference optimization, generation speedup, memory management
Related Terms
attention-mechanism
inference-optimization
More Large Language Models Terms
GGUF
File format for LLMs optimized for local inference
Nucleus Sampling
Generation using cumulative probability threshold for token selection
Constitutional AI
Training AI to follow explicit principles for consistency
Distillation LLM
Training smaller LLMs to replicate larger model behavior
BERT
Google's bidirectional transformer for context understanding
Context Window
Maximum tokens a language model can process at once