Large Language Models
157 views
Sliding Window Attention
Quick Definition
Attention limiting tokens to nearby window only
Full Definition
Attention limiting each token to attend to a fixed-size local window of nearby tokens.
Examples
efficient attention, long sequences, Mistral architecture
Related Terms
attention-mechanism
sparse-attention
More Large Language Models Terms
System Prompt
Special prompt defining AI assistant behavior and constraints
Quantization LLM
Reducing LLM weight precision for efficient inference
Semantic Search
Search technique understanding meaning rather than keywords
Structured Output
Generating LLM responses in predefined machine-readable formats
KV Cache
Memory optimization storing key-values for faster generation
LoRA
Parameter-efficient fine-tuning using low-rank matrix decomposition