Large Language Models
26 views
Sparse Attention
Quick Definition
Attention attending to only a subset of tokens
Full Definition
Attention where each token attends to only a subset of other tokens for efficiency.
Examples
Longformer, BigBird, efficient transformers
Related Terms
attention-mechanism
sliding-window-attention
More Large Language Models Terms
Speculative Decoding
Using draft model candidates verified by large model
Masked Language Model
Training objective predicting randomly masked input tokens
SentencePiece
Language-agnostic tokenizer operating on raw text bytes
Distillation LLM
Training smaller LLMs to replicate larger model behavior
Guardrails LLM
Frameworks for monitoring and controlling LLM I/O
Streaming
Sending LLM tokens to clients as they are generated