Large Language Models
283 views
Multi-Head Attention
Quick Definition
Parallel attention operations concatenated for richer representations
Full Definition
A transformer mechanism running multiple attention operations in parallel and concatenating their outputs.
Examples
transformer layers, contextual learning, multi-perspective analysis
Related Terms
attention-mechanism
self-attention
transformer
More Large Language Models Terms
Embeddings LLM
Dense vectors capturing semantic meaning of text
Tokenizer Vocabulary
Complete set of tokens a language model can process
Grounding LLM
Connecting LLM outputs to verifiable external sources
RoPE
Position encoding using rotation matrices for relative positions
Distillation LLM
Training smaller LLMs to replicate larger model behavior
System Prompt
Special prompt defining AI assistant behavior and constraints