Large Language Models
24 views
RoPE
Quick Definition
Position encoding using rotation matrices for relative positions
Full Definition
Rotary Position Embedding encoding relative positions using rotation matrices.
Examples
Llama, Mistral, length generalization
Related Terms
positional-encoding
transformer
More Large Language Models Terms
Vector Database
Specialized database for high-dimensional embedding queries
Multi-Head Attention
Parallel attention operations concatenated for richer representations
Temperature
Parameter controlling randomness in text generation
System Prompt
Special prompt defining AI assistant behavior and constraints
Tokenization
Breaking text into tokens for language model processing
Prompt Template
Reusable structured format with dynamic input placeholders