Large Language Models
135 views
Positional Encoding
Quick Definition
Incorporating token position information into transformers
Full Definition
Techniques incorporating token position information into transformer models.
Examples
sinusoidal encoding, RoPE, ALiBi, learned embeddings
Related Terms
transformer
attention-mechanism
More Large Language Models Terms
Semantic Search
Search technique understanding meaning rather than keywords
Prompt Caching
Storing key-value states for repeated prompt prefixes
Jailbreaking
Circumventing AI safety restrictions through prompt techniques
Adapter Layer
Small trainable modules in frozen transformer layers
DPO
Alignment method bypassing reward model for preference learning
Alignment Tax
Performance cost of aligning AI with human preferences