Large Language Models
20 views
Positional Encoding
Quick Definition
Incorporating token position information into transformers
Full Definition
Techniques incorporating token position information into transformer models.
Examples
sinusoidal encoding, RoPE, ALiBi, learned embeddings
Related Terms
transformer
attention-mechanism
More Large Language Models Terms
Grounding LLM
Connecting LLM outputs to verifiable external sources
DPO
Alignment method bypassing reward model for preference learning
WordPiece
Subword tokenization splitting words into frequent units
Top-K Sampling
Generation strategy considering top k most probable tokens
Large Language Model
Massive AI model trained on text for language understanding and generation
Tokenizer Vocabulary
Complete set of tokens a language model can process