Large Language Models
322 views
Tokenization
Quick Definition
Breaking text into tokens for language model processing
Full Definition
Breaking text into smaller units called tokens that language models can process.
Examples
BPE, SentencePiece, word embeddings, vocabulary building
Related Terms
large-language-model
embedding
bpe
More Large Language Models Terms
Linear Attention
Attention with linear complexity for long sequences
Alignment Tax
Performance cost of aligning AI with human preferences
RoPE
Position encoding using rotation matrices for relative positions
Llama
Meta AI's open-source large language models
RLHF
Aligning LLMs with human preferences using reinforcement learning
Flash Attention
Memory-efficient attention using GPU SRAM block computation