Large Language Models
129 views
BPE
Quick Definition
Subword tokenization algorithm merging frequent character pairs
Full Definition
Byte-Pair Encoding, a subword tokenization algorithm iteratively merging frequent character pairs.
Examples
GPT tokenization, vocabulary building, subword splitting
Related Terms
tokenization
vocabulary
More Large Language Models Terms
Flash Attention
Memory-efficient attention using GPU SRAM block computation
Sliding Window Attention
Attention limiting tokens to nearby window only
BERT
Google's bidirectional transformer for context understanding
SentencePiece
Language-agnostic tokenizer operating on raw text bytes
Next Token Prediction
Core LLM objective predicting the next token from context
GPT
OpenAI's family of autoregressive large language models