Large Language Models
256 views
BPE
Quick Definition
Subword tokenization algorithm merging frequent character pairs
Full Definition
Byte-Pair Encoding, a subword tokenization algorithm iteratively merging frequent character pairs.
Examples
GPT tokenization, vocabulary building, subword splitting
Related Terms
tokenization
vocabulary
More Large Language Models Terms
Training LLM
Adjusting parameters on large datasets to learn language
Adapter Layer
Small trainable modules in frozen transformer layers
WordPiece
Subword tokenization splitting words into frequent units
Streaming
Sending LLM tokens to clients as they are generated
Multi-Head Attention
Parallel attention operations concatenated for richer representations
Synthetic Data Generation
Using AI to create artificial training data replicating patterns