Large Language Models
22 views
WordPiece
Quick Definition
Subword tokenization splitting words into frequent units
Full Definition
A subword tokenization algorithm that splits words into frequent subword units based on likelihood.
Examples
BERT tokenization, vocabulary building, unknown word handling
Related Terms
tokenization
bpe
sentencepiece
More Large Language Models Terms
RoPE
Position encoding using rotation matrices for relative positions
Pre-training
Initial training learning general language patterns from text
GPT
OpenAI's family of autoregressive large language models
Function Calling
Structured mechanism for LLMs to generate API function calls
Hallucination LLM
LLMs generating plausible but incorrect information
RAG
Enhancing LLMs by retrieving relevant external knowledge