Large Language Models
138 views
Masked Language Model
Quick Definition
Training objective predicting randomly masked input tokens
Full Definition
A training objective where masked tokens are predicted from surrounding context.
Examples
BERT pre-training, fill-in-the-blank, contextual understanding
Related Terms
bert
pre-training
self-supervised-learning
More Large Language Models Terms
Scaling Laws
Relationships between model size, data, compute, and performance
Pre-training
Initial training learning general language patterns from text
Training LLM
Adjusting parameters on large datasets to learn language
GGUF
File format for LLMs optimized for local inference
Function Calling
Structured mechanism for LLMs to generate API function calls
WordPiece
Subword tokenization splitting words into frequent units