Large Language Models
276 views
Masked Language Model
Quick Definition
Training objective predicting randomly masked input tokens
Full Definition
A training objective where masked tokens are predicted from surrounding context.
Examples
BERT pre-training, fill-in-the-blank, contextual understanding
Related Terms
bert
pre-training
self-supervised-learning
More Large Language Models Terms
Reward Model
Model scoring LLM outputs based on human preferences
LoRA
Parameter-efficient fine-tuning using low-rank matrix decomposition
Synthetic Data Generation
Using AI to create artificial training data replicating patterns
Function Calling
Structured mechanism for LLMs to generate API function calls
Scaling Laws
Relationships between model size, data, compute, and performance
Chain-of-Thought Prompting
Technique encouraging step-by-step reasoning in LLMs