Large Language Models
19 views
Masked Language Model
Quick Definition
Training objective predicting randomly masked input tokens
Full Definition
A training objective where masked tokens are predicted from surrounding context.
Examples
BERT pre-training, fill-in-the-blank, contextual understanding
Related Terms
bert
pre-training
self-supervised-learning
More Large Language Models Terms
Flash Attention
Memory-efficient attention using GPU SRAM block computation
Next Token Prediction
Core LLM objective predicting the next token from context
Tool Use
Capability of LLMs to call external tools and APIs
System Prompt
Special prompt defining AI assistant behavior and constraints
Jailbreaking
Circumventing AI safety restrictions through prompt techniques
Autoregressive Model
Generative model predicting next token from previous tokens