Large Language Models
131 views
Next Token Prediction
Quick Definition
Core LLM objective predicting the next token from context
Full Definition
The core training objective of autoregressive LLMs predicting the most probable next token given previous context.
Examples
GPT text generation, autocomplete, language modeling
Related Terms
autoregressive-model
large-language-model
pre-training
More Large Language Models Terms
Autoregressive Model
Generative model predicting next token from previous tokens
Tree-of-Thought
Framework exploring multiple reasoning paths for optimal solutions
Fine-Tuning
Further training pre-trained models on specific domain data
Instruction Tuning
Fine-tuning on instruction-response pairs for better following
Model Merging
Combining fine-tuned LLMs without additional training
Alignment Tax
Performance cost of aligning AI with human preferences