Large Language Models
23 views
Next Token Prediction
Quick Definition
Core LLM objective predicting the next token from context
Full Definition
The core training objective of autoregressive LLMs predicting the most probable next token given previous context.
Examples
GPT text generation, autocomplete, language modeling
Related Terms
autoregressive-model
large-language-model
pre-training
More Large Language Models Terms
Sparse Attention
Attention attending to only a subset of tokens
Instruction Tuning
Fine-tuning on instruction-response pairs for better following
GGUF
File format for LLMs optimized for local inference
Chain-of-Thought Prompting
Technique encouraging step-by-step reasoning in LLMs
Large Language Model
Massive AI model trained on text for language understanding and generation
Tokenizer Vocabulary
Complete set of tokens a language model can process