Large Language Models
304 views
Autoregressive Model
Quick Definition
Generative model predicting next token from previous tokens
Full Definition
A generative model predicting the next token based on all previous tokens in sequence.
Examples
GPT text generation, language modeling, next-token prediction
Related Terms
large-language-model
transformer
More Large Language Models Terms
KV Cache
Memory optimization storing key-values for faster generation
BERT
Google's bidirectional transformer for context understanding
Training LLM
Adjusting parameters on large datasets to learn language
Mistral
Efficient open-source LLMs with strong performance
Distillation LLM
Training smaller LLMs to replicate larger model behavior
Masked Language Model
Training objective predicting randomly masked input tokens