Context Window
Quick Definition
Maximum tokens a language model can process at once
Full Definition
The maximum number of tokens a language model can process in a single input.
Examples
long document processing, conversation history, retrieval context
Related Terms
large-language-model
attention
More Large Language Models Terms
RoPE
Position encoding using rotation matrices for relative positions
Adapter Layer
Small trainable modules in frozen transformer layers
Jailbreaking
Circumventing AI safety restrictions through prompt techniques
Mixture of Experts
Architecture routing tokens to specialized expert sub-networks
Autoregressive Model
Generative model predicting next token from previous tokens
Llama
Meta AI's open-source large language models