Large Language Models
286 views
Linear Attention
Quick Definition
Attention with linear complexity for long sequences
Full Definition
Attention mechanism with linear complexity for efficient long-sequence processing.
Examples
efficient transformers, long documents, streaming models
Related Terms
attention-mechanism
sparse-attention
More Large Language Models Terms
RLHF
Aligning LLMs with human preferences using reinforcement learning
Jailbreaking
Circumventing AI safety restrictions through prompt techniques
Next Token Prediction
Core LLM objective predicting the next token from context
Llama
Meta AI's open-source large language models
Synthetic Data Generation
Using AI to create artificial training data replicating patterns
Nucleus Sampling
Generation using cumulative probability threshold for token selection