Artificial Intelligence
166 views
Self-Attention
Quick Definition
Mechanism allowing sequence positions to attend to all others
Full Definition
A mechanism allowing each position in a sequence to attend to all positions in the same sequence.
Examples
transformer layers, contextual embeddings, parallel processing
Related Terms
transformer
attention-mechanism
More Artificial Intelligence Terms
Anomaly Detection
Identifying unusual patterns that deviate from expected behavior
Neural Architecture Search
Automating the design of optimal neural network architectures
OCR
Technology converting images of text into machine-readable text
Convolutional Neural Network
Neural network for image and grid-like data processing
In-Context Learning
LLMs learning new tasks from prompt examples without weight updates
Neural Network
Computing system inspired by brain neurons