Artificial Intelligence
304 views
Self-Attention
Quick Definition
Mechanism allowing sequence positions to attend to all others
Full Definition
A mechanism allowing each position in a sequence to attend to all positions in the same sequence.
Examples
transformer layers, contextual embeddings, parallel processing
Related Terms
transformer
attention-mechanism
More Artificial Intelligence Terms
Synthetic Data
Artificially generated data mimicking real-world distributions
One-Shot Learning
Learning to recognize patterns from a single example
Neural Network
Computing system inspired by brain neurons
Recurrent Neural Network
Neural network with cyclic connections for sequence processing
AI Chip
Specialized hardware for accelerating AI computation
Named Entity Recognition
Identifying and classifying named entities in text