Artificial Intelligence
25 views
Self-Attention
Quick Definition
Mechanism allowing sequence positions to attend to all others
Full Definition
A mechanism allowing each position in a sequence to attend to all positions in the same sequence.
Examples
transformer layers, contextual embeddings, parallel processing
Related Terms
transformer
attention-mechanism
More Artificial Intelligence Terms
AI Safety
Research ensuring AI is developed safely without harm
Multi-Agent System
System of interacting autonomous agents solving complex problems
Meta-Learning
Designing models that learn how to learn new tasks
AutoML
Automation of the machine learning pipeline
Retrieval-Augmented Generation
Combining LLMs with external knowledge for accurate responses
Adversarial Attack
Crafting inputs that fool AI models into wrong predictions