Artificial Intelligence
287 views
Attention Mechanism
Quick Definition
Component allowing models to focus on relevant input parts
Full Definition
A neural network component that allows models to focus on relevant parts of the input when producing output.
Examples
self-attention, cross-attention, multi-head attention
Related Terms
transformer
self-attention
More Artificial Intelligence Terms
Sentiment Analysis
NLP technique determining emotional tone in text
Embedding
Learned vector representations of discrete objects in continuous space
AI Alignment
Ensuring AI systems act in accordance with human values
OCR
Technology converting images of text into machine-readable text
Word Embedding
Dense vector representation capturing word semantic meaning
Model Distillation
Creating smaller models that replicate larger model behavior