Large Language Models
263 views
Positional Encoding
Quick Definition
Incorporating token position information into transformers
Full Definition
Techniques incorporating token position information into transformer models.
Examples
sinusoidal encoding, RoPE, ALiBi, learned embeddings
Related Terms
transformer
attention-mechanism
More Large Language Models Terms
Mistral
Efficient open-source LLMs with strong performance
Large Language Model
Massive AI model trained on text for language understanding and generation
Training LLM
Adjusting parameters on large datasets to learn language
System Prompt
Special prompt defining AI assistant behavior and constraints
Model Merging
Combining fine-tuned LLMs without additional training
Embeddings LLM
Dense vectors capturing semantic meaning of text