Large Language Models
20 views
Scaling Laws
Quick Definition
Relationships between model size, data, compute, and performance
Full Definition
Empirical relationships between model size, dataset size, compute, and performance.
Examples
Chinchilla, compute-optimal training, model sizing
Related Terms
large-language-model
training
More Large Language Models Terms
Tool Use
Capability of LLMs to call external tools and APIs
Nucleus Sampling
Generation using cumulative probability threshold for token selection
QLoRA
Combining 4-bit quantization with LoRA for efficiency
Llama
Meta AI's open-source large language models
DPO
Alignment method bypassing reward model for preference learning
Semantic Search
Search technique understanding meaning rather than keywords