Large Language Models
353 views
Scaling Laws
Quick Definition
Relationships between model size, data, compute, and performance
Full Definition
Empirical relationships between model size, dataset size, compute, and performance.
Examples
Chinchilla, compute-optimal training, model sizing
Related Terms
large-language-model
training
More Large Language Models Terms
Speculative Decoding
Using draft model candidates verified by large model
Autoregressive Model
Generative model predicting next token from previous tokens
Llama
Meta AI's open-source large language models
GGUF
File format for LLMs optimized for local inference
Hallucination LLM
LLMs generating plausible but incorrect information
Fine-Tuning
Further training pre-trained models on specific domain data