Large Language Models
186 views
Scaling Laws
Quick Definition
Relationships between model size, data, compute, and performance
Full Definition
Empirical relationships between model size, dataset size, compute, and performance.
Examples
Chinchilla, compute-optimal training, model sizing
Related Terms
large-language-model
training
More Large Language Models Terms
Guardrails LLM
Frameworks for monitoring and controlling LLM I/O
WordPiece
Subword tokenization splitting words into frequent units
QLoRA
Combining 4-bit quantization with LoRA for efficiency
Streaming
Sending LLM tokens to clients as they are generated
SentencePiece
Language-agnostic tokenizer operating on raw text bytes
Pre-training
Initial training learning general language patterns from text