LoRA
Quick Definition
Parameter-efficient fine-tuning using low-rank matrix decomposition
Full Definition
Low-Rank Adaptation, a parameter-efficient fine-tuning method injecting low-rank matrices into transformer layers.
Examples
efficient fine-tuning, resource-constrained training
Related Terms
fine-tuning
qlora
adapter
More Large Language Models Terms
Prompt Caching
Storing key-value states for repeated prompt prefixes
SentencePiece
Language-agnostic tokenizer operating on raw text bytes
Pre-training
Initial training learning general language patterns from text
Embeddings LLM
Dense vectors capturing semantic meaning of text
Top-K Sampling
Generation strategy considering top k most probable tokens
WordPiece
Subword tokenization splitting words into frequent units