LoRA
Quick Definition
Parameter-efficient fine-tuning using low-rank matrix decomposition
Full Definition
Low-Rank Adaptation, a parameter-efficient fine-tuning method injecting low-rank matrices into transformer layers.
Examples
efficient fine-tuning, resource-constrained training
Related Terms
fine-tuning
qlora
adapter
More Large Language Models Terms
Beam Search
Search algorithm exploring multiple output sequences
Inference LLM
Using a trained LLM to generate outputs from inputs
Pre-training
Initial training learning general language patterns from text
Multi-Head Attention
Parallel attention operations concatenated for richer representations
Vector Database
Specialized database for high-dimensional embedding queries
Context Window
Maximum tokens a language model can process at once