Large Language Models
23 views
PEFT
Quick Definition
Fine-tuning methods updating only small parameter subsets
Full Definition
Parameter-Efficient Fine-Tuning, a family of methods updating only a small subset of model parameters.
Examples
LoRA, prefix tuning, adapter layers, efficient adaptation
Related Terms
lora
fine-tuning
qlora
More Large Language Models Terms
Adapter Layer
Small trainable modules in frozen transformer layers
Function Calling
Structured mechanism for LLMs to generate API function calls
Scaling Laws
Relationships between model size, data, compute, and performance
Masked Language Model
Training objective predicting randomly masked input tokens
Top-K Sampling
Generation strategy considering top k most probable tokens
LoRA
Parameter-efficient fine-tuning using low-rank matrix decomposition