Large Language Models
269 views
PEFT
Quick Definition
Fine-tuning methods updating only small parameter subsets
Full Definition
Parameter-Efficient Fine-Tuning, a family of methods updating only a small subset of model parameters.
Examples
LoRA, prefix tuning, adapter layers, efficient adaptation
Related Terms
lora
fine-tuning
qlora
More Large Language Models Terms
Multi-Head Attention
Parallel attention operations concatenated for richer representations
Structured Output
Generating LLM responses in predefined machine-readable formats
Pre-training
Initial training learning general language patterns from text
Inference LLM
Using a trained LLM to generate outputs from inputs
Tokenizer Vocabulary
Complete set of tokens a language model can process
Embeddings LLM
Dense vectors capturing semantic meaning of text