Large Language Models
135 views
PEFT
Quick Definition
Fine-tuning methods updating only small parameter subsets
Full Definition
Parameter-Efficient Fine-Tuning, a family of methods updating only a small subset of model parameters.
Examples
LoRA, prefix tuning, adapter layers, efficient adaptation
Related Terms
lora
fine-tuning
qlora
More Large Language Models Terms
Distillation LLM
Training smaller LLMs to replicate larger model behavior
Tool Use
Capability of LLMs to call external tools and APIs
Grounded Generation
Generating outputs faithful to provided source documents
Tokenizer Vocabulary
Complete set of tokens a language model can process
Sliding Window Attention
Attention limiting tokens to nearby window only
Vector Database
Specialized database for high-dimensional embedding queries