LoRA
Quick Definition
Parameter-efficient fine-tuning using low-rank matrix decomposition
Full Definition
Low-Rank Adaptation, a parameter-efficient fine-tuning method injecting low-rank matrices into transformer layers.
Examples
efficient fine-tuning, resource-constrained training
Related Terms
fine-tuning
qlora
adapter
More Large Language Models Terms
Next Token Prediction
Core LLM objective predicting the next token from context
Tokenization
Breaking text into tokens for language model processing
Prompt Template
Reusable structured format with dynamic input placeholders
Training LLM
Adjusting parameters on large datasets to learn language
Masked Language Model
Training objective predicting randomly masked input tokens
Function Calling
Structured mechanism for LLMs to generate API function calls