Large Language Models
25 views
Model Merging
Quick Definition
Combining fine-tuned LLMs without additional training
Full Definition
Combining multiple fine-tuned LLMs into a single model without additional training.
Examples
model soup, weight merging, multi-task models
Related Terms
fine-tuning
ensemble
More Large Language Models Terms
Quantization LLM
Reducing LLM weight precision for efficient inference
Autoregressive Model
Generative model predicting next token from previous tokens
Guardrails LLM
Frameworks for monitoring and controlling LLM I/O
Flash Attention
Memory-efficient attention using GPU SRAM block computation
Chain-of-Thought Prompting
Technique encouraging step-by-step reasoning in LLMs
Next Token Prediction
Core LLM objective predicting the next token from context