Large Language Models
21 views
Inference LLM
Quick Definition
Using a trained LLM to generate outputs from inputs
Full Definition
The process of using a trained LLM to generate outputs from new inputs.
Examples
text generation, chat responses, content creation
Related Terms
training
inference-optimization
More Large Language Models Terms
BPE
Subword tokenization algorithm merging frequent character pairs
Nucleus Sampling
Generation using cumulative probability threshold for token selection
Chain-of-Thought Prompting
Technique encouraging step-by-step reasoning in LLMs
PEFT
Fine-tuning methods updating only small parameter subsets
QLoRA
Combining 4-bit quantization with LoRA for efficiency
RoPE
Position encoding using rotation matrices for relative positions