Large Language Models
164 views
Streaming
Quick Definition
Sending LLM tokens to clients as they are generated
Full Definition
Sending LLM tokens to clients as they are generated for real-time responsiveness.
Examples
chat interfaces, real-time responses, SSE
Related Terms
inference-optimization
user-experience
More Large Language Models Terms
Reward Model
Model scoring LLM outputs based on human preferences
QLoRA
Combining 4-bit quantization with LoRA for efficiency
Temperature
Parameter controlling randomness in text generation
Scaling Laws
Relationships between model size, data, compute, and performance
GGUF
File format for LLMs optimized for local inference
DPO
Alignment method bypassing reward model for preference learning