Large Language Models
302 views
Streaming
Quick Definition
Sending LLM tokens to clients as they are generated
Full Definition
Sending LLM tokens to clients as they are generated for real-time responsiveness.
Examples
chat interfaces, real-time responses, SSE
Related Terms
inference-optimization
user-experience
More Large Language Models Terms
QLoRA
Combining 4-bit quantization with LoRA for efficiency
BPE
Subword tokenization algorithm merging frequent character pairs
Context Window
Maximum tokens a language model can process at once
Grounding LLM
Connecting LLM outputs to verifiable external sources
Autoregressive Model
Generative model predicting next token from previous tokens
KV Cache
Memory optimization storing key-values for faster generation