Large Language Models
29 views
Streaming
Quick Definition
Sending LLM tokens to clients as they are generated
Full Definition
Sending LLM tokens to clients as they are generated for real-time responsiveness.
Examples
chat interfaces, real-time responses, SSE
Related Terms
inference-optimization
user-experience
More Large Language Models Terms
Speculative Decoding
Using draft model candidates verified by large model
SentencePiece
Language-agnostic tokenizer operating on raw text bytes
Beam Search
Search algorithm exploring multiple output sequences
Sliding Window Attention
Attention limiting tokens to nearby window only
Context Window
Maximum tokens a language model can process at once
Guardrails LLM
Frameworks for monitoring and controlling LLM I/O