Large Language Models
144 views
Nucleus Sampling
Quick Definition
Generation using cumulative probability threshold for token selection
Full Definition
A generation strategy using cumulative probability threshold p for dynamic token selection.
Examples
natural text generation, conversation, creative content
Related Terms
top-k-sampling
temperature
beam-search
More Large Language Models Terms
Adapter Layer
Small trainable modules in frozen transformer layers
Next Token Prediction
Core LLM objective predicting the next token from context
Streaming
Sending LLM tokens to clients as they are generated
Top-K Sampling
Generation strategy considering top k most probable tokens
Speculative Decoding
Using draft model candidates verified by large model
KV Cache
Memory optimization storing key-values for faster generation