Large Language Models
282 views
Quantization LLM
Quick Definition
Reducing LLM weight precision for efficient inference
Full Definition
Reducing LLM weight precision to lower bits like INT8 or INT4 for efficient inference.
Examples
4-bit inference, GPTQ, AWQ, model deployment
Related Terms
quantization
inference-optimization
More Large Language Models Terms
Jailbreaking
Circumventing AI safety restrictions through prompt techniques
Temperature
Parameter controlling randomness in text generation
Flash Attention
Memory-efficient attention using GPU SRAM block computation
Tree-of-Thought
Framework exploring multiple reasoning paths for optimal solutions
Semantic Search
Search technique understanding meaning rather than keywords
System Prompt
Special prompt defining AI assistant behavior and constraints