Large Language Models
149 views
Quantization LLM
Quick Definition
Reducing LLM weight precision for efficient inference
Full Definition
Reducing LLM weight precision to lower bits like INT8 or INT4 for efficient inference.
Examples
4-bit inference, GPTQ, AWQ, model deployment
Related Terms
quantization
inference-optimization
More Large Language Models Terms
Semantic Search
Search technique understanding meaning rather than keywords
Vector Database
Specialized database for high-dimensional embedding queries
Temperature
Parameter controlling randomness in text generation
Prompt Caching
Storing key-value states for repeated prompt prefixes
System Prompt
Special prompt defining AI assistant behavior and constraints
Fine-Tuning
Further training pre-trained models on specific domain data