Large Language Models
143 views
Pre-training
Quick Definition
Initial training learning general language patterns from text
Full Definition
Initial training phase where models learn general language patterns from massive text corpora.
Examples
GPT pre-training, BERT training, foundation model creation
Related Terms
self-supervised-learning
fine-tuning
More Large Language Models Terms
Alignment Tax
Performance cost of aligning AI with human preferences
Quantization LLM
Reducing LLM weight precision for efficient inference
Chain-of-Thought Prompting
Technique encouraging step-by-step reasoning in LLMs
Prompt Template
Reusable structured format with dynamic input placeholders
Llama
Meta AI's open-source large language models
Tool Use
Capability of LLMs to call external tools and APIs