Large Language Models
27 views
Pre-training
Quick Definition
Initial training learning general language patterns from text
Full Definition
Initial training phase where models learn general language patterns from massive text corpora.
Examples
GPT pre-training, BERT training, foundation model creation
Related Terms
self-supervised-learning
fine-tuning
More Large Language Models Terms
Speculative Decoding
Using draft model candidates verified by large model
Hallucination LLM
LLMs generating plausible but incorrect information
Tree-of-Thought
Framework exploring multiple reasoning paths for optimal solutions
Tool Use
Capability of LLMs to call external tools and APIs
Mistral
Efficient open-source LLMs with strong performance
Constitutional AI
Training AI to follow explicit principles for consistency