Large Language Models
329 views
Alignment Tax
Quick Definition
Performance cost of aligning AI with human preferences
Full Definition
The performance cost incurred when aligning AI models to follow human preferences and safety guidelines.
Examples
capability reduction after RLHF, safety vs helpfulness tradeoff
Related Terms
rlhf
ai-alignment
fine-tuning
More Large Language Models Terms
Top-K Sampling
Generation strategy considering top k most probable tokens
Temperature
Parameter controlling randomness in text generation
Nucleus Sampling
Generation using cumulative probability threshold for token selection
Masked Language Model
Training objective predicting randomly masked input tokens
Inference LLM
Using a trained LLM to generate outputs from inputs
BERT
Google's bidirectional transformer for context understanding