Large Language Models
276 views
BERT
Quick Definition
Google's bidirectional transformer for context understanding
Full Definition
Bidirectional Encoder Representations from Transformers, a Google model processing text in both directions.
Examples
search ranking, question answering, sentiment analysis
Related Terms
transformer
encoder-only
More Large Language Models Terms
Adapter Layer
Small trainable modules in frozen transformer layers
Masked Language Model
Training objective predicting randomly masked input tokens
WordPiece
Subword tokenization splitting words into frequent units
Reward Model
Model scoring LLM outputs based on human preferences
Speculative Decoding
Using draft model candidates verified by large model
QLoRA
Combining 4-bit quantization with LoRA for efficiency