CLIP
CLIP trained joint image-text representations.
Why it matters
It linked image and text representations without a task-specific labeled dataset.
The web-derived training data can carry biases and noise into the model.
Research · January 5, 2021
Event record
- Event date
- January 5, 2021
- Timeline date
- Event date
- Verification
- Sources gathered automatically · September 11, 2026
- Lines
- ID
- evt-0015
Sources
- primary CLIP: Connecting text and images
OpenAI · Published January 5, 2021
- primary CLIP
arXiv · Published February 26, 2021
Related events
- Builds on Transformer
CLIP uses a Transformer text encoder, documented in the technical paper included alongside its announcement.
CLIP
Records that link to this one
- Related Muse Image and Muse Video
Both connect text and images: CLIP matches them, whereas Muse Image generates images. No direct model dependency is claimed.
- Related Stable Diffusion
The text condition is supplied through a representation trained to match text and images.
- Builds on LAION-400M: image-caption pairs in the open
CLIP did the selecting: a pair was kept when the cosine between the text and image embeddings was at least 0.3.
LAION-400-MILLION OPEN DATASET - Builds on Flamingo: vision and language from a few examples
Flamingo's vision encoder is pretrained contrastively "a la CLIP", with the two-term contrastive loss from the CLIP paper.
Flamingo: a Visual Language Model for Few-Shot Learning (arXiv:2204.14198v1) - Builds on LLaVA: an open assistant that sees
LLaVA's vision encoder is the pretrained CLIP ViT-L/14, joined to the language model by a single linear projection.
Visual Instruction Tuning (arXiv:2304.08485v1)