DistilCLIP

Overview

Dual-encoder CLIP built in pure PyTorch: ViT for images, DistilBERT for text. Trained with contrastive loss so image and caption embeddings land in the same space. Useful for zero-shot classification and text-to-image retrieval without relying on OpenAI weights.

Highlights

• From-scratch dual encoder with a lighter DistilBERT text tower
• Contrastive training on image-caption pairs for cross-modal retrieval
• End-to-end pipeline: preprocess → tokenize → dual forward → cosine similarity

Stack

PyTorch

Model, training loop, inference

ViT

Image encoder (patch sequence + transformer)

DistilBERT

Text encoder for captions

AdamW

Optimizer with cosine LR schedule