Published May 14, 2025 | Version v1
Software Open

CLIP benchmark

  • 1. ROR icon Forschungszentrum Jülich
  • 2. LAION
  • 1. ROR icon University of Washington

Description

The goal of CLIP_benchmark is to evaluate CLIP-like models on a standard set of datasets on different tasks such as zero-shot classification and zero-shot retrieval, and captioning.

Features

  • Support for zero-shot classification and zero-shot retrieval, linear probing, and captioning.
  • Support for OpenCLIP pre-trained models, Japanese CLIP, and NLLB CLIP for general multilingual abilities.
  • Support various datasets from torchvision, tensorflow datasets, and VTAB.
  • Support for various multilingual datasets for classification and retrieval
  • Support for compositionality tasks

 

Files

CLIP_benchmark-main.zip

Files (3.8 MB)

Name Size Download all
md5:eb483480d014adad257da4e286329dfe
3.8 MB Preview Download

Additional details

Software

Repository URL
https://github.com/LAION-AI/CLIP_benchmark
Programming language
Python
Development Status
Active