CLIP-based Few-Shot Multi-Label Classification Methods: A Comparative Study

Categorizing darkweb image content is critical for identifying and averting potential threats. However, this remains a challenge due to the nature of the data, which includes
multiple co-existing domains and intra-class variations. While many methods have been proposed to classify this image content, multi-label multi-class classification remains underexplored. The complexity of darkweb imagery, combined with the need for efficient classification systems, demands innovative approaches that can handle both the technical challenges and the sensitive nature of the content. In this paper, we present a comparative study of few-shot multi-label classification methods using the
multimodal model CLIP. Our research addresses the growing need for robust classification systems that can effectively categorize diverse and complex image content while maintaining high accuracy and computational efficiency. We particularly focus on the challenges of handling multiple labels simultaneously and the scalability of these systems in real-world applications. We analyze and compare four different approaches: CLIP+Label Empower Adapter, CLIP Sigmoid, SIGLIP, and CLIP+ML-Decoder. Our study evaluates these methods based on their precision, recall, and ability to handle increasing class numbers efficiently. Finally, our research contributes to the field by providing detailed insights into the strengths and limitations of each method.