BioCLIP

Images of the natural world are an abundant source of biological information. There are many computational methods and tools, particularly computer vision, for extracting information from images. However, existing methods consist of bespoke models, not adaptable or extendable from their targeted task to new questions, contexts, and datasets. We thus develop the first large-scale multimodal model, BioCLIP, for general biology questions on images. We leverage the unique properties of biology (abundance and variety of images and availability of rich structured biological knowledge) as the the application domain for computer vision.