LabPics dataset for visual understanding of Medical and Chemistry Labs

LabPics Chemistry Dataset

LabPics Medical Dataset

The dataset contains annotated images for both material and vessels in chemistry labs, medical labs, and any area where liquids and solids are handled within vessels. There are two levels of annotation for each image. One annotation set for vessels and the second for the material phases inside these vessels. Vessels are defined as any container that can carry materials such as Jars, Erlenmayers, Tubes, Funnels, syringes, IV bags, and any other labware or glassware that can contain or carry materials. Material phases are any material contained within or on the vessel. For example, for two-phase separating liquids, each liquid phase is annotated as one instance. If there is foam above the liquid or a chunk of solid inside the liquid, the foam, liquid, and solid will be annotated as different phases. In addition, vessel parts like cork, labels, and valves are annotated as instances. For each instance, there is a list of all the classes it belongs to, and a list of its property. For vessels, the instance classes are the vessel type (Cup, jar, Separatory-funnel…) and the vessel properties (Transparent, Opaque…). For materials, the classes are the material types ( Liquid, solid, suspension, foam, powder…) and their properties (Scattered, On vessel surface…), and for parts, the part type (cork/label). In addition, the relations between instances are annotated. This includes which materials instances are inside which vessels, which vessels are linked to each other or are inside each other (for vessels inside other vessels), and which material phase is immersed inside another material phase. In addition to instance segmentation maps, the dataset also includes semantic segmentation maps that give for each pixel in the image all the classes to which it belongs. In other words, for each class (Liquid, Solid, Vessel, Foam), there is a map of all the regions in the image belonging to this class. Note that every pixel and every instance can have several classes. In addition, instances often overlap, like in the case of material inside the vessel, vessel inside the vessel, and material phase immerse inside other material (like solid inside liquid).

LabPics 1: Older version of LabPics with 2K images (Much easier to use) can be found in this URL: https://zenodo.org/record/3697452

**New dataset with 50k simulated (CGI) images of materials, liquid, and objects inside transparent vessels, with 2D and 3D annotation can be downloaded from this url:

https://zenodo.org/record/5508261#.YVN_y3tE1H7

File sources and copyright

Creating this dataset was impossible without a community of chemists who take and share high-quality photos of their experiments. Most of the images of this dataset were shared by these sources. Copyright for all images belongs to their contributors. The dataset is shared for academic purposes only. For any other use of the images, inquire with the image source. The source of every image is mention in the source.txt file located next to the image file. Specific copyright for each image also appears in the same file. If no copyright is mentioned, contact the image source for inquiries. See Image sources and copyright file in main folder and source.txt in image folder for detail.

"This research was developed with funding from the Defense Advanced Research Projects Agency (DARPA). The views, opinions and/or findings expressed are those of the author and should not be interpreted as representing the official views or policies of the Department of Defense or the U.S. Government."

grant# HR00111920027