PointPrompt: A Visual Prompting Dataset based on the Segment Anything Model

Each folder in 'Prompting data.zip' corresponds to a single category (Bird, Cat, Bus etc), and each of these contain folders corresponding to a single participant (st1, st2 etc). Each participant folder should contain 5 subfolders:

• 'masks' contains the binary masks produced for each image, in the format a_b_mask.png, where 'a' corresponds to the image number (0 to 399) and 'b' indexes through timestamps in the prompting process
• 'points' contains the inclusion and exclusion points formatted as a_green.npy and a_red.npy respectively, where 'a' corresponds to the image number. Each of these files is a list of lists corresponding to the prompted points at each timestep. The outer list is of size (t,), where 't' is the number of timesteps for that image, an each inner list is fo size (n,2), where 'n' is the number of points at a given timestep
• 'scores' contains the scores at each timestep for every image (mIoU)
• 'sorts' contains sorted timestamp indexes, going from max to min based on the score
• 'eachround' indicates which timesteps belong to each of the two rounds (if they exist). Each file contains a list of lenght t (number of timestamps) where values of 0 corresponds to timestamps that belong to the first round and values of 1 correspond to timestamps that belong to the second round

Quick usage:

-To get the best (highes score) mask for a given image : masks[sorts[0]] 
-To get the best set of prompts for that image : green[sorts[0]] and red[sorts[0]]
-To get which round produced the highest score in that image : eachround[sorts[0]]

The codebase associated with this work can be found at this Github.

Please refer to our lab-wide github for more information regarding the code associated with our other papers.