Published September 29, 2024 | Version v5
Dataset Open

Recursive InPainting with Stable Difussion 2 Inpainting

  • 1. ROR icon Universidad Politécnica de Madrid
  • 2. ROR icon Carlos III University of Madrid
  • 3. Universidad Antonio Nebrija
  • 4. Universidad de Valladolid

Description

Recursive inpainting with Stable Difussion

The dataset contains images extracted from WikiArt that have been recursively versioned using inpainting techniques with a version of the Stable Diffusion model oriented towards instructional inpainting called stable-diffusion-2-inpainting, used in the paper How Stable is Stable Diffusion under Recursive InPainting (RIP)? Presented at the GenAI Evaluation KDD2024: KDD workshop on Evaluation and Trustworthiness of Generative AI Models

 

The dataset contains two folders with two different experiments. Each experiment consists of performing recursive inpainting on 512x512 px images until reaching a percentage of pixels affected by the inpainting of 400% compared to the original image. Each experiment contains the following files:

  • original_imgs_grid.png* file with all the base images in an ordered grid or dataset folder with the original images.
  • mask_64x64/[number_figure]_64x64.png* with the inpainting version of [number_figure] and 64x64 mask
  • mask_128x128/[number_figure]_128x128.png* with the inpainting version of [number_figure] and 128x128 mask
  • mask_256x256/[number_figure]_256x256.png* with the inpainting version of [number_figure] and 256x256 mask
  • lpips_distances.xlsx with the results of applying the Learned Perceptual Image Path Similarity (LPIPS) metric in its variants SqueezeNet, AlexNet, and VGG. The metrics are calculated with respect to the original image and the previous image.

* Only images that license allow to share

There are three different experiments:

  • lpips_distances_100_images_1_seed: contains recursive inpaintings of 100 different images with a single seed. This allows the analysis of different images when performing recursive inpainting.
  • lpips_distances_10_images_15_seeds: contains recursive inpaintings of 10 different images with 15 different seeds. This allows the analysis of the differences in inpaintings when applying different seeds in the inpainting on the same image.
  • lpisp_distances_styles: contains recursive inpaintings of 80 images taken from seven pictorial styles and from sketches of an architect in subsets to 10 images.
    • Styles:
    • Mods folder: contains the same inpaintings but for modifications of the base image:
      • Black_White
      • Blue_Green: version with no red components.
      • Red_Blue: version with no green components.
      • Red_Green: version with no blue components.

The folder code includes the a README.md with the instructions and the scripts to run the experiments

Cite our work in:

@misc{conde2024stablestablediffusionrecursive,
      title={How Stable is Stable Diffusion under Recursive InPainting (RIP)?}, 
      author={Javier Conde and Miguel González and Gonzalo Martínez and Fernando Moral and Elena Merino-Gómez and Pedro Reviriego},
      year={2024},
      eprint={2407.09549},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2407.09549}, 
}

Files

lpisp_distances_100_images_1_seeds.zip

Files (1.9 GB)

Name Size Download all
md5:2c08c2de1245e82e47d1c85a516f5031
10.5 kB Preview Download
md5:9d3234dbdcd28f2a0f55aa3a4123b9cd
780.4 MB Preview Download
md5:5716fccd0bb26911a9db226e58a8dbf5
1.1 GB Preview Download
md5:da6e0b06ed320a63d1b33f7e604f9a52
1.2 MB Preview Download