Bridging the Gap in Vision Language Models in Identifying Unsafe Concepts Across Modalities

This is the repository for the paper "Bridging the Gap in Vision Language Models in Identifying Unsafe Concepts Across Modalities," where we promise to release the UnsafeConcepts dataset, finetuned LLaVA checkpoints, and the implementation code.

Disclaimer. This repo contains examples of unsafe or hateful images. Reader discretion is recommended. This repo is intended for research purposes only. Any misuse is strictly prohibited.

Overview

This repo includes:

• Access to the UnsafeConcepts dataset, a manually annotated image dataset containing 1.5K examples;
• Access to the VLM-generated responses, and response classifiers
• Access to the fine-tuned checkpoints using SFT, DPO, and PPO, and their training scripts
• Scripts for reproducing key results in the paper

Environment Setup

cd SaferVLM

bash setup.sh

conda activate llava

UnsafeConcepts Dataset

We have taken great care to share our dataset responsibly due to the presence of unsafe images. Therefore, this dataset has restricted access and is available upon requests for research purposes only.

First request the access, then download the dataset from huggingface "yiting/UnsafeConcepts"

from huggingface_hub import snapshot_download

local_dir = snapshot_download(
    repo_id="yiting/UnsafeConcepts",
    repo_type="dataset",
    local_dir="data/images")

Reproduce Measurement Results

Download VLM-generated responses here.

Download response classifier:

from huggingface_hub import snapshot_download

local_dir = snapshot_download(
    repo_id="yiting/perception_classifier",
    repo_type="model",
    local_dir="checkpoints/perception_classifier")

local_dir = snapshot_download(
    repo_id="yiting/alignment_classifier",
    repo_type="model",
    local_dir="checkpoints/alignment_classifier")

python measure.py --measure_mode perception --response_dir data/VLM_responses

python measure.py --measure_mode alignment --response_dir data/VLM_responses

python measure.py --measure_mode alignment_text_only --response_dir data/VLM_responses

RLHF

Download and save the fined-tuned checkpoints at checkpoints/rlhf/*

from huggingface_hub import snapshot_download

local_dir = snapshot_download(
    repo_id="yiting/llava-lora",
    repo_type="model",
    local_dir="checkpoints/rlhf")

Evaluate their performance with different datasets, e.g., UnsafeConcepts_TEST

run

python eval_rlhf.py --dataset_name UnsafeConcepts_TEST --lora_path checkpoints/rlhf/sft --save_dir results/sft

python eval_rlhf.py --dataset_name UnsafeConcepts_TEST --lora_path checkpoints/rlhf/dpo --save_dir results/dpo

python eval_rlhf.py --dataset_name UnsafeConcepts_TEST --lora_path checkpoints/rlhf/ppo --save_dir results/ppo

To train these models, run

cd RLHF

bash scripts/train_sft.sh

bash scripts/train_dpo.sh

bash scripts/train_ppo.sh