chenruiRae/GalaxySD: v1.0.1

Chenrui Ma

doi:10.5281/zenodo.17112014

Published September 13, 2025 | Version v1.0.1

Software Open

chenruiRae/GalaxySD: v1.0.1

Chenrui Ma¹

1. Tsinghua University

Contributors

Project manager:

Ma, Chenrui

Project members:

source code of the paper: Can AI Dream of Unseen Galaxies? Conditional Diffusion Model for Galaxy Morphology Augmentation.

🌌 GalaxySD

We fine-tuned sd-1.5 specialized for galaxy image generation by galaxy images with annoted morphological description based on GZ2. The galaxy morphological description dataset in natural language insteal of vote fractions will release soon. Before all experiments, you need to unzip the zipped folder and then follow the below instructions.

Our project HOMEPAGE.

🧠 Arcitecture

Schematic diagram of our model and downstream tasks in our paper. Please see the schematic diagram in our homepage or paper.

🛠️ Git and create environment

git clone https://github.com/chenruiRae/GalaxySD.git
cd GalaxySD

conda create -n galaxysd
conda activate galaxysd
pip install -r requirements.txt

Now you have set up the workspace and could fine-tune a GalaxySD model.

⚙️ Customize configurations

For example, full fine-tuning training configurations are in `GalaxySD/cfgs/train/examples/fine-tuning_galaxy.yaml`. You could customize it before using. The parameters that must be modified to ensure the pipeline run well and corresponding descriptions in `fine-tuning_galaxy.yaml` are in the following table. The fine-tuning tool we used is HCP-Diffusion.

Training Parameter	Description	Example
`pretrained_model_name_or_path`	Pretrained model name in hugging-face / downloaded local path	`stable-diffusion-v1-5/stable-diffusion-v1-5`
`img_root`	Image path	a folder of `.jpg` files
`caption_file`	Caption path	a folder of `.txt` files whose filenames are same as corresponding images.
`resume`	Continue the previous training by filling this part or start a new training by set it to null

By setting these and the rest parameters in configuration, you could start full fine-tuning.

Before inference, you must modify the inference configurations in GalaxySD/cfgs/infer/text2img_galaxy_full.yaml.

Inference Parameter Description Example

pretrained_model Pretrained model name in hugging-face / downloaded local path stable-diffusion-v1-5/stable-diffusion-v1-5

condition

Control the generation

type: i2i

image: 'galaxy_cond.jpg'

🚀 Get started

Training

bash ./sub_gal_train_full.sh

Inference

Fill model name and steps and give prompts in `infer_script_full.sh`. You could use the model weights in 🤗HF (doi:10.57967/hf/6479)
bash ./infer_script_full.sh

If you wanna view a summary of generation, uncomment the last line of `infer_script_full.sh` and keep the prompts in `create_summary.py` consistent with those in inference script.

🔗 Project Resources

Files