ms-swift/examples/pytorch/stable_diffusion

Stable Diffusion Example

Modelscope Hub
中文  ｜  English

Features

1. Support Stable Diffusion LoRA method.
2. Support Stable Diffusion XL LoRA method.

Prepare the Environment

pip install -r requirements.txt

Train and Inference

# Clone the repository and enter the code directory.
git clone https://github.com/modelscope/swift.git

# Stable Diffusion LoRA
bash examples/pytorch/stable_diffusion/run_train_lora.sh

# Stable Diffusion XL LoRA
bash examples/pytorch/stable_diffusion/run_train_lora_xl.sh

Extend Datasets

The buptwq/lora-stable-diffusion-finetune dataset used in the example is from ModelScope Hub, you can replace different datasets ID by modifying the train_dataset_name parameter. In addition, you can also use local datasets. Fill in the path of the dataset file in train_dataset_name parameter, which needs to include a train.csv file to map image files and text prompts. Please organize it into the following format:

Text,Target:FILE
[prompt], [image dir]
......

Here is an example of train.csv file:

Text,Target:FILE
a dog,target/00.jpg
a dog,target/01.jpg
a dog,target/02.jpg
a dog,target/03.jpg
a dog,target/04.jpg