Skip to content

Commit ae5c115

Browse files
davide-coccominidavide-coccomini
authored
Update README.md
1 parent 980e3c9 commit ae5c115

File tree

1 file changed

+113
-1
lines changed

1 file changed

+113
-1
lines changed

‎README.md‎

Lines changed: 113 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,4 +5,116 @@ Code for paper available on [Arxiv](https://arxiv.org/abs/2303.05275)
55

66
This paper explores the task of detecting images generated by text-to-image diffusion models. To evaluate this, we consider images generated from captions in the MSCOCO and Wikimedia datasets using two state-of-the-art models: Stable Diffusion and GLIDE. Our experiments show that it is possible to detect the generated images using simple Multi-Layer Perceptrons (MLPs), starting from features extracted by CLIP, or traditional Convolutional Neural Networks (CNNs). We also observe that models trained on images generated by Stable Diffusion can detect images generated by GLIDE relatively well, however, the reverse is not true. Lastly, we find that incorporating the associated textual information with the images rarely leads to significant improvement in detection results but that the type of subject depicted in the image can have a significant impact on performance. This work provides insights into the feasibility of detecting generated images, and has implications for security and privacy concerns in real-world applications.
77

8-
![approach](images/detector.png)
8+
![approach](images/detector.png)
9+
10+
11+
# Preprocessing
12+
13+
To train and test a new model it is necessary to generate one synthetic image for each prompt available in the two considered datasets.
14+
- MSCOCO: https://cocodataset.org/
15+
- Wikipedia Dataset: https://www.kaggle.com/c/wikipedia-image-caption
16+
17+
Run the following command to start the generation:
18+
'''
19+
python3 generate_images.py
20+
'''
21+
22+
You can customize it with the following parameters:
23+
- --workers: Number of workers (Default: 100)
24+
- --n_samples: Number of fake images generated for each prompt (default: 4)
25+
- --considered_images: Number of pristine images (and prompts) to be considered from the dataset (default: 6000)
26+
- --dataset: Which dataset to be used for the generation (0: COCO, 1: Wikipedia) (default: 0)
27+
- --list_file: Path to the json containing the list of images (default: "../datasets/coco/annotations/captions_val2014.json")
28+
- --copy_files: If used, the process will also copy the pristine image associated with the caption, it is due for the first running (default: False)
29+
- --force_captions: Consider only the captions contained in a specific folder (the name of the folders)
30+
- --excluded_images: Exclude images/captions contained in the paths indicated
31+
- --output_path: Path in which the images should be saved
32+
33+
This should be run for each dataset and for each part of the dataset (train/val/test). The output folders will look like:
34+
35+
- datasets
36+
- train
37+
- Prompt 0
38+
- Fake Image 0
39+
- ...
40+
- ...
41+
- Fake Image N
42+
- Pristine Image
43+
44+
- ...
45+
- ...
46+
- Prompt N
47+
- ...
48+
- val
49+
- Prompt 0
50+
- Fake Image 0
51+
- ...
52+
- ...
53+
- Fake Image N
54+
- Pristine Image
55+
56+
- ...
57+
- ...
58+
- Prompt N
59+
- ...
60+
- test
61+
- Prompt 0
62+
- Fake Image 0
63+
- ...
64+
- ...
65+
- Fake Image N
66+
- Pristine Image
67+
68+
- ...
69+
- ...
70+
- Prompt N
71+
- ...
72+
73+
74+
# Training
75+
76+
To train a model on the prepared dataset run the following command:
77+
'''
78+
python3 train.py --config path/to/config
79+
'''
80+
81+
You can customize it with the following parameters:
82+
- --num_epochs: Number of epochs for training (default: 30)
83+
- --workers: Number of workers (default: 100)
84+
- --training_path: Path to the training images constructed as in the preprocessing
85+
- --validation_path: Path to the validation images constructed as in the preprocessing
86+
- --resume: Path to the weights from which restart the training
87+
- --model_name: Name of the model to be saved (default: "model")
88+
- --model_path: Path in which to save the models
89+
- --gpu_id: ID of GPU to be used
90+
- --config: Path to the configuration file contained in the config folder
91+
- --model: Which model to be trained between those available
92+
- --clip_mode: Which CLIP version should be used and which features should be used in the case of text+image setup (Resnet50 or ViT Features)
93+
- --mode: If training should be conducted in Image-Only or Image+Text setup
94+
- --patience: Number of epochs should be waited before stopping for validation loss not improving (default: 5)
95+
- --use_pretrained: Automatically load pretrained model when available (default: True)
96+
- --show_stats: Allow additional stats prints (default: True)
97+
- --logger_name: Path in which save logs
98+
99+
100+
# Inference
101+
102+
To test a model on the prepared dataset run the following command:
103+
'''
104+
python3 test.py --config path/to/config --model_weights path/to/model/weights
105+
'''
106+
107+
You can customize it with the following parameters:
108+
- --workers: Number of workers (default: 100)
109+
- --test_path: Path containing the test images constructed as in preprocessing
110+
- --model_weights: Path to the weights of the trained model to be tested
111+
- --model_name: Name of the model for saving statistics (default: Model)
112+
- --display_pre: Print some text before the statistics (default: "")
113+
- --display_post: Print some text after the statistics (default: "")
114+
- --gpu_id: Id of GPU to be used
115+
- --config: Path to the configuration file contained in the config folder
116+
- --model: Model to be tested
117+
- --clip_mode: Which CLIP version should be used and which features should be used in the case of text+image setup (Resnet50 or ViT Features)
118+
- --mode: If the test should be conducted in Image-Only or Image+Text setup
119+
- --show_stats: Allow additional stats prints (default: False)
120+
- --analyze_tags: (Available only on Wikipedia dataset) Use tags.txt file for error analysis as in the paper (default: False)

0 commit comments

Comments
 (0)