Automatic Aesthetic Evaluation and Prompt Controllability in Generative Image Models

This repository contains the code used to conduct experiments and generate results for the paper Automatic Aesthetic Evaluation and Prompt Controllability in Generative Image Models, submitted to ICCC 2025.

📌 Overview

The project explores automatic methods for assessing the aesthetic quality of images generated by deep learning models. Additionally, it investigates how prompt variations influence the generated output, analyzing controllability aspects in text-to-image models.

📁 Repository Structure

What each file do? Or is here for?

• .gitgnore: self-explanatory
• amostra.py: Sample the original database so we can work with an expressive part of the hole, without the computational cost associeated with.
• get_descriptions.py: Generate the descriptions based on the respective image of the dataset, using Janus 7b.
• get_gen_img.py: Generate the sintetic data based on the generated description in the script above.
• generate_scores.py: Evaluate the generated images, using the image, only, to assess the different criteria encompassed by the ArtClip model
• oficial_script.sh: Automatize running the scripts
• manda_email.py: In oficial_script may need to change the directory. It's used to notify when the script has finished to run.
• metricas.py: Sum-up the returned scores in order to produce a statistical analysis.
• README.md: self-explanatory
• requirements.txt: self-explanatory

⚙️ Setup

To set up the environment, there are some phases you must go through.

1) Create a venv.

First you create a enviroment using:

python3 -m venv venv

Then you create a folder to be your home, where you will install your dependencies:

mkdir "/minha_home"

Then start your venv using:

source venv/bin/activate

Set your macros for the path of the created folder above:

export HOME="/minha_home"
export TRANSFORMERS_CACHE="<path>/minha_home/.cache/huggingface"
export CLIP_CACHE="<path>/minha_home/.cache/clip"
export HF_HOME="<path>/minha_home/.cache/huggingface"
export XDG_CACHE_HOME="/<path>/minha_home/.cache"
export MPLCONFIGDIR="<path>/minha_home/.matplotlib"

Finally copy the present repository with the command below:

https://github.com/LarissaDG/ICCC.git

And install the requirements:

pip install --no-cache-dir -r "<paht>/requirements.txt" && touch "<paht>/requirements_installed"

2) Downloading and setting APDDv2

The APDDv2 paper, publicly available at https://arxiv.org/abs/2411.08545, introduces the Aesthetics of Paintings and Drawings Dataset (APDDv2). This dataset comprises 10,023 images, each annotated with scores across 10 aesthetic attributes and categorized into 24 distinct artistic styles and subject types. These categories encompass various painting techniques, artistic movements, and depicted themes. Additionally, the paper presents ArtCLIP, a pre-trained model specifically designed for aesthetic evaluation of artistic images. ArtCLIP leverages the APDDv2 dataset through a fine-tuned adaptation of the CLIP model, integrating multimodal learning to enhance image aesthetic assessment.

For more details in how to install APDDv2 dependêncies check their GitHub: https://github.com/BestiVictory/APDDv2?tab=readme-ov-file

Essentialy you need to follow the steps below:

First, copy the repository:

git clone https://github.com/BestiVictory/APDDv2?tab=readme-ov-file 

Then download the images. For doing this I suggest the follow commands:

gdown --id "1ap5dhuEgpPC5PrJozAu2VFmUNIRZrar2" -O "APDDv2images.zip"

This folder must be unzip inside the APDDv2 folder created by the git clone step above.

Then you must download the pre-trained models. For this use the command:

gdown --folder "1AOVKmSqZCW09J_Ypr7KzSYfRxQre-w_m"

The models downloaded must be inside the folder: modle_weights. So the folder hierarchy must be similar to:

APDDv2/
├── APDDv2images/
│ ├── image1.png
│ └── image2.png
├── model_weights/
│ ├── model_v1.pth
│ └── model_v2.pth
└── README.md

Now it is time for the settings. First install the requirements:

pip install --no-cache-dir -r "<path>/APDDv2/requirements.txt" && touch "<path>/requirements_installed_2"

Finally test whether the sistem works. In order to do this you must run the scripts:

python3 "eval.py" || echo "Fail to execute eval.py."

python3 "demo.py" || echo "Fail to execute demo.py."

Essentially, both of these files performs an image evaluation using the proposed ArtClip model. The first just return the "general" score, while the second returns all the scores across the 10 aesthetic attributes. However, it worth noting that speacially the model 6 (6.The sense of order_reg_weight--e5-train0.3708-test0.6206_best.pth) was not able to be used, due to model bugs. Note that you might need to do some adaptations in these scripts in order to acess the right model files, and image files.

3) Downloading and setting Janus

For more details in how to install APDDv2 dependêncies check their GitHub: https://github.com/deepseek-ai/Janus.git

First, copy the repository:

git clone https://github.com/deepseek-ai/Janus.git

Now it is time for the settings. First install the requirements:

pip install -e "<path>/Janus"

Finally test whether the sistem works. In order to do this you must run the script:

python3 "<path>/exemple_janus.py" || echo "Erro ao executar exemple_janus.py."

🚀 Running Experiments

To reproduce the experiments, execute:

sbatch oficial_script.sh

📊 Results & Analysis

The results include:

• Aesthetic score comparisons across different models.
• Insights into prompt engineering techniques for better controllability.

📜 Citation

If you use this code, please cite our work:

@inproceedings{gomide2025iccc,
      title={Automatic Aesthetic Evaluation and Prompt Controllability in Generative Image Models},
      author={Larissa Gomide and Lucas Nascimento Ferreira and Wagner Meira Jr.},
      booktitle={Proceedings of the ICCC 2025},
      year={2025}
    }

Or as indicated by the CITATION.cff

📄 Repository Licenses

This repository contains different types of content, each with its own license. Please follow the instructions below when using any part of the material.

Content	License	How to cite / give credit
💻 Software		You must retain credit in the code and redistributions. See the LICENSE file for details.
🌐 Website		Apply CC BY license to text and images; include credits and citation instructions in the site footer.
📊 Dataset		Include a LICENSE file and a README with clear citation instructions and bibliographic references.

📬 Contact

For any questions or collaborations, feel free to reach out:
📧 lainterlocucao@gmail.com