Simple Aesthetic Scorer for Bad Hands
as Produced by Diffusion Models

A couple years too late to the party now that modern diffusion models do a generally awesome job with hands and other fine details, but here's a simple proof-of-concept aesthetic scoring framework. The gist is that we are simply checking whether an image's CLIP encoding is closer to the vanilla CLIP model's embeddings or to a textual embedding trained on a variety of malformed hands and producing a "score" based on the result. There's plenty of room for improvement via training MLP / linear probing or switching to/augmenting with DINO, but for something really simple to build and extremely lightweight the results are subjectively pretty decent.

---

Repository Contents & Core Workflow

• ComfyUI Workflows (badhand_workflow.json / goodhand_workflow.json): Pipelines for generating images with bad hands (via using the text inversions in the positive prompt) and for generating good hands (via using a modern model). This makes for some passable test fodder.
• grab.sh: A utility shell script that grabs some test images from imagebench.ai where they are testing Vision-LLMs for similar tasks.
• bake_anchors.py: Extracts the positive and negative embedding anchors from the CLIP model and the text embeddings into a consolidated array artifact saved as hand_anchors.npz.
• score_image.py: Scores a single target image asset and outputs the calculated value directly to the console.
• batch_score.py: Iterates through an entire directory of images and records the output scores to a simple, headerless scores.csv table.

Note

Each of the Python scripts will require a local environment containing torch, transformers, and standard scientific dependencies.

---

Quick Start Example

# Clone the repository
git clone [https://github.com/FNGarvin/badhands_scorer.git](https://github.com/FNGarvin/badhands_scorer.git) && cd badhands_scorer

# Fetch the benchmarking dataset
./grab.sh

# Download the required vanilla CLIP model assets (several GB)
hf download openai/clip-vit-large-patch14 --local-dir ./openai/clip-vit-large-patch14

# Score the target dataset
./batch_score.py

Running the batch operation should produce a scores.csv file artifact along with console metrics looking something like this:

==================================================
BATCH EXECUTION PERFORMANCE SUMMARY
==================================================
Total Runtime:         9.3100 seconds
Images Processed:      24
Average Time / Image:  0.3879 seconds
==================================================

---

Evaluation Matrix Summary

The generated scores.csv is a simple map of filename to score that can be opened with a text editor or imported into other tools like a spreadsheet for analysis. Here is the sorted data matching the benchmark example set retrieved via ImageBench.ai:

Filename	Score
bonsai_HumanRealism_Hands_Easy__p1.png	-0.1
bonsai_HumanRealism_Hands_Easy__p2.png	0
bonsai_HumanRealism_Hands_Easy__p3.png	0.4
bonsai_HumanRealism_Hands_Extreme__p1.png	-0.1
bonsai_HumanRealism_Hands_Extreme__p2.png	0.6
bonsai_HumanRealism_Hands_Extreme__p3.png	-0.3
bonsai_HumanRealism_Hands_Hard__p1.png	0
bonsai_HumanRealism_Hands_Hard__p2.png	1.1
bonsai_HumanRealism_Hands_Hard__p3.png	-1
bonsai_HumanRealism_Hands_Medium__p1.png	-0.1
bonsai_HumanRealism_Hands_Medium__p2.png	-0.7
bonsai_HumanRealism_Hands_Medium__p3.png	0.9
nano_HumanRealism_Hands_Easy__p1.png	0.8
nano_HumanRealism_Hands_Easy__p2.png	1.2
nano_HumanRealism_Hands_Easy__p3.png	0.5
nano_HumanRealism_Hands_Extreme__p1.png	-0.1
nano_HumanRealism_Hands_Extreme__p2.png	0.8
nano_HumanRealism_Hands_Extreme__p3.png	-0.4
nano_HumanRealism_Hands_Hard__p1.png	0.1
nano_HumanRealism_Hands_Hard__p2.png	1
nano_HumanRealism_Hands_Hard__p3.png	0.9
nano_HumanRealism_Hands_Medium__p1.png	-0.1
nano_HumanRealism_Hands_Medium__p2.png	0.2
nano_HumanRealism_Hands_Medium__p3.png	0.6

---

Analysis Limits and Interpretations

Scores around zero are neutral ambiguous. Higher confidence scores either feature pretty decent hands or at least the kinds of hands the models have seen. Anomalies will happen because the model was trained on things like exaggerated thumbs from training on emojis or football fans with decorative mittens or whatever. Or in the reverse direction because the training of the embedding may have picked up correlations that either are not true or that do not correspond with the way humans gauge hands (eg, the model might be noticing a correlation between fuzzy earlobes and malformed hands or some other bizarre metric).

In my brief tests, the results seem surprisingly decent overall. When a "good" hand is rated down, I'm frequently surprised to find that studying the image reveals some fault. Similarly, bad hands that receive decent scores frequently have a plausible explanation. Motivated reasoning is a possible bias, but the nature of such a scorer is that it can't really explain why an image is good or bad only that it more closely resembles a bad image than a good one.

---

Credits and Acknowledgements

• Textual Embedding: This project is using and redistributing the badhandsv4 textual embedding by Euge_, which is generously distributed with very liberal permissions.
• Base Model Configuration: This project uses the OpenAI/CLIP model architecture.
• Underlying Engines: The bulk of the heavy lifting is being driven natively by the PyTorch and Transformers platforms.