Diffusion vs. Flow Matching for Face-to-Comics Translation

Comparative Analysis of Diffusion and Flow Matching Models for Face-to-Comic Translation 확산 모델과 플로우 매칭을 이용한 얼굴-만화 변환 성능 비교 연구

A controlled, inference-only study of the accuracy–efficiency trade-off between a score-based diffusion model and a velocity-based flow matching model on the paired face-to-comics image translation task.

Undergraduate research paper (MEU4400), Department of Mechanical Engineering, Yonsei University, Spring 2026. The paper body is written in Korean; the abstract, code, and this README are in English.

Summary

Both Stable Diffusion 1.5 (score-based diffusion) and Rectified Flow InstaFlow-0.9B (velocity-based flow matching) steer conditional generation through classifier-free guidance (CFG) — yet the same guidance formula acts on mathematically different objects (a score field vs. a velocity field). We fix the dataset, evaluation code, and hardware identically and treat the CFG scale as the sole controlled axis, measuring FID, IS, LPIPS, NFE, and per-sample latency over w ∈ {1, 3, 5, 7.5, 12}, complemented by an NFE×CFG ablation and a qualitative failure-mode taxonomy.

Key findings

• The score-based model shows a U-shaped FID curve, minimized at FID 119.81 (w = 5).
• The velocity-based model stays on a flat plateau of FID 407–426 across the entire CFG range (~3.4× higher FID) with output-diversity collapse (low IS).
• The velocity-based model instead reaches its best quality and ~11× faster inference at a single-step (NFE = 2) operating point.
• → The two families occupy opposite extremes of the quality–speed Pareto frontier; an identical nominal CFG scale does not imply identical effective guidance.

Repository structure

.
├── paper/            # Final paper: Markdown sources, IEEE-styled .docx, build scripts
│   ├── 01-introduction.md … 06-conclusion.md, _titlepage.md, _frontmatter.md
│   ├── sections-en/  # English section drafts (archive)
│   ├── 연구논문_박도울_2026-06-12.pdf   # archival PDF (Zenodo primary file)
│   ├── 연구논문_박도울_2026-06-12.docx  # editable source
│   └── build-final-paper.sh, style-docx.py, ieee.csl
├── code/
│   ├── eval/         # FID / IS / LPIPS / NFE / latency package (see code/eval/README.md)
│   └── notebooks/    # Colab notebooks: baselines, CFG sweep, ablation, figures
│       └── 04-sweep/ # sweep_results.csv, ablation_results.csv, outputs/ (figures)
├── research/
│   ├── theory/       # Mathematical formulations (diffusion, flow matching, comparison)
│   ├── literature/   # refs.bib (IEEE), key-papers.md
│   └── experiments/  # Experiment logs, failure-mode taxonomy
├── presentation/     # Final presentation slides (PDF)
├── CITATION.cff
├── requirements.txt
├── LICENSE           # MIT — applies to code
└── LICENSE-CONTENT   # CC BY 4.0 — applies to paper, figures, and documentation

Dataset & checkpoints

• Dataset: Sxela/face2comics v1.0.0 — 10,000 paired 512×512 RGB images, MIT license. Not redistributed here; download from the original release.
• Checkpoints: pretrained Stable Diffusion 1.5 and InstaFlow-0.9B are fetched from the Hugging Face Hub at run time (a HF token may be required). No weights are stored in this repository.

Reproduction

Experiments run on Google Colab (NVIDIA A100 40 GB). The full CFG sweep completes in ~54 min wall-clock.

pip install -r requirements.txt

The evaluation package reads JSONL inference logs and emits one CSV row per (model, cfg_scale, seed) cell:

import sys; sys.path.insert(0, "code")          # add code/ to PYTHONPATH
from eval.runner import run_eval                 # FID/IS/LPIPS/NFE/latency

See code/eval/README.md for the CSV/JSONL schemas, pinned metric library versions, and the NFE counting convention. Aggregated results are checked in at code/notebooks/04-sweep/sweep_results.csv and ablation_results.csv; figures are regenerated by the scripts/notebooks under code/notebooks/04-sweep/ and code/notebooks/06-figures/.

Paper

The final paper (28 pp.; Korean body with an English abstract) is in paper/:

• 연구논문_박도울_2026-06-12.pdf — the archival PDF and the primary file for Zenodo.
• 연구논문_박도울_2026-06-12.docx — editable source.
• Markdown sources (01-introduction.md … 06-conclusion.md); rebuild the .docx with paper/build-final-paper.sh (Pandoc + ieee.csl).

Citation

If you use this work, please cite it via CITATION.cff, or:

D. Park, "Comparative Analysis of Diffusion and Flow Matching Models for Face-to-Comic Translation," Undergraduate research paper, Dept. of Mechanical Engineering, Yonsei University, 2026. doi: 10.5281/zenodo.20631855.

Archived on Zenodo: https://doi.org/10.5281/zenodo.20631855

License

• Code (code/, build scripts): MIT.
• Paper, figures, and documentation (paper/, presentation/, research/, README): CC BY 4.0.

The face2comics dataset and the pretrained model checkpoints retain their own respective licenses.

Contact

Doul Park — doulzang@gmail.com

Repository URL is a placeholder until the public repo is created: https://github.com/luodkrap/face2comics-diffusion-vs-flow-matching