Survey on Sketch-to-photo Translation

Diego Donoso*, Jose M. Saavedra*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

Abstract

Sketch-based understanding is involved in human communication and cognitive development, making it essential in visual perception. A specific task in this domain is sketch-to-photo translation, where a model produces realistic images from simple drawings. To this end, large paired training datasets are commonly required, which is impractical in real applications. Thus, this work studies conditional generative models for sketch-to-photo translation, overcoming the lack of training datasets by a self-supervised approach that produces sketch-photo pairs from a target catalog. Our study shows the benefit of cycle-consistency loss and UNet architectures that, together with the proposed dataset generation, improve performance in real applications like eCommerce. Our results also reveal the weakness of conditional DDPMs for generating images resembling the input sketch, even though they achieve a high FID score.

Original languageEnglish
Article number22
Pages (from-to)1–25
Number of pages25
JournalACM Computing Surveys
Volume56
Issue number1
DOIs
StatePublished - 26 Aug 2023

Bibliographical note

Publisher Copyright:
© 2023 Copyright held by the owner/author(s). Publication rights licensed to ACM.

Keywords

  • Additional Key Words and PhrasesGenerative models
  • conditional GANs
  • deep learning
  • sketch-to-photo translation

Fingerprint

Dive into the research topics of 'Survey on Sketch-to-photo Translation'. Together they form a unique fingerprint.

Cite this