VETE: improving visual embeddings through text descriptions for eCommerce search engines

Guillermo Martínez*, Jose M. Saavedra*, Nils Murrugara-Llerena*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review


A search engine is a critical component in the success of eCommerce. Searching for a particular product can be frustrating when users want specific product features that cannot be easily represented by a simple text search or catalog filter. Due to the advances in artificial intelligence and deep learning, content-based visual search engines are included in eCommerce search bars. A visual search is instantaneous, just take a picture and search; and it is fully expressive of image details. However, visual search in eCommerce still undergoes a large semantic gap. Traditionally, visual search models are trained in a supervised manner with large collections of images that do not represent well the semantic of a target eCommerce catalog. Therefore, we propose VETE (Visual Embedding modulated by TExt) to boost visual embeddings in eCommerce leveraging textual information of products in the target catalog. with real eCommerce data. Our proposal improves the baseline visual space for global and fine-grained categories in real-world eCommerce data. We achieved an average improvement of 3.48% for catalog-like queries, and 3.70% for noisy ones.

Original languageEnglish
Pages (from-to)41343-41379
Number of pages37
JournalMultimedia Tools and Applications
Issue number26
StatePublished - Nov 2023

Bibliographical note

Publisher Copyright:
© 2023, The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature.


  • Content-based image retrieval
  • Self-supervised representation learning
  • Visual and text embeddings


Dive into the research topics of 'VETE: improving visual embeddings through text descriptions for eCommerce search engines'. Together they form a unique fingerprint.

Cite this