RST-SHELO: sketch-based image retrieval using sketch tokens and square root normalization

Sketch-based image retrieval (SBIR) is an emergent research area with a variety of applications, specially when an example image is not available for querying. Moreover, making a sketch has become a very attractive and simple task due to the already ubiquitous touch-screen and mobile technologies. Although a sketch is a natural way for representing the structure of a thought object, it may easily get confused in a dataset with high variability turning the retrieval task a quite challenging problem. Indeed, the state-of-the-art methods still show low performance on diverse evaluation datasets. Thereby, a robust sketch descriptor together with a better strategy for representing regular images as sketches are demanded. In this work, we present RST-SHELO, and improved version of SHELO (Soft Histogram of Edge Logal Orientations), an efficient state-of-the-art method for describing sketches. The proposed improvements comes from two aspects: a better technique for obtaining sketch-like representations and a better normalization strategy of SHELO. For the first case, we propose to use the sketch token approach [21], aiming to detect image contours by means of mid-level features. For the second case, we demonstrate that a square root normalization positively affect the effectiveness on the retrieval task. Based on our improvements, we present new state-of-the-art performance. To validate our achievements, we have conducted diverse experiments using two public datasets, Flickr15K and Saavedra’s. Our results show an effectiveness gain of 62 % in the first and 5 % in the second dataset.