Quadruplet-Based Deep Cross-Modal Hashing

Figure 2

Flowchart of the proposed quadruplet-based deep cross-modal hashing (QDCMH) method. QDCMH encompasses three steps: (1) a quadruplet-based cross-modal semantic preserving module, (2) a classical convolutional neural network is used to learn image-modality features and the TxtNet in SSAH [15] is adopted to learn the text-modality features, and (3) an intermodal quadruplet loss is utilized to efficiently capture the relevant semantic information during the feature learning process and a quantization loss is used to decrease information loss during the hash codes generation procedure. (a) Quadruplet , which utilizes an image instance to retrieve three text instances: , , and . and have at least one common labels, while and , and , and and are three pairwise instances and the two instances in each pairwise have no common label. (b) Quadruplet , which utilizes a text instance to retrieve three image instances: , , and . and have at least one common labels, while and , and , and and are three pairwise instances and the two instances in each pairwise have no common label.

(a)

(b)