Please use this identifier to cite or link to this item:
https://hdl.handle.net/10316/114664
Title: | TECMH: Transformer-Based Cross-Modal Hashing For Fine-Grained Image-Text Retrieval | Authors: | Li, Qiqi Ma, Longfei Jiang, Zheng Li, Mingyong Jin, Bo |
Keywords: | Deep learning; cross-modal retrieval; hash learning; transformer | Issue Date: | 2023 | Publisher: | Tech Science Press | Serial title, monograph or event: | Computers, Materials and Continua | Volume: | 75 | Issue: | 2 | Abstract: | In recent years, cross-modal hash retrieval has become a popular research field because of its advantages of high efficiency and low storage. Cross-modal retrieval technology can be applied to search engines, crossmodalmedical processing, etc. The existing main method is to use amulti-label matching paradigm to finish the retrieval tasks. However, such methods do not use fine-grained information in the multi-modal data, which may lead to suboptimal results. To avoid cross-modal matching turning into label matching, this paper proposes an end-to-end fine-grained cross-modal hash retrieval method, which can focus more on the fine-grained semantic information of multi-modal data. First, the method refines the image features and no longer uses multiple labels to represent text features but uses BERT for processing. Second, this method uses the inference capabilities of the transformer encoder to generate global fine-grained features. Finally, in order to better judge the effect of the fine-grained model, this paper uses the datasets in the image text matching field instead of the traditional label-matching datasets. This article experiment on Microsoft COCO (MS-COCO) and Flickr30K datasets and compare it with the previous classicalmethods. The experimental results show that this method can obtain more advanced results in the cross-modal hash retrieval field. | URI: | https://hdl.handle.net/10316/114664 | ISSN: | 1546-2226 | DOI: | 10.32604/cmc.2023.037463 | Rights: | openAccess |
Appears in Collections: | FCTUC Eng.Electrotécnica - Artigos em Revistas Internacionais I&D ISR - Artigos em Revistas Internacionais |
Files in This Item:
File | Description | Size | Format | |
---|---|---|---|---|
TECMH-TransformerBased-CrossModal-Hashing-for-FineGrained-ImageText-RetrievalComputers-Materials-and-Continua.pdf | 900.33 kB | Adobe PDF | View/Open |
Page view(s)
87
checked on Oct 16, 2024
Download(s)
40
checked on Oct 16, 2024
Google ScholarTM
Check
Altmetric
Altmetric
This item is licensed under a Creative Commons License