Related papers: Symmetric Network with Spatial Relationship Modeling for Natural Language-based Vehicle Retrieval

Symmetric Network with Spatial Relationship Modeling for Natural Language-based Vehicle Retrieval

URL: http://arxiv.org/abs/2206.10879v1
Date: Wed, 22 Jun 2022 07:02:04 GMT
Title: Symmetric Network with Spatial Relationship Modeling for Natural Language-based Vehicle Retrieval
Authors: Chuyang Zhao and Haobo Chen and Wenyuan Zhang and Junru Chen and Sipeng Zhang and Yadong Li and Boxun Li
Abstract summary: Natural language (NL) based vehicle retrieval aims to search specific vehicle given text description. We propose a Symmetric Network with Spatial Relationship Modeling (SSM) method for NL-based vehicle retrieval. We achieve 43.92% MRR accuracy on the test set of the 6th AI City Challenge on natural language-based vehicle retrieval track.
Score: 3.610372087454382
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Abstract: Natural language (NL) based vehicle retrieval aims to search specific vehicle given text description. Different from the image-based vehicle retrieval, NL-based vehicle retrieval requires considering not only vehicle appearance, but also surrounding environment and temporal relations. In this paper, we propose a Symmetric Network with Spatial Relationship Modeling (SSM) method for NL-based vehicle retrieval. Specifically, we design a symmetric network to learn the unified cross-modal representations between text descriptions and vehicle images, where vehicle appearance details and vehicle trajectory global information are preserved. Besides, to make better use of location information, we propose a spatial relationship modeling methods to take surrounding environment and mutual relationship between vehicles into consideration. The qualitative and quantitative experiments verify the effectiveness of the proposed method. We achieve 43.92% MRR accuracy on the test set of the 6th AI City Challenge on natural language-based vehicle retrieval track, yielding the 1st place among all valid submissions on the public leaderboard. The code is available at https://github.com/hbchen121/AICITY2022_Track2_SSM.