Attribute Descent: Simulating Object-Centric Datasets on the Content Level and Beyond

• URL: http://arxiv.org/abs/2202.14034v2
• Date: Wed, 29 Nov 2023 23:39:40 GMT
• Title: Attribute Descent: Simulating Object-Centric Datasets on the Content Level and Beyond
• Authors: Yue Yao, Liang Zheng, Xiaodong Yang, Milind Napthade, and Tom Gedeon
• Abstract summary: Between synthetic and real, a two-level domain gap exists, involving content level and appearance level. We propose an attribute descent approach that automatically optimize engine attributes to enable synthetic data to approximate real-world data. Experiments on image classification and object re-identification confirm that adapted synthetic data can be effectively used in three scenarios.
• Score: 17.949962340691673
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This article aims to use graphic engines to simulate a large number of training data that have free annotations and possibly strongly resemble to real-world data. Between synthetic and real, a two-level domain gap exists, involving content level and appearance level. While the latter is concerned with appearance style, the former problem arises from a different mechanism, i.e, content mismatch in attributes such as camera viewpoint, object placement and lighting conditions. In contrast to the widely-studied appearance-level gap, the content-level discrepancy has not been broadly studied. To address the content-level misalignment, we propose an attribute descent approach that automatically optimizes engine attributes to enable synthetic data to approximate real-world data. We verify our method on object-centric tasks, wherein an object takes up a major portion of an image. In these tasks, the search space is relatively small, and the optimization of each attribute yields sufficiently obvious supervision signals. We collect a new synthetic asset VehicleX, and reformat and reuse existing the synthetic assets ObjectX and PersonX. Extensive experiments on image classification and object re-identification confirm that adapted synthetic data can be effectively used in three scenarios: training with synthetic data only, training data augmentation and numerically understanding dataset content.

Related papers

• Web-Scale Visual Entity Recognition: An LLM-Driven Data Approach [56.55633052479446]
  Web-scale visual entity recognition presents significant challenges due to the lack of clean, large-scale training data. We propose a novel methodology to curate such a dataset, leveraging a multimodal large language model (LLM) for label verification, metadata generation, and rationale explanation. Experiments demonstrate that models trained on this automatically curated data achieve state-of-the-art performance on web-scale visual entity recognition tasks.
  arXiv  Detail & Related papers  (2024-10-31T06:55:24Z)
• Massively Annotated Datasets for Assessment of Synthetic and Real Data in Face Recognition [0.2775636978045794]
  We study the drift between the performance of models trained on real and synthetic datasets. We conduct studies on the differences between real and synthetic datasets on the attribute set. Interestingly enough, we have verified that while real samples suffice to explain the synthetic distribution, the opposite could not be further from being true.
  arXiv  Detail & Related papers  (2024-04-23T17:10:49Z)
• Modeling Entities as Semantic Points for Visual Information Extraction in the Wild [55.91783742370978]
  We propose an alternative approach to precisely and robustly extract key information from document images. We explicitly model entities as semantic points, i.e., center points of entities are enriched with semantic information describing the attributes and relationships of different entities. The proposed method can achieve significantly enhanced performance on entity labeling and linking, compared with previous state-of-the-art models.
  arXiv  Detail & Related papers  (2023-03-23T08:21:16Z)
• Synthetic-to-Real Domain Adaptation for Action Recognition: A Dataset and Baseline Performances [76.34037366117234]
  We introduce a new dataset called Robot Control Gestures (RoCoG-v2) The dataset is composed of both real and synthetic videos from seven gesture classes. We present results using state-of-the-art action recognition and domain adaptation algorithms.
  arXiv  Detail & Related papers  (2023-03-17T23:23:55Z)
• Synthetic Data for Object Classification in Industrial Applications [53.180678723280145]
  In object classification, capturing a large number of images per object and in different conditions is not always possible. This work explores the creation of artificial images using a game engine to cope with limited data in the training dataset.
  arXiv  Detail & Related papers  (2022-12-09T11:43:04Z)
• TRoVE: Transforming Road Scene Datasets into Photorealistic Virtual Environments [84.6017003787244]
  This work proposes a synthetic data generation pipeline to address the difficulties and domain-gaps present in simulated datasets. We show that using annotations and visual cues from existing datasets, we can facilitate automated multi-modal data generation.
  arXiv  Detail & Related papers  (2022-08-16T20:46:08Z)
• CrossLoc: Scalable Aerial Localization Assisted by Multimodal Synthetic Data [2.554905387213586]
  We present a visual localization system that learns to estimate camera poses in the real world with the help of synthetic data. To mitigate the data scarcity issue, we introduce TOPO-DataGen, a versatile synthetic data generation tool. We also introduce CrossLoc, a cross-modal visual representation learning approach to pose estimation.
  arXiv  Detail & Related papers  (2021-12-16T18:05:48Z)
• Salient Objects in Clutter [130.63976772770368]
  This paper identifies and addresses a serious design bias of existing salient object detection (SOD) datasets. This design bias has led to a saturation in performance for state-of-the-art SOD models when evaluated on existing datasets. We propose a new high-quality dataset and update the previous saliency benchmark.
  arXiv  Detail & Related papers  (2021-05-07T03:49:26Z)
• Detection and Segmentation of Custom Objects using High Distraction Photorealistic Synthetic Data [0.5076419064097732]
  We show a straightforward and useful methodology for performing instance segmentation using synthetic data. The goal is to achieve high performance on manually-gathered and annotated real-world data of custom objects. This white-paper provides strong evidence that photorealistic simulated data can be used in practical real world applications.
  arXiv  Detail & Related papers  (2020-07-28T16:33:42Z)
• Learning to simulate complex scenes [18.51564016785853]
  This paper explores content adaptation in the context of semantic segmentation. We propose a scalable discretization-and-relaxation (SDR) approach to optimize the attribute values and obtain a training set of similar content to real-world data. Experiment shows our system can generate reasonable and useful scenes, from which we obtain promising real-world segmentation accuracy.
  arXiv  Detail & Related papers  (2020-06-25T17:51:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.