Humans need not label more humans: Occlusion Copy & Paste for Occluded Human Instance Segmentation

• URL: http://arxiv.org/abs/2210.03686v1
• Date: Fri, 7 Oct 2022 16:44:05 GMT
• Title: Humans need not label more humans: Occlusion Copy & Paste for Occluded Human Instance Segmentation
• Authors: Evan Ling, Dezhao Huang and Minhoe Hur
• Abstract summary: We propose Occlusion Copy & Paste to introduce occluded examples to models during training. It improves instance segmentation performance on occluded scenarios for "free" just by leveraging on existing large-scale datasets. In a principled study, we show whether various proposed add-ons to the copy & paste augmentation indeed contribute to better performance.
• Score: 0.3867363075280543
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Modern object detection and instance segmentation networks stumble when picking out humans in crowded or highly occluded scenes. Yet, these are often scenarios where we require our detectors to work well. Many works have approached this problem with model-centric improvements. While they have been shown to work to some extent, these supervised methods still need sufficient relevant examples (i.e. occluded humans) during training for the improvements to be maximised. In our work, we propose a simple yet effective data-centric approach, Occlusion Copy & Paste, to introduce occluded examples to models during training - we tailor the general copy & paste augmentation approach to tackle the difficult problem of same-class occlusion. It improves instance segmentation performance on occluded scenarios for "free" just by leveraging on existing large-scale datasets, without additional data or manual labelling needed. In a principled study, we show whether various proposed add-ons to the copy & paste augmentation indeed contribute to better performance. Our Occlusion Copy & Paste augmentation is easily interoperable with any models: by simply applying it to a recent generic instance segmentation model without explicit model architectural design to tackle occlusion, we achieve state-of-the-art instance segmentation performance on the very challenging OCHuman dataset. Source code is available at https://github.com/levan92/occlusion-copy-paste.

Related papers

• Ins-HOI: Instance Aware Human-Object Interactions Recovery [44.02128629239429]
  We propose an end-to-end Instance-aware Human-Object Interactions recovery (Ins-HOI) framework. Ins-HOI supports instance-level reconstruction and provides reasonable and realistic invisible contact surfaces. We collect a large-scale, high-fidelity 3D scan dataset, including 5.2k high-quality scans with real-world human-chair and hand-object interactions.
  arXiv  Detail & Related papers  (2023-12-15T09:30:47Z)
• Ref-Diff: Zero-shot Referring Image Segmentation with Generative Models [68.73086826874733]
  We introduce a novel Referring Diffusional segmentor (Ref-Diff) for referring image segmentation. We demonstrate that without a proposal generator, a generative model alone can achieve comparable performance to existing SOTA weakly-supervised models. This indicates that generative models are also beneficial for this task and can complement discriminative models for better referring segmentation.
  arXiv  Detail & Related papers  (2023-08-31T14:55:30Z)
• Combining Public Human Activity Recognition Datasets to Mitigate Labeled Data Scarcity [1.274578243851308]
  We propose a novel strategy to combine publicly available datasets with the goal of learning a generalized HAR model. Our experimental evaluation, which includes experimenting with different state-of-the-art neural network architectures, shows that combining public datasets can significantly reduce the number of labeled samples.
  arXiv  Detail & Related papers  (2023-06-23T18:51:22Z)
• Prototype-Sample Relation Distillation: Towards Replay-Free Continual Learning [14.462797749666992]
  We propose a holistic approach to jointly learn the representation and class prototypes. We propose a novel distillation loss that constrains class prototypes to maintain relative similarities as compared to new task data. This method yields state-of-the-art performance in the task-incremental setting.
  arXiv  Detail & Related papers  (2023-03-26T16:35:45Z)
• CellTranspose: Few-shot Domain Adaptation for Cellular Instance Segmentation [4.38301148531795]
  We address the problem of designing an approach that requires minimal amounts of new annotated data as well as training time. We do so by designing specialized contrastive losses that leverage the few annotated samples very efficiently.
  arXiv  Detail & Related papers  (2022-12-28T23:00:50Z)
• X-Paste: Revisiting Scalable Copy-Paste for Instance Segmentation using CLIP and StableDiffusion [137.84635386962395]
  Copy-Paste is a simple and effective data augmentation strategy for instance segmentation. We revisit Copy-Paste at scale with the power of newly emerged zero-shot recognition models. X-Paste provides impressive improvements over the strong baseline CenterNet2 with Swin-L as the backbone.
  arXiv  Detail & Related papers  (2022-12-07T18:59:59Z)
• Decoupled Multi-task Learning with Cyclical Self-Regulation for Face Parsing [71.19528222206088]
  We propose a novel Decoupled Multi-task Learning with Cyclical Self-Regulation for face parsing. Specifically, DML-CSR designs a multi-task model which comprises face parsing, binary edge, and category edge detection. Our method achieves the new state-of-the-art performance on the Helen, CelebA-HQ, and LapaMask datasets.
  arXiv  Detail & Related papers  (2022-03-28T02:12:30Z)
• Revisiting Contrastive Methods for Unsupervised Learning of Visual Representations [78.12377360145078]
  Contrastive self-supervised learning has outperformed supervised pretraining on many downstream tasks like segmentation and object detection. In this paper, we first study how biases in the dataset affect existing methods. We show that current contrastive approaches work surprisingly well across: (i) object- versus scene-centric, (ii) uniform versus long-tailed and (iii) general versus domain-specific datasets.
  arXiv  Detail & Related papers  (2021-06-10T17:59:13Z)
• Contrastive Model Inversion for Data-Free Knowledge Distillation [60.08025054715192]
  We propose Contrastive Model Inversion, where the data diversity is explicitly modeled as an optimizable objective. Our main observation is that, under the constraint of the same amount of data, higher data diversity usually indicates stronger instance discrimination. Experiments on CIFAR-10, CIFAR-100, and Tiny-ImageNet demonstrate that CMI achieves significantly superior performance when the generated data are used for knowledge distillation.
  arXiv  Detail & Related papers  (2021-05-18T15:13:00Z)
• The Devil is in Classification: A Simple Framework for Long-tail Object Detection and Instance Segmentation [93.17367076148348]
  We investigate performance drop of the state-of-the-art two-stage instance segmentation model Mask R-CNN on the recent long-tail LVIS dataset. We unveil that a major cause is the inaccurate classification of object proposals. We propose a simple calibration framework to more effectively alleviate classification head bias with a bi-level class balanced sampling approach.
  arXiv  Detail & Related papers  (2020-07-23T12:49:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.