MaskSearch: Querying Image Masks at Scale

• URL: http://arxiv.org/abs/2305.02375v2
• Date: Mon, 8 Jan 2024 08:26:33 GMT
• Title: MaskSearch: Querying Image Masks at Scale
• Authors: Dong He, Jieyu Zhang, Maureen Daum, Alexander Ratner, Magdalena Balazinska
• Abstract summary: MaskSearch is a system that focuses on accelerating queries over databases of image masks while guaranteeing the correctness of query results. Experiments with our prototype show that MaskSearch, using indexes approximately 5% of the compressed data size, accelerates individual queries by up to two orders of magnitude.
• Score: 60.82746984506577
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Machine learning tasks over image databases often generate masks that annotate image content (e.g., saliency maps, segmentation maps, depth maps) and enable a variety of applications (e.g., determine if a model is learning spurious correlations or if an image was maliciously modified to mislead a model). While queries that retrieve examples based on mask properties are valuable to practitioners, existing systems do not support them efficiently. In this paper, we formalize the problem and propose MaskSearch, a system that focuses on accelerating queries over databases of image masks while guaranteeing the correctness of query results. MaskSearch leverages a novel indexing technique and an efficient filter-verification query execution framework. Experiments with our prototype show that MaskSearch, using indexes approximately 5% of the compressed data size, accelerates individual queries by up to two orders of magnitude and consistently outperforms existing methods on various multi-query workloads that simulate dataset exploration and analysis processes.

Related papers

• Shift and matching queries for video semantic segmentation [0.0]
  We propose a method to extend a query-based image segmentation model to video. The method uses a query-based architecture, where decoded queries represent segmentation masks. Experimental results on CityScapes-VPS and VSPW show significant improvements from the baselines.
  arXiv  Detail & Related papers  (2024-10-10T06:07:33Z)
• Masked Image Modeling: A Survey [73.21154550957898]
  Masked image modeling emerged as a powerful self-supervised learning technique in computer vision. We construct a taxonomy and review the most prominent papers in recent years. We aggregate the performance results of various masked image modeling methods on the most popular datasets.
  arXiv  Detail & Related papers  (2024-08-13T07:27:02Z)
• ColorMAE: Exploring data-independent masking strategies in Masked AutoEncoders [53.3185750528969]
  Masked AutoEncoders (MAE) have emerged as a robust self-supervised framework. We introduce a data-independent method, termed ColorMAE, which generates different binary mask patterns by filtering random noise. We demonstrate our strategy's superiority in downstream tasks compared to random masking.
  arXiv  Detail & Related papers  (2024-07-17T22:04:00Z)
• Demonstration of MaskSearch: Efficiently Querying Image Masks for Machine Learning Workflows [3.7576528194977965]
  MaskSearch is a system designed to accelerate queries over databases of image masks generated by machine learning models. It formalizes and accelerates a new category of queries for retrieving images and their corresponding masks based on mask properties.
  arXiv  Detail & Related papers  (2024-04-09T18:27:59Z)
• Variance-insensitive and Target-preserving Mask Refinement for Interactive Image Segmentation [68.16510297109872]
  Point-based interactive image segmentation can ease the burden of mask annotation in applications such as semantic segmentation and image editing. We introduce a novel method, Variance-Insensitive and Target-Preserving Mask Refinement to enhance segmentation quality with fewer user inputs. Experiments on GrabCut, Berkeley, SBD, and DAVIS datasets demonstrate our method's state-of-the-art performance in interactive image segmentation.
  arXiv  Detail & Related papers  (2023-12-22T02:31:31Z)
• Improving Masked Autoencoders by Learning Where to Mask [65.89510231743692]
  Masked image modeling is a promising self-supervised learning method for visual data. We present AutoMAE, a framework that uses Gumbel-Softmax to interlink an adversarially-trained mask generator and a mask-guided image modeling process. In our experiments, AutoMAE is shown to provide effective pretraining models on standard self-supervised benchmarks and downstream tasks.
  arXiv  Detail & Related papers  (2023-03-12T05:28:55Z)
• Few-shot semantic segmentation via mask aggregation [5.886986014593717]
  Few-shot semantic segmentation aims to recognize novel classes with only very few labelled data. Previous works have typically regarded it as a pixel-wise classification problem. We introduce a mask-based classification method for addressing this problem.
  arXiv  Detail & Related papers  (2022-02-15T07:13:09Z)
• Scaling up instance annotation via label propagation [69.8001043244044]
  We propose a highly efficient annotation scheme for building large datasets with object segmentation masks. We exploit these similarities by using hierarchical clustering on mask predictions made by a segmentation model. We show that we obtain 1M object segmentation masks with a total annotation time of only 290 hours.
  arXiv  Detail & Related papers  (2021-10-05T18:29:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.