GUI Element Detection Using SOTA YOLO Deep Learning Models

• URL: http://arxiv.org/abs/2408.03507v1
• Date: Wed, 7 Aug 2024 02:18:39 GMT
• Title: GUI Element Detection Using SOTA YOLO Deep Learning Models
• Authors: Seyed Shayan Daneshvar, Shaowei Wang,
• Abstract summary: Detection of Graphical User Interface (GUI) elements is a crucial task for automatic code generation from images and sketches, GUI testing, and GUI search. Recent studies have leveraged both old-fashioned and modern computer vision (CV) techniques. In this study, we evaluate the performance of the four most recent successful YOLO models for general object detection tasks on GUI element detection.
• Score: 5.835026544704744
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Detection of Graphical User Interface (GUI) elements is a crucial task for automatic code generation from images and sketches, GUI testing, and GUI search. Recent studies have leveraged both old-fashioned and modern computer vision (CV) techniques. Oldfashioned methods utilize classic image processing algorithms (e.g. edge detection and contour detection) and modern methods use mature deep learning solutions for general object detection tasks. GUI element detection, however, is a domain-specific case of object detection, in which objects overlap more often, and are located very close to each other, plus the number of object classes is considerably lower, yet there are more objects in the images compared to natural images. Hence, the studies that have been carried out on comparing various object detection models, might not apply to GUI element detection. In this study, we evaluate the performance of the four most recent successful YOLO models for general object detection tasks on GUI element detection and investigate their accuracy performance in detecting various GUI elements.

Related papers

• Accelerating Object Detection with YOLOv4 for Real-Time Applications [0.276240219662896]
  Convolutional Neural Network (CNN) have emerged as a powerful tool for recognizing image content and in computer vision approach for most problems. This paper introduces the brief introduction of deep learning and object detection framework like Convolutional Neural Network(CNN)
  arXiv  Detail & Related papers  (2024-10-17T17:44:57Z)
• Learning-based Relational Object Matching Across Views [63.63338392484501]
  We propose a learning-based approach which combines local keypoints with novel object-level features for matching object detections between RGB images. We train our object-level matching features based on appearance and inter-frame and cross-frame spatial relations between objects in an associative graph neural network.
  arXiv  Detail & Related papers  (2023-05-03T19:36:51Z)
• Fast and Accurate Object Detection on Asymmetrical Receptive Field [0.0]
  This article proposes methods for improving object detection accuracy from the perspective of changing receptive fields. The structure of the head part of YOLOv5 is modified by adding asymmetrical pooling layers. The performances of the new model in this article are compared with original YOLOv5 model and analyzed from several parameters.
  arXiv  Detail & Related papers  (2023-03-15T23:59:18Z)
• Hybrid Optimized Deep Convolution Neural Network based Learning Model for Object Detection [0.0]
  Object identification is one of the most fundamental and difficult issues in computer vision. In recent years, deep learning-based object detection techniques have grabbed the public's interest. In this study, a unique deep learning classification technique is used to create an autonomous object detecting system. The suggested framework has a detection accuracy of 0.9864, which is greater than current techniques.
  arXiv  Detail & Related papers  (2022-03-02T04:39:37Z)
• Recent Trends in 2D Object Detection and Applications in Video Event Recognition [0.76146285961466]
  We discuss the pioneering works in object detection, followed by the recent breakthroughs that employ deep learning. We highlight recent datasets for 2D object detection both in images and videos, and present a comparative performance summary of various state-of-the-art object detection techniques.
  arXiv  Detail & Related papers  (2022-02-07T14:15:11Z)
• Contrastive Object Detection Using Knowledge Graph Embeddings [72.17159795485915]
  We compare the error statistics of the class embeddings learned from a one-hot approach with semantically structured embeddings from natural language processing or knowledge graphs. We propose a knowledge-embedded design for keypoint-based and transformer-based object detection architectures.
  arXiv  Detail & Related papers  (2021-12-21T17:10:21Z)
• You Better Look Twice: a new perspective for designing accurate detectors with reduced computations [56.34005280792013]
  BLT-net is a new low-computation two-stage object detection architecture. It reduces computations by separating objects from background using a very lite first-stage. Resulting image proposals are then processed in the second-stage by a highly accurate model.
  arXiv  Detail & Related papers  (2021-07-21T12:39:51Z)
• Few-Shot Learning for Video Object Detection in a Transfer-Learning Scheme [70.45901040613015]
  We study the new problem of few-shot learning for video object detection. We employ a transfer-learning framework to effectively train the video object detector on a large number of base-class objects and a few video clips of novel-class objects.
  arXiv  Detail & Related papers  (2021-03-26T20:37:55Z)
• A Simple and Effective Use of Object-Centric Images for Long-Tailed Object Detection [56.82077636126353]
  We take advantage of object-centric images to improve object detection in scene-centric images. We present a simple yet surprisingly effective framework to do so. Our approach can improve the object detection (and instance segmentation) accuracy of rare objects by 50% (and 33%) relatively.
  arXiv  Detail & Related papers  (2021-02-17T17:27:21Z)
• Learning Object Detection from Captions via Textual Scene Attributes [70.90708863394902]
  We argue that captions contain much richer information about the image, including attributes of objects and their relations. We present a method that uses the attributes in this "textual scene graph" to train object detectors. We empirically demonstrate that the resulting model achieves state-of-the-art results on several challenging object detection datasets.
  arXiv  Detail & Related papers  (2020-09-30T10:59:20Z)
• Object Detection for Graphical User Interface: Old Fashioned or Deep Learning or a Combination? [21.91118062303175]
  We conduct the first large-scale empirical study of seven representative GUI element detection methods on over 50k GUI images. This study sheds the light on the technical challenges to be addressed and informs the design of new GUI element detection methods. Our evaluation on 25,000 GUI images shows that our method significantly advances the start-of-the-art performance in GUI element detection.
  arXiv  Detail & Related papers  (2020-08-12T06:36:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.