Flexible image analysis for law enforcement agencies with deep neural networks to determine: where, who and what

• URL: http://arxiv.org/abs/2405.09194v1
• Date: Wed, 15 May 2024 09:02:17 GMT
• Title: Flexible image analysis for law enforcement agencies with deep neural networks to determine: where, who and what
• Authors: Henri Bouma, Bart Joosten, Maarten C Kruithof, Maaike H T de Boer, Alexandru Ginsca, Benjamin Labbe, Quoc T Vuong,
• Abstract summary: Law enforcement agencies (LEAs) are inspecting images and videos to findradicalization, propaganda for terrorist organizations and illegal products on darknet markets. Instead of an undirected search, LEAs would like to adapt to new crimes and threats, and focus only on data from specificlocations, persons or objects. Visual concept detection with deepconvolutional neural networks (CNNs) is a crucial component to understand the image content.
• Score: 36.136619420474766
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Due to the increasing need for effective security measures and the integration of cameras in commercial products, a hugeamount of visual data is created today. Law enforcement agencies (LEAs) are inspecting images and videos to findradicalization, propaganda for terrorist organizations and illegal products on darknet markets. This is time consuming.Instead of an undirected search, LEAs would like to adapt to new crimes and threats, and focus only on data from specificlocations, persons or objects, which requires flexible interpretation of image content. Visual concept detection with deepconvolutional neural networks (CNNs) is a crucial component to understand the image content. This paper has fivecontributions. The first contribution allows image-based geo-localization to estimate the origin of an image. CNNs andgeotagged images are used to create a model that determines the location of an image by its pixel values. The secondcontribution enables analysis of fine-grained concepts to distinguish sub-categories in a generic concept. The proposedmethod encompasses data acquisition and cleaning and concept hierarchies. The third contribution is the recognition ofperson attributes (e.g., glasses or moustache) to enable query by textual description for a person. The person-attributeproblem is treated as a specific sub-task of concept classification. The fourth contribution is an intuitive image annotationtool based on active learning. Active learning allows users to define novel concepts flexibly and train CNNs with minimalannotation effort. The fifth contribution increases the flexibility for LEAs in the query definition by using query expansion.Query expansion maps user queries to known and detectable concepts. Therefore, no prior knowledge of the detectableconcepts is required for the users. The methods are validated on data with varying locations (popular and non-touristiclocations), varying person attributes (CelebA dataset), and varying number of annotations.

Related papers

• Exploring Fine-Grained Representation and Recomposition for Cloth-Changing Person Re-Identification [78.52704557647438]
  We propose a novel FIne-grained Representation and Recomposition (FIRe$2$) framework to tackle both limitations without any auxiliary annotation or data. Experiments demonstrate that FIRe$2$ can achieve state-of-the-art performance on five widely-used cloth-changing person Re-ID benchmarks.
  arXiv  Detail & Related papers  (2023-08-21T12:59:48Z)
• Learning Transferable Pedestrian Representation from Multimodal Information Supervision [174.5150760804929]
  VAL-PAT is a novel framework that learns transferable representations to enhance various pedestrian analysis tasks with multimodal information. We first perform pre-training on LUPerson-TA dataset, where each image contains text and attribute annotations. We then transfer the learned representations to various downstream tasks, including person reID, person attribute recognition and text-based person search.
  arXiv  Detail & Related papers  (2023-04-12T01:20:58Z)
• Knowledge Mining with Scene Text for Fine-Grained Recognition [53.74297368412834]
  We propose an end-to-end trainable network that mines implicit contextual knowledge behind scene text image. We employ KnowBert to retrieve relevant knowledge for semantic representation and combine it with image features for fine-grained classification. Our method outperforms the state-of-the-art by 3.72% mAP and 5.39% mAP, respectively.
  arXiv  Detail & Related papers  (2022-03-27T05:54:00Z)
• Hierarchical Semantic Segmentation using Psychometric Learning [17.417302703539367]
  We develop a novel approach to collect segmentation annotations from experts based on psychometric testing. Our method consists of the psychometric testing procedure, active query selection, query enhancement, and a deep metric learning model. We show the merits of our method with evaluation on the synthetically generated image, aerial image and histology image.
  arXiv  Detail & Related papers  (2021-07-07T13:38:33Z)
• Data Augmentation for Object Detection via Differentiable Neural Rendering [71.00447761415388]
  It is challenging to train a robust object detector when annotated data is scarce. Existing approaches to tackle this problem include semi-supervised learning that interpolates labeled data from unlabeled data. We introduce an offline data augmentation method for object detection, which semantically interpolates the training data with novel views.
  arXiv  Detail & Related papers  (2021-03-04T06:31:06Z)
• Learning to Focus: Cascaded Feature Matching Network for Few-shot Image Recognition [38.49419948988415]
  Deep networks can learn to accurately recognize objects of a category by training on a large number of images. A meta-learning challenge known as a low-shot image recognition task comes when only a few images with annotations are available for learning a recognition model for one category. Our method, called Cascaded Feature Matching Network (CFMN), is proposed to solve this problem. Experiments for few-shot learning on two standard datasets, emphminiImageNet and Omniglot, have confirmed the effectiveness of our method.
  arXiv  Detail & Related papers  (2021-01-13T11:37:28Z)
• Multi-Modal Retrieval using Graph Neural Networks [1.8911962184174562]
  We learn a joint vision and concept embedding in the same high-dimensional space. We model the visual and concept relationships as a graph structure. We also introduce a novel inference time control, based on selective neighborhood connectivity.
  arXiv  Detail & Related papers  (2020-10-04T19:34:20Z)
• Contextual Semantic Interpretability [16.18912769522768]
  We look into semantic bottlenecks that capture context. We use a two-layer semantic bottleneck that gathers attributes into interpretable, sparse groups. Our model yields in predictions as accurate as a non-interpretable baseline when applied to a real-world test set of Flickr images.
  arXiv  Detail & Related papers  (2020-09-18T09:47:05Z)
• Understanding the Role of Individual Units in a Deep Neural Network [85.23117441162772]
  We present an analytic framework to systematically identify hidden units within image classification and image generation networks. First, we analyze a convolutional neural network (CNN) trained on scene classification and discover units that match a diverse set of object concepts. Second, we use a similar analytic method to analyze a generative adversarial network (GAN) model trained to generate scenes.
  arXiv  Detail & Related papers  (2020-09-10T17:59:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.