Combining AI and AM - Improving Approximate Matching through Transformer Networks

• URL: http://arxiv.org/abs/2208.11367v3
• Date: Thu, 27 Apr 2023 15:43:32 GMT
• Title: Combining AI and AM - Improving Approximate Matching through Transformer Networks
• Abstract summary: Approximate matching (AM) is a concept in digital forensics to determine the similarity between digital artifacts. We propose an improved matching algorithm based on transformer models from the field of natural language processing.
• Score: 12.466612431413711
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Approximate matching (AM) is a concept in digital forensics to determine the similarity between digital artifacts. An important use case of AM is the reliable and efficient detection of case-relevant data structures on a blacklist, if only fragments of the original are available. For instance, if only a cluster of indexed malware is still present during the digital forensic investigation, the AM algorithm shall be able to assign the fragment to the blacklisted malware. However, traditional AM functions like TLSH and ssdeep fail to detect files based on their fragments if the presented piece is relatively small compared to the overall file size. A second well-known issue with traditional AM algorithms is the lack of scaling due to the ever-increasing lookup databases. We propose an improved matching algorithm based on transformer models from the field of natural language processing. We call our approach Deep Learning Approximate Matching (DLAM). As a concept from artificial intelligence (AI), DLAM gets knowledge of characteristic blacklisted patterns during its training phase. Then DLAM is able to detect the patterns in a typically much larger file, that is DLAM focuses on the use case of fragment detection. We reveal that DLAM has three key advantages compared to the prominent conventional approaches TLSH and ssdeep. First, it makes the tedious extraction of known to be bad parts obsolete, which is necessary until now before any search for them with AM algorithms. This allows efficient classification of files on a much larger scale, which is important due to exponentially increasing data to be investigated. Second, depending on the use case, DLAM achieves a similar or even significantly higher accuracy in recovering fragments of blacklisted files. Third, we show that DLAM enables the detection of file correlations in the output of TLSH and ssdeep even for small fragment sizes.

Related papers

• Attribution-Guided Multimodal Deepfake Detection via Cross-Modal Forensic Fingerprints [4.598350938853635]
  We argue that a detector incapable of identifying how a video was forged is likely learning the wrong signal.<n>Unlike binary detection, attribution-guided learning imposes a stronger geometric constraint on the shared embedding space.<n>We propose the Attribution-Guided Multimodal Deepfake Detection framework, which jointly learns to detect and attribute manipulation.
  arXiv  Detail & Related papers  (2026-04-29T09:11:13Z)
• Leave No TRACE: Black-box Detection of Copyrighted Dataset Usage in Large Language Models via Watermarking [51.74368870268278]
  We propose TRACE, a framework for fully black-box detection of copyrighted dataset usage in large language models.<n>textttTRACE rewrites datasets with distortion-free watermarks guided by a private key.<n>Across diverse datasets and model families, TRACE consistently achieves significant detections.
  arXiv  Detail & Related papers  (2025-10-03T12:53:02Z)
• Pay Less Attention to Deceptive Artifacts: Robust Detection of Compressed Deepfakes on Online Social Networks [81.21729774122554]
  Existing deepfake detection methods overlook the block effects" introduced by compression in Online Social Networks (OSNs)<n>We propose PLADA, a novel framework designed to tackle the lack of paired data and the ineffective use of compressed images.<n>PLADA consists of two core modules: Block Effect Eraser (B2E), which uses a dual-stage attention mechanism to handle block effects, and Open Data Aggregation (ODA), which processes both paired and unpaired data to improve detection.
  arXiv  Detail & Related papers  (2025-06-25T15:46:41Z)
• Deep Homography Estimation for Visual Place Recognition [49.235432979736395]
  We propose a transformer-based deep homography estimation (DHE) network. It takes the dense feature map extracted by a backbone network as input and fits homography for fast and learnable geometric verification. Experiments on benchmark datasets show that our method can outperform several state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-02-25T13:22:17Z)
• Hidden Markov Models with Random Restarts vs Boosting for Malware Detection [5.414308305392762]
  We compare boosted HMMs (using AdaBoost) to HMMs trained with multiple random restarts, in the context of malware detection. We find that random restarts perform surprisingly well in comparison to boosting.
  arXiv  Detail & Related papers  (2023-07-17T13:21:58Z)
• DOAD: Decoupled One Stage Action Detection Network [77.14883592642782]
  Localizing people and recognizing their actions from videos is a challenging task towards high-level video understanding. Existing methods are mostly two-stage based, with one stage for person bounding box generation and the other stage for action recognition. We present a decoupled one-stage network dubbed DOAD, to improve the efficiency for-temporal action detection.
  arXiv  Detail & Related papers  (2023-04-01T08:06:43Z)
• Itemset Utility Maximization with Correlation Measure [8.581840054840335]
  High utility itemset mining (HUIM) is used to find out interesting but hidden information (e.g., profit and risk) In this paper, we propose a novel algorithm called the Itemset Utility Maximization with Correlation Measure (CoIUM) Two upper bounds and four pruning strategies are utilized to effectively prune the search space. And a concise array-based structure named utility-bin is used to calculate and store the adopted upper bounds in linear time and space.
  arXiv  Detail & Related papers  (2022-08-26T10:06:24Z)
• Using Convolutional Neural Networks to Detect Compression Algorithms [0.0]
  We use a base dataset, compressed every file with various algorithms, and designed a model based on that. The used model was accurately able to identify files compressed using compress, lzip and bzip2.
  arXiv  Detail & Related papers  (2021-11-17T11:03:16Z)
• MD-CSDNetwork: Multi-Domain Cross Stitched Network for Deepfake Detection [80.83725644958633]
  Current deepfake generation methods leave discriminative artifacts in the frequency spectrum of fake images and videos. We present a novel approach, termed as MD-CSDNetwork, for combining the features in the spatial and frequency domains to mine a shared discriminative representation.
  arXiv  Detail & Related papers  (2021-09-15T14:11:53Z)
• On the Importance of Encrypting Deep Features [15.340540198612823]
  We analyze model inversion attacks with only two assumptions: feature vectors of user data are known, and a black-box API for inference is provided. Experiments have been conducted on state-of-the-art models in person re-identification, and two attack scenarios (i.e., recognizing auxiliary attributes and reconstructing user data) are investigated. Results show that an adversary could successfully infer sensitive information even under severe constraints.
  arXiv  Detail & Related papers  (2021-08-16T15:22:33Z)
• Comprehensive Graph-conditional Similarity Preserving Network for Unsupervised Cross-modal Hashing [97.44152794234405]
  Unsupervised cross-modal hashing (UCMH) has become a hot topic recently. In this paper, we devise a deep graph-neighbor coherence preserving network (DGCPN) DGCPN regulates comprehensive similarity preserving losses by exploiting three types of data similarities.
  arXiv  Detail & Related papers  (2020-12-25T07:40:59Z)
• EnCoD: Distinguishing Compressed and Encrypted File Fragments [0.9239657838690228]
  We show that current approaches cannot reliably tell apart encryption and compression, even for large fragment sizes. We design EnCoD, a learning-based classifier which can reliably distinguish compressed and encrypted data, starting with fragments as small as 512 bytes. We evaluate EnCoD against current approaches over a large dataset of different data types, showing that it outperforms current state-of-the-art for most considered fragment sizes and data types.
  arXiv  Detail & Related papers  (2020-10-15T13:55:55Z)
• Saliency Enhancement using Gradient Domain Edges Merging [65.90255950853674]
  We develop a method to merge the edges with the saliency maps to improve the performance of the saliency. This leads to our proposed saliency enhancement using edges (SEE) with an average improvement of at least 3.4 times higher on the DUT-OMRON dataset. The SEE algorithm is split into 2 parts, SEE-Pre for preprocessing and SEE-Post pour postprocessing.
  arXiv  Detail & Related papers  (2020-02-11T14:04:56Z)
• Depthwise Non-local Module for Fast Salient Object Detection Using a Single Thread [136.2224792151324]
  We propose a new deep learning algorithm for fast salient object detection. The proposed algorithm achieves competitive accuracy and high inference efficiency simultaneously with a single CPU thread.
  arXiv  Detail & Related papers  (2020-01-22T15:23:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.