Image-based Freeform Handwriting Authentication with Energy-oriented Self-Supervised Learning
- URL: http://arxiv.org/abs/2408.09676v1
- Date: Mon, 19 Aug 2024 03:33:39 GMT
- Title: Image-based Freeform Handwriting Authentication with Energy-oriented Self-Supervised Learning
- Authors: Jingyao Wang, Luntian Mou, Changwen Zheng, Wen Gao,
- Abstract summary: SherlockNet is an energy-oriented two-branch contrastive self-supervised learning framework for robust and fast freeform handwriting authentication.
We construct EN-HA, a novel dataset that simulates data forgery and severe damage in real applications.
- Score: 17.584355583447323
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Freeform handwriting authentication verifies a person's identity from their writing style and habits in messy handwriting data. This technique has gained widespread attention in recent years as a valuable tool for various fields, e.g., fraud prevention and cultural heritage protection. However, it still remains a challenging task in reality due to three reasons: (i) severe damage, (ii) complex high-dimensional features, and (iii) lack of supervision. To address these issues, we propose SherlockNet, an energy-oriented two-branch contrastive self-supervised learning framework for robust and fast freeform handwriting authentication. It consists of four stages: (i) pre-processing: converting manuscripts into energy distributions using a novel plug-and-play energy-oriented operator to eliminate the influence of noise; (ii) generalized pre-training: learning general representation through two-branch momentum-based adaptive contrastive learning with the energy distributions, which handles the high-dimensional features and spatial dependencies of handwriting; (iii) personalized fine-tuning: calibrating the learned knowledge using a small amount of labeled data from downstream tasks; and (iv) practical application: identifying individual handwriting from scrambled, missing, or forged data efficiently and conveniently. Considering the practicality, we construct EN-HA, a novel dataset that simulates data forgery and severe damage in real applications. Finally, we conduct extensive experiments on six benchmark datasets including our EN-HA, and the results prove the robustness and efficiency of SherlockNet.
Related papers
- Text2Data: Low-Resource Data Generation with Textual Control [104.38011760992637]
Natural language serves as a common and straightforward control signal for humans to interact seamlessly with machines.
We propose Text2Data, a novel approach that utilizes unlabeled data to understand the underlying data distribution through an unsupervised diffusion model.
It undergoes controllable finetuning via a novel constraint optimization-based learning objective that ensures controllability and effectively counteracts catastrophic forgetting.
arXiv Detail & Related papers (2024-02-08T03:41:39Z) - Offline Detection of Misspelled Handwritten Words by Convolving
Recognition Model Features with Text Labels [0.0]
We introduce the task of comparing a handwriting image to text.
Our model's classification head is trained entirely on synthetic data created using a state-of-the-art generative adversarial network.
Such massive performance gains can lead to significant productivity increases in applications utilizing human-in-the-loop automation.
arXiv Detail & Related papers (2023-09-18T21:13:42Z) - Independent Distribution Regularization for Private Graph Embedding [55.24441467292359]
Graph embeddings are susceptible to attribute inference attacks, which allow attackers to infer private node attributes from the learned graph embeddings.
To address these concerns, privacy-preserving graph embedding methods have emerged.
We propose a novel approach called Private Variational Graph AutoEncoders (PVGAE) with the aid of independent distribution penalty as a regularization term.
arXiv Detail & Related papers (2023-08-16T13:32:43Z) - CSSL-RHA: Contrastive Self-Supervised Learning for Robust Handwriting
Authentication [23.565017967901618]
We propose a novel Contrastive Self-Supervised Learning framework for Robust Handwriting Authentication.
It can dynamically learn complex yet important features and accurately predict writer identities.
Our proposed model can still effectively achieve authentication even under abnormal circumstances, such as data falsification and corruption.
arXiv Detail & Related papers (2023-07-18T02:20:46Z) - SURDS: Self-Supervised Attention-guided Reconstruction and Dual Triplet
Loss for Writer Independent Offline Signature Verification [16.499360910037904]
Offline Signature Verification (OSV) is a fundamental biometric task across various forensic, commercial and legal applications.
We propose a two-stage deep learning framework that leverages self-supervised representation learning as well as metric learning for writer-independent OSV.
The proposed framework has been evaluated on two publicly available offline signature datasets and compared with various state-of-the-art methods.
arXiv Detail & Related papers (2022-01-25T07:26:55Z) - Transformer-Based Approach for Joint Handwriting and Named Entity
Recognition in Historical documents [1.7491858164568674]
This work presents the first approach that adopts the transformer networks for named entity recognition in handwritten documents.
We achieve the new state-of-the-art performance in the ICDAR 2017 Information Extraction competition using the Esposalles database.
arXiv Detail & Related papers (2021-12-08T09:26:21Z) - Vectorization and Rasterization: Self-Supervised Learning for Sketch and
Handwriting [168.91748514706995]
We propose two novel cross-modal translation pre-text tasks for self-supervised feature learning: Vectorization and Rasterization.
Our learned encoder modules benefit both-based and vector-based downstream approaches to analysing hand-drawn data.
arXiv Detail & Related papers (2021-03-25T09:47:18Z) - Detecting Ongoing Events Using Contextual Word and Sentence Embeddings [110.83289076967895]
This paper introduces the Ongoing Event Detection (OED) task.
The goal is to detect ongoing event mentions only, as opposed to historical, future, hypothetical, or other forms or events that are neither fresh nor current.
Any application that needs to extract structured information about ongoing events from unstructured texts can take advantage of an OED system.
arXiv Detail & Related papers (2020-07-02T20:44:05Z) - Fairness by Learning Orthogonal Disentangled Representations [50.82638766862974]
We propose a novel disentanglement approach to invariant representation problem.
We enforce the meaningful representation to be agnostic to sensitive information by entropy.
The proposed approach is evaluated on five publicly available datasets.
arXiv Detail & Related papers (2020-03-12T11:09:15Z) - Learning Not to Learn in the Presence of Noisy Labels [104.7655376309784]
We show that a new class of loss functions called the gambler's loss provides strong robustness to label noise across various levels of corruption.
We show that training with this loss function encourages the model to "abstain" from learning on the data points with noisy labels.
arXiv Detail & Related papers (2020-02-16T09:12:27Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.