Diff-Oracle: Deciphering Oracle Bone Scripts with Controllable Diffusion Model
- URL: http://arxiv.org/abs/2312.13631v2
- Date: Mon, 8 Jul 2024 07:34:29 GMT
- Title: Diff-Oracle: Deciphering Oracle Bone Scripts with Controllable Diffusion Model
- Authors: Jing Li, Qiu-Feng Wang, Siyuan Wang, Rui Zhang, Kaizhu Huang, Erik Cambria,
- Abstract summary: Deciphering oracle bone scripts plays an important role in Chinese archaeology and philology.
Diff-Oracle is a novel approach based on diffusion models to generate controllable oracle characters.
Diff-Oracle substantially benefits downstream oracle character recognition, outperforming all existing SOTAs by a large margin.
- Score: 48.956844881630886
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Deciphering oracle bone scripts plays an important role in Chinese archaeology and philology. However, a significant challenge remains due to the scarcity of oracle character images. To overcome this issue, we propose Diff-Oracle, a novel approach based on diffusion models to generate a diverse range of controllable oracle characters. Unlike traditional diffusion models that operate primarily on text prompts, Diff-Oracle incorporates a style encoder that utilizes style reference images to control the generation style. This encoder extracts style prompts from existing oracle character images, where style details are converted into a text embedding format via a pretrained language-vision model. On the other hand, a content encoder is integrated within Diff-Oracle to capture specific content details from content reference images, ensuring that the generated characters accurately represent the intended glyphs. To effectively train Diff-Oracle, we pre-generate pixel-level paired oracle character images (i.e., style and content images) by an image-to-image translation model. Extensive qualitative and quantitative experiments are conducted on datasets Oracle-241 and OBC306. While significantly surpassing present generative methods in terms of image generation, Diff-Oracle substantially benefits downstream oracle character recognition, outperforming all existing SOTAs by a large margin. In particular, on the challenging OBC306 dataset, Diff-Oracle leads to an accuracy gain of 7.70% in the zero-shot setting and is able to recognize unseen oracle character images with the accuracy of 84.62%, achieving a new benchmark for deciphering oracle bone scripts.
Related papers
- Oracle Bone Inscriptions Multi-modal Dataset [58.20314888996118]
Oracle bone inscriptions(OBI) is the earliest developed writing system in China, bearing invaluable written exemplifications of early Shang history and paleography.
This paper proposes an Oracle Bone Inscriptions Multi-modal dataset, which includes annotation information for 10,077 pieces of oracle bones.
This dataset can be used for a variety of AI-related research tasks relevant to the field of OBI, such as OBI Character Detection and Recognition, Rubbing Denoising, Character Matching, Character Generation, Reading Sequence Prediction, Missing Characters Completion task and so on.
arXiv Detail & Related papers (2024-07-04T12:47:32Z) - Heracles: A Hybrid SSM-Transformer Model for High-Resolution Image and Time-Series Analysis [23.511807886483087]
Heracles is a novel SSM that integrates a local SSM, a global SSM, and an attention-based token interaction module.
Heracles achieves state-of-the-art performance on the ImageNet dataset with 84.5% top-1 accuracy.
Heracles excels in transfer learning tasks on datasets such as CIFAR-10, CIFAR-100, Oxford Flowers, and Stanford Cars.
arXiv Detail & Related papers (2024-03-26T19:29:21Z) - An open dataset for the evolution of oracle bone characters: EVOBC [72.91231825135665]
The earliest extant Chinese characters originate from oracle bone inscriptions, which are closely related to other East Asian languages.
In this study, we systematically collected ancient characters from authoritative texts and websites spanning six historical stages.
We constructed an extensive dataset, consisting of 229,170 images representing 13,714 distinct character categories.
arXiv Detail & Related papers (2024-01-23T03:30:47Z) - Image Captioning with Multi-Context Synthetic Data [16.961112970612447]
Large models have excelled in producing high-quality images and text.
We present an innovative pipeline that introduces multi-context data generation.
Our model is exclusively trained on synthetic image-text pairs crafted through this process.
arXiv Detail & Related papers (2023-05-29T13:18:59Z) - Scaling Autoregressive Models for Content-Rich Text-to-Image Generation [95.02406834386814]
Parti treats text-to-image generation as a sequence-to-sequence modeling problem.
Parti uses a Transformer-based image tokenizer, ViT-VQGAN, to encode images as sequences of discrete tokens.
PartiPrompts (P2) is a new holistic benchmark of over 1600 English prompts.
arXiv Detail & Related papers (2022-06-22T01:11:29Z) - Oracle-MNIST: a Realistic Image Dataset for Benchmarking Machine
Learning Algorithms [57.29464116557734]
We introduce the Oracle-MNIST dataset, comprising of 28$times $28 grayscale images of 30,222 ancient characters.
The training set totally consists of 27,222 images, and the test set contains 300 images per class.
arXiv Detail & Related papers (2022-05-19T09:57:45Z) - Unsupervised Structure-Texture Separation Network for Oracle Character
Recognition [70.29024469395608]
Oracle bone script is the earliest-known Chinese writing system of the Shang dynasty and is precious to archeology and philology.
We propose a structure-texture separation network (STSN), which is an end-to-end learning framework for joint disentanglement, transformation, adaptation and recognition.
arXiv Detail & Related papers (2022-05-13T10:27:02Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.