Emojich -- zero-shot emoji generation using Russian language: a technical report

• URL: http://arxiv.org/abs/2112.02448v1
• Date: Sat, 4 Dec 2021 23:37:32 GMT
• Title: Emojich -- zero-shot emoji generation using Russian language: a technical report
• Authors: Alex Shonenkov (1 and 2), Daria Bakshandaeva (1), Denis Dimitrov (1), Aleksandr Nikolich (1) ((1) Sber AI, (2) MIPT)
• Abstract summary: "Emojich" is a text-to-image neural network that generates emojis using captions in Russian language as a condition. We aim to keep the generalization ability of a pretrained big model ruDALL-E Malevich (XL) 1.3B parameters at the fine-tuning stage.
• Score: 52.77024349608834
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This technical report presents a text-to-image neural network "Emojich" that generates emojis using captions in Russian language as a condition. We aim to keep the generalization ability of a pretrained big model ruDALL-E Malevich (XL) 1.3B parameters at the fine-tuning stage, while giving special style to the images generated. Here are presented some engineering methods, code realization, all hyper-parameters for reproducing results and a Telegram bot where everyone can create their own customized sets of stickers. Also, some newly generated emojis obtained by "Emojich" model are demonstrated.

Related papers

• Conditional Text-to-Image Generation with Reference Guidance [81.99538302576302]
  This paper explores using additional conditions of an image that provides visual guidance of the particular subjects for diffusion models to generate. We develop several small-scale expert plugins that efficiently endow a Stable Diffusion model with the capability to take different references. Our expert plugins demonstrate superior results than the existing methods on all tasks, each containing only 28.55M trainable parameters.
  arXiv  Detail & Related papers  (2024-11-22T21:38:51Z)
• Unleashing the Power of Emojis in Texts via Self-supervised Graph Pre-Training [22.452853652070413]
  We release the emoji's power in social media data mining. We propose a graph pre-train framework for text and emoji co-modeling.
  arXiv  Detail & Related papers  (2024-09-22T18:29:10Z)
• Semantics Preserving Emoji Recommendation with Large Language Models [47.94761630160614]
  Existing emoji recommendation methods are primarily evaluated based on their ability to match the exact emoji a user chooses in the original text. We propose a new semantics preserving evaluation framework for emoji recommendation, which measures a model's ability to recommend emojis that maintain the semantic consistency with the user's text.
  arXiv  Detail & Related papers  (2024-09-16T22:27:46Z)
• EmojiLM: Modeling the New Emoji Language [44.23076273155259]
  We develop a text-emoji parallel corpus, Text2Emoji, from a large language model. Based on the parallel corpus, we distill a sequence-to-sequence model, EmojiLM, which is specialized in the text-emoji bidirectional translation. Our proposed model outperforms strong baselines and the parallel corpus benefits emoji-related downstream tasks.
  arXiv  Detail & Related papers  (2023-11-03T07:06:51Z)
• Emoji Prediction in Tweets using BERT [0.0]
  We propose a transformer-based approach for emoji prediction using BERT, a widely-used pre-trained language model. We fine-tuned BERT on a large corpus of text (tweets) containing both text and emojis to predict the most appropriate emoji for a given text. Our experimental results demonstrate that our approach outperforms several state-of-the-art models in predicting emojis with an accuracy of over 75 percent.
  arXiv  Detail & Related papers  (2023-07-05T06:38:52Z)
• Zero-Shot Video Captioning with Evolving Pseudo-Tokens [79.16706829968673]
  We introduce a zero-shot video captioning method that employs two frozen networks: the GPT-2 language model and the CLIP image-text matching model. The matching score is used to steer the language model toward generating a sentence that has a high average matching score to a subset of the video frames. Our experiments show that the generated captions are coherent and display a broad range of real-world knowledge.
  arXiv  Detail & Related papers  (2022-07-22T14:19:31Z)
• Emoji-aware Co-attention Network with EmoGraph2vec Model for Sentiment Anaylsis [9.447106020795292]
  We propose a method to learn emoji representations called EmoGraph2vec and design an emoji-aware co-attention network. Our model designs a co-attention mechanism to incorporate the text and emojis, and integrates a squeeze-and-excitation block into a convolutional neural network. Experimental results show that the proposed model can outperform several baselines for sentiment analysis on benchmark datasets.
  arXiv  Detail & Related papers  (2021-10-27T08:01:10Z)
• Semantic Journeys: Quantifying Change in Emoji Meaning from 2012-2018 [66.28665205489845]
  We offer the first longitudinal study of how emoji semantics changes over time, applying techniques from computational linguistics to six years of Twitter data. We identify five patterns in emoji semantic development and find evidence that the less abstract an emoji is, the more likely it is to undergo semantic change. To aid future work on emoji and semantics, we make our data publicly available along with a web-based interface that anyone can use to explore semantic change in emoji.
  arXiv  Detail & Related papers  (2021-05-03T13:35:10Z)
• Emoji Prediction: Extensions and Benchmarking [30.642840676899734]
  The emoji prediction task aims at predicting the proper set of emojis associated with a piece of text. We extend the existing setting of the emoji prediction task to include a richer set of emojis and to allow multi-label classification. We propose novel models for multi-class and multi-label emoji prediction based on Transformer networks.
  arXiv  Detail & Related papers  (2020-07-14T22:41:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.