Word embedding and neural network on grammatical gender -- A case study of Swedish

• URL: http://arxiv.org/abs/2007.14222v1
• Date: Tue, 28 Jul 2020 13:50:17 GMT
• Title: Word embedding and neural network on grammatical gender -- A case study of Swedish
• Authors: Marc Allassonni\`ere-Tang and Ali Basirat
• Abstract summary: We show how the information about grammatical gender in language can be captured by word embedding models and artificial neural networks. We analyze the errors made by the computational model from a linguistic perspective.
• Score: 0.5243215690489517
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We analyze the information provided by the word embeddings about the grammatical gender in Swedish. We wish that this paper may serve as one of the bridges to connect the methods of computational linguistics and general linguistics. Taking nominal classification in Swedish as a case study, we first show how the information about grammatical gender in language can be captured by word embedding models and artificial neural networks. Then, we match our results with previous linguistic hypotheses on assignment and usage of grammatical gender in Swedish and analyze the errors made by the computational model from a linguistic perspective.

Related papers

• What an Elegant Bridge: Multilingual LLMs are Biased Similarly in Different Languages [51.0349882045866]
  This paper investigates biases of Large Language Models (LLMs) through the lens of grammatical gender. We prompt a model to describe nouns with adjectives in various languages, focusing specifically on languages with grammatical gender. We find that a simple classifier can not only predict noun gender above chance but also exhibit cross-language transferability.
  arXiv  Detail & Related papers  (2024-07-12T22:10:16Z)
• Learning Phonotactics from Linguistic Informants [54.086544221761486]
  Our model iteratively selects or synthesizes a data-point according to one of a range of information-theoretic policies. We find that the information-theoretic policies that our model uses to select items to query the informant achieve sample efficiency comparable to, or greater than, fully supervised approaches.
  arXiv  Detail & Related papers  (2024-05-08T00:18:56Z)
• Investigating grammatical abstraction in language models using few-shot learning of novel noun gender [0.0]
  We conduct a noun learning experiment to assess whether an LSTM and a decoder-only transformer can achieve human-like abstraction of grammatical gender in French. We find that both language models effectively generalise novel noun gender from one to two learning examples and apply the learnt gender across agreement contexts. While the generalisation behaviour of models suggests that they represent grammatical gender as an abstract category, like humans, further work is needed to explore the details.
  arXiv  Detail & Related papers  (2024-03-15T14:25:59Z)
• Using Artificial French Data to Understand the Emergence of Gender Bias in Transformer Language Models [5.22145960878624]
  This work takes an initial step towards exploring the less researched topic of how neural models discover linguistic properties of words, such as gender, as well as the rules governing their usage. We propose to use an artificial corpus generated by a PCFG based on French to precisely control the gender distribution in the training data and determine under which conditions a model correctly captures gender information or, on the contrary, appears gender-biased.
  arXiv  Detail & Related papers  (2023-10-24T14:08:37Z)
• Measuring Harmful Representations in Scandinavian Language Models [14.895663939509634]
  We show that Scandinavian pre-trained language models contain harmful and gender-based stereotypes. This finding goes against the general expectations related to gender equality in Scandinavian countries.
  arXiv  Detail & Related papers  (2022-11-21T17:46:39Z)
• Measuring Gender Bias in Word Embeddings of Gendered Languages Requires Disentangling Grammatical Gender Signals [3.0349733976070015]
  We demonstrate that word embeddings learn the association between a noun and its grammatical gender in grammatically gendered languages. We show that disentangling grammatical gender signals from word embeddings may lead to improvement in semantic machine learning tasks.
  arXiv  Detail & Related papers  (2022-06-03T17:11:00Z)
• Analyzing Gender Representation in Multilingual Models [59.21915055702203]
  We focus on the representation of gender distinctions as a practical case study. We examine the extent to which the gender concept is encoded in shared subspaces across different languages.
  arXiv  Detail & Related papers  (2022-04-20T00:13:01Z)
• A Massively Multilingual Analysis of Cross-linguality in Shared Embedding Space [61.18554842370824]
  In cross-lingual language models, representations for many different languages live in the same space. We compute a task-based measure of cross-lingual alignment in the form of bitext retrieval performance. We examine a range of linguistic, quasi-linguistic, and training-related features as potential predictors of these alignment metrics.
  arXiv  Detail & Related papers  (2021-09-13T21:05:37Z)
• Towards Zero-shot Language Modeling [90.80124496312274]
  We construct a neural model that is inductively biased towards learning human languages. We infer this distribution from a sample of typologically diverse training languages. We harness additional language-specific side information as distant supervision for held-out languages.
  arXiv  Detail & Related papers  (2021-08-06T23:49:18Z)
• An exploration of the encoding of grammatical gender in word embeddings [0.6461556265872973]
  The study of grammatical gender based on word embeddings can give insight into discussions on how grammatical genders are determined. It is found that there is an overlap in how grammatical gender is encoded in Swedish, Danish, and Dutch embeddings.
  arXiv  Detail & Related papers  (2020-08-05T06:01:46Z)
• Bridging Linguistic Typology and Multilingual Machine Translation with Multi-View Language Representations [83.27475281544868]
  We use singular vector canonical correlation analysis to study what kind of information is induced from each source. We observe that our representations embed typology and strengthen correlations with language relationships. We then take advantage of our multi-view language vector space for multilingual machine translation, where we achieve competitive overall translation accuracy.
  arXiv  Detail & Related papers  (2020-04-30T16:25:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.