Improving Gender Translation Accuracy with Filtered Self-Training (original) (raw)
Related papers
GFST: Gender-Filtered Self-Training for More Accurate Gender in Translation
Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 2021
Targeted evaluations have found that machine translation systems often output incorrect gender in translations, even when the gender is clear from context. Furthermore, these incorrectly gendered translations have the potential to reflect or amplify social biases. We propose gender-filtered self-training (GFST) to improve gender translation accuracy on unambiguously gendered inputs. Our GFST approach uses a source monolingual corpus and an initial model to generate gender-specific pseudo-parallel corpora which are then filtered and added to the training data. We evaluate GFST on translation from English into five languages, finding that it improves gender accuracy without damaging generic quality. We also show the viability of GFST on several experimental settings, including retraining from scratch, fine-tuning, controlling the gender balance of the data, forward translation, and back-translation. 1
Gender-Fair (Machine) Translation
Proceedings of the New Trends in Translation and Technology Conference - NeTTT 2022, 2023
Recent years have seen an increased visibility of non-binary people in public discourse. Accordingly, gender-fair language strategies that go beyond a binary conception of gender have been proposed. Such strategies pose a challenge for both translators and machine translation (MT), and gender-fair (machine) translation represents a relatively novel research field. With this survey and discussion, we hope to provide a starting point for this field and contribute a detailed overview of (machine) translation strategies to counteract the misrepresentation of an individual's gender. The results show that gender-fair translation studies (TS) approaches largely focus on media translation, such as subtitles or news articles, and the MT results show that the need to include non-binary debiasing methods is increasingly acknowledged, however, hardly ever implemented. Ideas on a closer mutually beneficial interaction between MT and translation studies are presented to advance multilingual gender-fair language use.
Getting Gender Right in Neural Machine Translation
Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing
Speakers of different languages must attend to and encode strikingly different aspects of the world in order to use their language correctly (Sapir, 1921; Slobin, 1996). One such difference is related to the way gender is expressed in a language. Saying "I am happy" in English, does not encode any additional knowledge of the speaker that uttered the sentence. However, many other languages do have grammatical gender systems and so such knowledge would be encoded. In order to correctly translate such a sentence into, say, French, the inherent gender information needs to be retained/recovered. The same sentence would become either "Je suis heureux", for a male speaker or "Je suis heureuse" for a female one. Apart from morphological agreement, demographic factors (gender, age, etc.) also influence our use of language in terms of word choices or even on the level of syntactic constructions (Tannen, 1991; Pennebaker et al., 2003). We integrate gender information into NMT systems. Our contribution is twofold: (1) the compilation of large datasets with speaker information for 20 language pairs, and (2) a simple set of experiments that incorporate gender information into NMT for multiple language pairs. Our experiments show that adding a gender feature to an NMT system significantly improves the translation quality for some language pairs.
arXiv (Cornell University), 2023
Gender bias is a significant issue in machine translation, leading to ongoing research efforts in developing bias mitigation techniques. However, most works focus on debiasing bilingual models without much consideration for multilingual systems. In this paper, we specifically target the gender bias issue of multilingual machine translation models for unambiguous cases where there is a single correct translation, and propose a bias mitigation method based on a novel approach. Specifically, we propose Gender-Aware Contrastive Learning, GACL, which encodes contextual gender information into the representations of non-explicit gender words. Our method is target languageagnostic and is applicable to pre-trained multilingual machine translation models via finetuning. Through multilingual evaluation, we show that our approach improves gender accuracy by a wide margin without hampering translation performance. We also observe that incorporated gender information transfers and benefits other target languages regarding gender accuracy. Finally, we demonstrate that our method is applicable and beneficial to models of various sizes. 1 * Work done during internship at MSRA.
A Prompt Response to the Demand for Automatic Gender-Neutral Translation
arXiv (Cornell University), 2024
Gender-neutral translation (GNT) that avoids biased and undue binary assumptions is a pivotal challenge for the creation of more inclusive translation technologies. Advancements for this task in Machine Translation (MT), however, are hindered by the lack of dedicated parallel data, which are necessary to adapt MT systems to satisfy neutral constraints. For such a scenario, large language models offer hitherto unforeseen possibilities, as they come with the distinct advantage of being versatile in various (sub)tasks when provided with explicit instructions. In this paper, we explore this potential to automate GNT by comparing MT with the popular GPT-4 model. Through extensive manual analyses, our study empirically reveals the inherent limitations of current MT systems in generating GNTs and provides valuable insights into the potential and challenges associated with prompting for neutrality. 7 We adopt the best performing prompt by Peng et al.
Investigating Failures of Automatic Translation in the Case of Unambiguous Gender
ArXiv, 2021
Transformer based models are the modern work horses for neural machine translation (NMT), reaching state of the art across several benchmarks. Despite their impressive accuracy, we observe a systemic and rudimentary class of errors made by transformer based models with regards to translating from a language that doesn’t mark gender on nouns into others that do. We find that even when the surrounding context provides unambiguous evidence of the appropriate grammatical gender marking, no transformer based model we tested was able to accurately gender occupation nouns systematically. We release an evaluation scheme and dataset for measuring the ability of transformer based NMT models to translate gender morphology correctly in unambiguous contexts across syntactically diverse sentences. Our dataset translates from an English source into 20 languages from several different language families. With the availability of this dataset, our hope is that the NMT community can iterate on solutio...
Gender Bias in Machine Translation
2021
Machine bias in Artificial Intelligence (AI) has the detrimental potential to cause perpetual gender inequality in today’s society. With AI rapidly becoming a source of gender bias in new technologies such as facial recognition and automated recruitment tools, the requirement of a fair and unbiased model has grown. Clearly, these biases stem from real-world stereotypes, showing how differently women are treated. This work sheds light on this issue with specific focus on gender bias in Machine Translation (MT). We hope this work will promote social awareness and lead to more conversations concerning machine bias.
Evaluating Gender Bias in Speech Translation
ArXiv, 2020
The scientific community is more and more aware of the necessity to embrace pluralism and consistently represent major and minor social groups. In this direction, there is an urgent need to provide evaluation sets and protocols to measure existing biases in our automatic systems. This paper introduces WinoST, a new freely available challenge set for evaluating gender bias in speech translation. WinoST is the speech version of WinoMT which is an MT challenge set and both follow an evaluation protocol to measure gender accuracy. Using a state-of-the-art end-to-end speech translation system, we report the gender bias evaluation on 4 language pairs, and we show that gender accuracy in speech translation is more than 23% lower than in MT.
How to Measure Gender Bias in Machine Translation: Optimal Translators, Multiple Reference Points
arXiv (Cornell University), 2020
In this paper-as a case study-we present a systematic study of gender bias in machine translation with Google Translate. We translated sentences containing names of occupations from Hungarian, a language with gender-neutral pronouns, into English. Our aim was to present a fair measure for bias by comparing the translations to an optimal non-biased translator. When assessing bias, we used the following reference points: (1) the distribution of men and women among occupations in both the source and the target language countries, as well as (2) the results of a Hungarian survey that examined if certain jobs are generally perceived as feminine or masculine. We also studied how expanding sentences with adjectives referring to occupations effect the gender of the translated pronouns. As a result, we found bias against both genders, but biased results against women are much more frequent. Translations are closer to our perception of occupations than to objective occupational statistics. Finally, occupations have a greater effect on translation than adjectives.