Supervised Classification Based Machine Translation Quality Estimation (original) (raw)
Related papers
Analyzing Quality Estimation Of English-Hindi Machine Translation System
International Journal of Scientific & Technology Research, 2020
Automatically estimating the translation quality is a challenging topic of research in the field of MT. This paper describes the approach used for sentence level quality estimation problem on English-Hindi language pair. The purpose of the translation quality estimation (QE) is to predict a quality for unseen translated text without considering the reference translation. To perform the proposed technique, this submission conceived the quality estimation problem as a supervised learning approach. Feature extraction is an important step for supervised ML based quality estimation, and therefore, in this paper, we experimented with a set of multiple features along with the different ensemble type of learning algorithms. From the experimental results on the test set, we have found that Extra Tree based QE models gain improvements over the other two ensemble regressors. Moreover, the analysis of the performance evaluation measures show that the quality of the translation generated by the ...
The Prague Bulletin of Mathematical Linguistics, 2013
In this paper we present QE, an open source framework for machine translation quality estimation. The framework includes a feature extraction component and a machine learning component. We describe the architecture of the system and its use, focusing on the feature extraction component and on how to add new feature extractors. We also include experiments with features and learning algorithms available in the framework using the dataset of the WMT13 Quality Estimation shared task.
Quality estimation for machine translation output using linguistic analysis and decoding features
We describe a submission to the WMT12 Quality Estimation task, including an extensive Machine Learning experimentation. Data were augmented with features from linguistic analysis and statistical features from the SMT search graph. Several Feature Selection algorithms were employed. The Quality Estimation problem was addressed both as a regression task and as a discretised classification task, but the latter did not generalise well on the unseen testset. The most successful regression methods had an RMSE of 0.86 and were trained with a feature set given by Correlation-based Feature Selection. Indications that RMSE is not always sufficient for measuring performance were observed.
Estimating the Sentence-Level Quality of Machine Translation Systems
2009
We investigate the problem of predicting the quality of sentences produced by ma- chine translation systems when reference translations are not available. The prob- lem is addressed as a regression task and a method that takes into account the con- tribution of different features is proposed. We experiment with this method for trans- lations produced by various MT systems and
An investigation on the effectiveness of features for translation quality estimation
We describe a systematic analysis on the effectiveness of features commonly exploited for the problem of predicting machine translation quality. Using a feature selection technique based on Gaussian Processes, we identify small subsets of features that perform well across many datasets for different language pairs, text domains, machine translation systems and quality labels. In addition, we show the potential of the reduced feature sets resulting from our feature selection technique to lead to significantly better performance in most datasets, as compared to the complete feature sets.
This paper describes a set of experiments on two sub-tasks of Quality Estimation of Machine Translation (MT) output. Sentence-level ranking of alternative MT outputs is done with pairwise classi-fiers using Logistic Regression with black-box features originating from PCFG Parsing , language models and various counts. Post-editing time prediction uses regression models, additionally fed with new elaborate features from the Statistical MT decoding process. These seem to be better indicators of post-editing time than black-box features. Prior to training the models, feature scoring with ReliefF and Information Gain is used to choose feature sets of decent size and avoid computational complexity .
The UU Submission to the Machine Translation Quality Estimation Task
Proceedings of the First Conference on Machine Translation: Volume 2, Shared Task Papers, 2016
This paper outlines the UU-SVM system for Task 1 of the WMT16 Shared Task in Quality Estimation. Our system uses Support Vector Machine Regression to investigate the impact of a series of features aiming to convey translation quality. We propose novel features measuring reordering and noun translation errors. We show that we can outperform the baseline when we combine it with a subset of our new features.
An efficient and user-friendly tool for machine translation quality estimation
We present a new version of QUEST-an open source framework for machine translation quality estimation-which brings a number of improvements: (i) it provides a Web interface and functionalities such that non-expert users, e.g. translators or lay-users of machine translations, can get quality predictions (or internal features of the framework) for translations without having to install the toolkit, obtain resources or build prediction models; (ii) it significantly improves over the previous runtime performance by keeping resources (such as language models) in memory; (iii) it provides an option for users to submit the source text only and automatically obtain translations from Bing Translator; (iv) it provides a ranking of multiple translations submitted by users for each source text according to their estimated quality. We exemplify the use of this new version through some experiments with the framework.
2013
This paper describes a set of experiments on two sub-tasks of Quality Estimation of Machine Translation (MT) output. Sentence-level ranking of alternative MT outputs is done with pairwise classifiers using Logistic Regression with blackbox features originating from PCFG Parsing, language models and various counts. Post-editing time prediction uses regression models, additionally fed with new elaborate features from the Statistical MT decoding process. These seem to be better indicators of post-editing time than blackbox features. Prior to training the models, feature scoring with ReliefF and Information Gain is used to choose feature sets of decent size and avoid computational complexity.
This paper is to introduce our participation in the WMT13 shared tasks on Quality Estimation for machine translation without using reference translations. We submitted the results for Task 1.1 (sentence-level quality estimation), Task 1.2 (system selection) and Task 2 (word-level quality estimation). In Task 1.1, we used an enhanced version of BLEU metric without using reference translations to evaluate the translation quality. In Task 1.2, we utilized a probability model Naï ve Bayes (NB) as a classification algorithm with the features borrowed from the traditional evaluation metrics. In Task 2, to take the contextual information into account, we employed a discriminative undirected probabilistic graphical model Conditional random field (CRF), in addition to the NB algorithm. The training experiments on the past WMT corpora showed that the designed methods of this paper yielded promising results especially the statistical models of CRF and NB. The official results show that our CRF model achieved the highest F-score 0.8297 in binary classification of Task 2.