Maximum Entropy with Maximum J-Divergence Discrimination for TextClassification (original) (raw)

Classification Using Maximum Entropy

2006

In organizations, a large amount of information exists in text documents. Therefore, it is important to use text mining to discover knowledge from these unstructured data. Automatic text classification considered as one of important applications in text mining. It is the process of assigning a text document to one or more predefined categories based on their content. This paper focus on classifying Arabic text documents. Arabic language is highly inflectional and derivational language which makes text mining a complex task. In our approach, we first preprocessed data using natural language processing techniques such as tokenizing, stemming and partof-speech. Then, we used maximum entropy method to classify Arabic documents. We experimented our approach using real data, then we compared the results with other existing systems. صخلم : يصن لكشب ةدوجوم تامولعملا نم ريثكلا كانه تاسسؤملا يف . ةرـيبك ةيمهأ كانه كلذل عونلا اذه نم تانايبلا نع بيقنتلل . بيقنتلا لاجم يف ةمهملا تاقيبطتلا دحأ رب...

Generalised max entropy classifiers

International Conference on Belief Functions (BELIEF 2018), 2018

In this paper we propose a generalised maximum-entropy classification framework, in which the empirical expectation of the feature functions is bounded by the lower and upper expectations associated with the lower and upper probabilities associated with a belief measure. This generalised setting permits a more cautious appreciation of the information content of a training set. We analytically derive the Karush-Kuhn-Tucker conditions for the generalised max-entropy classifier in the case in which a Shannon-like entropy is adopted.

SIMILARITY BASED ENTROPY ON FEATURE SELECTION FOR HIGH DIMENSIONAL DATA CLASSIFICATION

Jurnal Ilmu Komputer dan Informasi, 2014

Curse of dimensionality is a major problem in most classification tasks. Feature transformation and feature selection as a feature reduction method can be applied to overcome this problem. Despite of its good performance, feature transformation is not easily interpretable because the physical meaning of the original features cannot be retrieved. On the other side, feature selection with its simple computational process is able to reduce unwanted features and visualize the data to facilitate data understanding. We propose a new feature selection method using similarity based entropy to overcome the high dimensional data problem. Using 6 datasets with high dimensional feature, we have computed the similarity between feature vector and class vector. Then we find the maximum similarity that can be used for calculating the entropy values of each feature. The selected features are features that having higher entropy than mean entropy of overall features. The fuzzy k-NN classifier was implemented to evaluate the selected features. The experiment result shows that proposed method is able to deal with high dimensional data problem with average accuracy of 80.5%.