Bootstrapping and Rule-Based Model for Recognizing Vietnamese Named Entity (original) (raw)
Related papers
Construction of a Vietnamese Corpora for Named Entity Recognition
2007
In order to build an automatic named entity recognition (NER) system using a machine learning approach, a large tagged corpus is widely seen as one necessary knowledge resource. Nevertheless, manual construction is time consuming, labor intensive and expensive. Building NER corpora for European languages has been extensively studied while some less-studied languages such as Vietnamese have not yet received much attention. This paper describes construction of a Vietnamese corpus, Vietnamese guidelines for annotators and a tagging tool that we make publicly available. We report on a comparison with the English named entity (NE) corpus in our multilingual NER system.
CONSTRUCTION OF VIETNAMESE CORPORA FOR NAMED ENTITY RECOGNITION
In order to build an automatic named entity recognition (NER) system using a machine learning approach, a large tagged corpus is widely seen as one necessary knowledge resource. Nevertheless, manual construction is time consuming, labor intensive and expensive. Building NER corpora for European languages has been extensively studied while some less-studied languages such as Vietnamese have not yet received much attention. This paper describes construction of a Vietnamese corpus, Vietnamese guidelines for annotators and a tagging tool that we make publicly available. We report on a comparison with the English named entity (NE) corpus in our multilingual NER system.
Named Entity Recognition for Vietnamese
Named Entity Recognition is an important task but is still relatively new for Vietnamese. It is partly due to the lack of a large annotated corpus. In this paper, we present a systematic approach in building a named entity annotated corpus while at the same time building rules to recognize Vietnamese named entities. The resulting open source system achieves an F-measure of 83%, which is better compared to existing Vietnamese NER systems.
Named entity recognition in Vietnamese documents
2007
Named Entity Recognition (NER) aims to classify words in a document into pre-defined target entity classes and is now considered to be fundamental for many natural language processing tasks such as information retrieval, machine translation, information extraction and question answering. This paper presents the results of an experiment in which a Support Vector Machine (SVM) based NER model is applied to the Vietnamese language. Though this state of the art machine learning method has been widely applied to NER in several well-studied languages, this is the first time this method has been applied to Vietnamese. In a comparison against Conditional Random Fields (CRFs) the SVM model was shown to outperform CRF by optimizing its feature window size, obtaining an overall F-score of 87.75. The paper also presents a detailed discussion about the characteristics of the Vietnamese language and provides an analysis of the factors which influence performance in this task.
A Rule-Based Named-Entity Recognition for Malay Articles
Lecture Notes in Computer Science, 2013
A Named-Entity Recognition (NER) is part of the process in Text Mining used for information extraction. This NER tool can be used to assist user in identifying and detecting entities such as person, location or organization. Different languages may have different morphologies and thus require different NER processes. For instance, an English NER process cannot be applied in processing Malay articles due to the different morphology used in different languages. This paper proposes a Rule-Based Named-Entity Recognition algorithm for Malay articles. The proposed Malay NER is designed based on a Malay part-of-speech (POS) tagging features and contextual features that had been implemented to handle Malay articles. Based on the POS results, proper names will be identified or detected as the possible candidates for annotation. Besides that, there are some symbols and conjunctions that will also be considered in the process of identifying named-entity for Malay articles. Several manually constructed dictionaries will be used to handle three named-entities; Person, Location and Organizations. The experimental results show a reasonable output of 89.47% for the F-Measure value. The proposed Malay NER algorithm can be further improved by having more complete dictionaries and refined rules to be used in order to identify the correct Malay entities system.
Malay Named Entity Recognition Based on Rule-Based Approach
A Named-Entity Recognition (NER) is part of the process in Text Mining and it is a very useful process for information extraction. This NER tool can be used to assist user in identifying and detecting entities such as person, location or organization. However, different languages may have different morphologies and thus require different NER processes. For instance, an English NER process cannot be applied in processing Malay articles due to the different morphology used in different languages. This paper proposes a Rule-Based Named-Entity Recognition algorithm for Malay articles. The proposed Malay NER is designed based on a Malay part-of-speech (POS) tagging features and contextual features that had been implemented to handle Malay articles. Based on the POS results, proper names will be identified or detected as the possible candidates for annotation. Besides that, there are some symbols and conjunctions that will also be considered in the process of identifying named-entity for Malay articles. Several manually constructed dictionaries will be used to handle three named-entities; Person, Location and Organizations. The experimental results show a reasonable output of 89.47% for the F-Measure value. The proposed Malay NER algorithm can be further improved by having more complete dictionaries and refined rules to be used in order to identify the correct Malay entities system.
Application of named entity recognition method for Indonesian datasets: a review
Bulletin of Electrical Engineering and Informatics, 2023
A name entity (NE) is a proper name that designates a person, location, or organization. For humans, named entity recognition (NER) is a straightforward process insofar as many named entities are self-names, and most of them have initial capital letters and can be easily recognized, but it is very difficult for machines. This study discusses research trends in the application of NER to Indonesian datasets, particularly as it concerns certain tasks, datasets, methods/techniques, and entity labels. By conducting a systematic literature review (SLR) and bibliometric analysis with VOSviewer, this article hopes to provide opportunities for adopting old methods, combining models from previous research, and even proposing new methods. In addition, the motivation for doing SLR at NER is to look for new strategies in the supervision of financial technology (Fintech). If machines can find illegal Fintech entities on social media and online news, it can help the government to block these illegal Fintech entities. To this end, this study provides an overview of research trends in applying the NER method to Bahasa Indonesia (Indonesian) datasets, including the extraction of news articles, the monitoring of floods, and traffic.
A Concise Review of Named Entity Recognition System: Methods and Features
IOP Conference Series: Materials Science and Engineering
Named Entity Recognition (NER) is an elementary tool for all application areas in Natural Language Processing (NLP) such as Automatic Summarization, Information Extraction, Information Retrieval, Text Mining, Machine Translation, Question Answering, and Genetics. NER is a task to discover and categorises the named entities ('atomic elements') in the text into predefined classes such as the names of persons, organizations, locations, terminologies of time, quantity and etc. Different languages may have different morphologies and thus involve dissimilar NER procedures. For example, an Arabic NER system cannot be practically used in processing Malay texts due to the different morphological features. The morphological features of every language are rich and complex and donates to the difficulties of implementing an actual method to develop the accurate NER system. In this paper, we review on three main techniques that commonly used to develop an NER system well-known as Rule-Based, Machine Learning, and Hybrid approach. This paper also highlights the features of each technique.
Advances in Knowledge Discovery …, 2011
Named entity recognition (NER) is the process of seeking to locate atomic elements in text into predefined categories such as the names of persons, organizations and locations. Most existing NER systems are based on supervised learning. This method often requires a large amount of labelled training data, which is very time-consuming to build. To solve this problem, we introduce a semi-supervised learning method for recognizing named entities in Vietnamese text by combining proper name coreference, named-ambiguity heuristics with a powerful sequential learning model, Conditional Random Fields. Our approach inherits the idea of Liao and Veeramachaneni [6] and expands it by using proper name coreference. Starting by training the model using a small data set that is annotated manually, the learning model extracts high confident named entities and finds low confident ones by using proper name coreference rules. The low confident named entities are put in the training set to learn new context features. The F-scores of the system for extracting "Person", "Location" and "Organization" entities are 83.36%, 69.53% and 65.71% when applying heuristics proposed by Liao and Veeramachaneni. Those values when using our proposed heuristics are 93.13%, 88.15% and 79.35%, respectively. It shows that our method is good in increasing the system accuracy.
NAMED ENTITY RECOGNITION USING WEB DOCUMENT CORPUS
This paper introduces a named entity recognition approach in textual corpus. This Named Entity (NE) can be a named: location, person, organization, date, time, etc., characterized by instances. A NE is found in texts accompanied by contexts: words that are left or right of the NE. The work mainly aims at identifying contexts inducing the NE's nature. As such, The occurrence of the word "President" in a text, means that this word or context may be followed by the name of a president as President "Obama". Likewise, a word preceded by the string "footballer" induces that this is the name of a footballer. NE recognition may be viewed as a classification method, where every word is assigned to a NE class, regarding the context. The aim of this study is then to identify and classify the contexts that are most relevant to recognize a NE, those which are frequently found with the NE. A learning approach using training corpus: web documents, constructed from learning examples is then suggested. Frequency representations and modified tf-idf representations are used to calculate the context weights associated to context frequency, learning example frequency, and document frequency in the corpus.