SENSEVAL-2 (original) (raw)
SENSEVAL-2: Second International Workshop on Evaluating Word Sense Disambiguation Systems
5-6 July 2001, Toulouse, France
An ACL SIGLEX Event
Also supported by EURALEX, ELSNET, EPSRC (grant GR/RO2337/01), and ELRA
There are now many computer programs for automatically determining the sense of a word in context (Word Sense Disambiguation or WSD). The purpose of SENSEVAL is to evaluate the strengths and weaknesses of such programs with respect to different words, different varieties of language, and different languages.
The first SENSEVAL took place in the summer of 1998 for English, French, and Italian, culminating in a workshop held at Herstmonceux Castle, Sussex, England on September 2-4.
VISIT THE MAIN SENSEVAL WEBSITE
MAILING LIST
Join the SENSEVAL Mailing List: http://listserv.hum.gu.se/mailman/listinfo/senseval-discuss
SENSEVAL-2 DATA
Data (system answers, scores, training and testing corpora, scoring software)
SENSEVAL-2 COMPETITION
SENSEVAL-2 is now over!
We evaluated word sense disambiguation systems on three types of task over 12 languages. In the "all-words" task, the evaluation is on almost all of the content words in a sample of texts. In the "lexical sample" task, first we sample the lexicon, then we find instances in context of the sample words and the evaluation is on those instances only. In the "translation task" (Japanese only), senses corresponded to distinct translations of a word into another language. The tasks were
| All-words | Czech, Dutch, English, Estonian |
|---|---|
| Lexical sample | Basque, Chinese, Danish, English, Italian, Japanese, Korean, Spanish, Swedish |
| Translation | Japanese |
About 35 teams participated, submitting over 90 systems. The review of the workshop gives more statistics and some followup information. We will be publishing a proceedings of the workshop later this year (free to workshop attendees, otherwise available through the ACL). All of the results of the evaluation and data is now in the public domain:
- Results (system scores and graphs)
- Data (system answers, scores, training and testing corpora, scoring software)
- System descriptions
- Task descriptions (original descriptions; now possibly out-of-date)
- Workshop program
- Archive
A related workshop, Word Sense Disambiguation: Recent Successes and Future Directions, was held Thursday 11 July 2002, Philadelphia, USA. (at ACL-02)
SENSEVAL-2 ORGANIZATION COMMITTEE
Scott Cotton, University of Pennsylvania
Phil Edmonds, Sharp Laboratories of Europe
Adam Kilgarriff, ITRI, University of Brighton
Martha Palmer, University of Pennsylvania
SENSEVAL-2 Website: http://www.sle.sharp.co.uk/senseval2
last updated: 14 November, 2002 17:45 -0000