Conference paper Open Access
Reliable Part-of-Speech Tagging of Low-frequency Phenomena in the Social Media Domain
Horsmann, Tobias; Beißwenger, Michael; Zesch, Torsten
We present a series of experiments to fit a part-of-speech (PoS) tagger towards tagging extremely infrequent PoS tags of which we only have a limited amount of training data.
The objective is to implement a tagger that tags this phenomenon with a high degree of correctness in order to be able to use it as a corpus query tool on plain text corpora, so that new instances of this phenomenon can be easily found in plain text.
We focused on avoiding manual annotation as much as possible and experimented with altering the frequency weight of the PoS tag of interest in the small training data set we have.
This approach was compared to adding machine tagged training data in which only the phenomenon of interest is manually corrected.
We find that adding more training data is unavoidable but machine tagging data and hand correcting the tag of interest is sufficient.
Furthermore, the choice of the tagger plays an important role as some taggers are equipped to deal with rare phenomena more adequately than others.
The best trade off between precision and recall of the phenomenon of interest was achieved by a separation of the tagging into two steps
An evaluation of this phenomenon-fitted tagger on social media plain-text confirmed that the tagger serves as a useful corpus query tool that retrieves instances of the phenomenon including many unseen ones.
Files
(96.3 kB)
| Name | Size | |
|---|---|---|
|
cmccorpora17-4.pdf
md5:5b2ffa534454f3b287ab3a7dea1a7fb8 |
96.3 kB | Download |
69
59
views
downloads
| All versions | This version | |
|---|---|---|
| Views | 69 | 69 |
| Downloads | 59 | 59 |
| Data volume | 5.7 MB | 5.7 MB |
| Unique views | 52 | 52 |
| Unique downloads | 51 | 51 |
Indexed in
- Publication date:
- September 30, 2017
- DOI:
-
- Keyword(s):
- part-of-speech social media rare phenomena
- Published in:
- Proceedings of the 5th Conference on CMC and Social Media Corpora for the Humanities (cmccorpora17), cmc-corpora conference series.
- Meeting:
- 5th Conference on CMC and Social Media Corpora for the Humanities (cmccorpora17), Bolzano, Italy, 3-4 October 2017
- Related identifiers:
- Part of
- Communities:
- License (for files):
- Creative Commons Attribution 4.0 International