3524999
doi
10.5281/zenodo.3524999
oai:zenodo.org:3524999
user-iwslt2019
Herold, Christian
Human Language Technology and Pattern Recognition Group, Computer Science Department, RWTH Aachen University, D-52056 Aachen, Germany
Wang, Weiyue
Human Language Technology and Pattern Recognition Group, Computer Science Department, RWTH Aachen University, D-52056 Aachen, Germany
Ney, Hermann
Human Language Technology and Pattern Recognition Group, Computer Science Department, RWTH Aachen University, D-52056 Aachen, Germany
Exploring Kernel Functions in the Softmax Layer for Contextual Word Classification
Gao, Yingbo
Human Language Technology and Pattern Recognition Group, Computer Science Department, RWTH Aachen University, D-52056 Aachen, Germany
info:eu-repo/semantics/openAccess
Creative Commons Attribution 4.0 International
https://creativecommons.org/licenses/by/4.0/legalcode
<p>Prominently used in support vector machines and logistic re-gressions, kernel functions (kernels) can implicitly map data points into high dimensional spaces and make it easier to learn complex decision boundaries. In this work, by replacing the inner product function in the softmax layer, we explore the use of kernels for contextual word classification. In order to compare the individual kernels, experiments are conducted on standard language modeling and machine translation tasks. We observe a wide range of performances across different kernel settings. Extending the results, we look at the gradient properties, investigate various mixture strategies and examine the disambiguation abilities.</p>
Zenodo
2019-11-02
info:eu-repo/semantics/conferencePaper
3524998
user-iwslt2019
1579538849.723596
394878
md5:01f629061ed819d11ef2f96bc6169d5b
https://zenodo.org/records/3524999/files/IWSLT2019_paper_15.pdf
public
10.5281/zenodo.3524998
isVersionOf
doi