Conference paper Open Access
Rosendahl, Jan; Tran, Viet Anh Khoa; Wang, Weiyue; Ney, Hermann
<?xml version='1.0' encoding='utf-8'?> <resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://datacite.org/schema/kernel-4" xsi:schemaLocation="http://datacite.org/schema/kernel-4 http://schema.datacite.org/meta/kernel-4.1/metadata.xsd"> <identifier identifierType="DOI">10.5281/zenodo.3525024</identifier> <creators> <creator> <creatorName>Rosendahl, Jan</creatorName> <givenName>Jan</givenName> <familyName>Rosendahl</familyName> <affiliation>Human Language Technology and Pattern Recognition Group, RWTH Aachen University, Germany</affiliation> </creator> <creator> <creatorName>Tran, Viet Anh Khoa</creatorName> <givenName>Viet Anh Khoa</givenName> <familyName>Tran</familyName> <affiliation>Human Language Technology and Pattern Recognition Group, RWTH Aachen University, Germany</affiliation> </creator> <creator> <creatorName>Wang, Weiyue</creatorName> <givenName>Weiyue</givenName> <familyName>Wang</familyName> <affiliation>Human Language Technology and Pattern Recognition Group, RWTH Aachen University, Germany</affiliation> </creator> <creator> <creatorName>Ney, Hermann</creatorName> <givenName>Hermann</givenName> <familyName>Ney</familyName> <affiliation>Human Language Technology and Pattern Recognition Group, RWTH Aachen University, Germany</affiliation> </creator> </creators> <titles> <title>Analysis of Positional Encodings for Neural Machine Translation</title> </titles> <publisher>Zenodo</publisher> <publicationYear>2019</publicationYear> <dates> <date dateType="Issued">2019-11-02</date> </dates> <resourceType resourceTypeGeneral="Text">Conference paper</resourceType> <alternateIdentifiers> <alternateIdentifier alternateIdentifierType="url">https://zenodo.org/record/3525024</alternateIdentifier> </alternateIdentifiers> <relatedIdentifiers> <relatedIdentifier relatedIdentifierType="DOI" relationType="IsVersionOf">10.5281/zenodo.3525023</relatedIdentifier> <relatedIdentifier relatedIdentifierType="URL" relationType="IsPartOf">https://zenodo.org/communities/iwslt2019</relatedIdentifier> </relatedIdentifiers> <rightsList> <rights rightsURI="https://creativecommons.org/licenses/by/4.0/legalcode">Creative Commons Attribution 4.0 International</rights> <rights rightsURI="info:eu-repo/semantics/openAccess">Open Access</rights> </rightsList> <descriptions> <description descriptionType="Abstract"><p>In this work we analyze and compare the behavior of the Transformer architecture when using different positional encoding methods. While absolute and relative positional encoding perform equally strong overall, we show that relative positional encoding is vastly superior (4.4% to 11.9% BLEU) when translating a sentence that is longer than any observed training sentence. We further propose and analyze variations of relative positional encoding and observe that the number of trainable parameters can be reduced without a performance loss, by using fixed encoding vectors or by removing some of the positional encoding vectors.</p></description> </descriptions> </resource>
All versions | This version | |
---|---|---|
Views | 150 | 151 |
Downloads | 131 | 130 |
Data volume | 63.6 MB | 63.1 MB |
Unique views | 136 | 137 |
Unique downloads | 123 | 122 |