Transfer Learning of Automatic Post-Editing with Cross-lingual Language Model
- Title
- Transfer Learning of Automatic Post-Editing with Cross-lingual Language Model
- Authors
- 이지형
- Date Issued
- 2021
- Publisher
- 포항공과대학교
- Abstract
- In this thesis, we present a new approach to Automatic Post-Editing (APE) that uses the cross-lingual language model (XLM) as a pre-trained model for the English-German language pair. To initialize our APE model's encoder, we use the weights of a pre-trained XLM that has been jointly trained with a translation language modeling (TLM) objective by using parallel corpora. Besides, we experimented with various decoder-initialization methods such as using the weights of pre-trained models and tying parameters. Experimental results imply that training a model with both a masked language modeling (MLM) and a TLM objective is more effective than training the same model with an MLM objective only. In experiments conducted on the WMT16, '17, '18 test datasets, Our APE results showed significant improvements over the MT outputs in translation quality.
- URI
- http://postech.dcollection.net/common/orgView/200000366296
https://oasis.postech.ac.kr/handle/2014.oak/111752
- Article Type
- Thesis
- Files in This Item:
- There are no files associated with this item.
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.