Open Access System for Information Sharing

Login Library

 

Article
Cited 0 time in webofscience Cited 0 time in scopus
Metadata Downloads

Target-Oriented Knowledge Distillation with Language-Family-Based Grouping for Multilingual NMT SCIE SCOPUS

Title
Target-Oriented Knowledge Distillation with Language-Family-Based Grouping for Multilingual NMT
Authors
Heejin DoLEE, GARY GEUNBAE
Date Issued
2023-02
Publisher
ACM
Abstract
Multilingual NMT has been developed rapidly, but still has performance degradation caused by language diversity and model capacity constraints. To achieve the competitive accuracy of multilingual translation despite such limitations, knowledge distillation, which improves the student network by matching the teacher network’s output, has been applied and shown enhancement by focusing on the important parts of the teacher distribution. However, existing knowledge distillation methods for multilingual NMT rarely consider the knowledge, which has an important function as the student model’s target, in the process. In this paper, we propose two distillation strategies that effectively use the knowledge to improve the accuracy of multilingual NMT. First, we introduce a language-family-based approach, guiding to select appropriate knowledge for each language pair. By distilling the knowledge of multilingual teachers that each processes a group of languages classified by language families, the multilingual model overcomes accuracy degradation caused by linguistic diversity. Second, we propose target-oriented knowledge distillation, which intensively focuses on the ground-truth target of knowledge with a penalty strategy. Our method provides a sensible distillation by penalizing samples without actual targets, while additionally targeting the ground-truth targets. Experiments using TED Talk datasets demonstrate the effectiveness of our method with BLEU scores increment. Discussions of distilled knowledge and further observations of the methods also validate our results.
URI
https://oasis.postech.ac.kr/handle/2014.oak/115618
DOI
10.1145/3546067
ISSN
2375-4699
Article Type
Article
Citation
Acm Transactions on Asian and Low-resource Language Information Processing, vol. 22, no. 2, 2023-02
Files in This Item:
There are no files associated with this item.

qr_code

  • mendeley

Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.

Related Researcher

Views & Downloads

Browse