Weighted finite-state transducers for normalization of historical texts

This paper presents a study about methods for normalization of historical texts. The aim of these methods
is learning relations between historical and contemporary word forms. We have compiled training and test
corpora for different languages and scenarios, and we have tried to read the results related to the features
of the corpora and languages. Our proposed method, based on weighted finite-state transducers, is com-
pared to previously published ones. Our method learns to map phonological changes using a noisy channel
model; it is a simple solution that can use a limited amount of supervision in order to achieve adequate
performance. The compiled corpora are ready to be used for other researchers in order to compare results.
Concerning the amount of supervision for the task, we investigate how the size of training corpus affects
the results and identify some interesting factors to anticipate the difficulty of the task.

Egileak: 
Izaskun Etxeberria, Iñaki Alegria, Larraitz Uria

Argitalpen alorra:

Urtea: 
2019
Balorazioa: 

JCR Impact Factor: 0,8. COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE: Q4

Agerpen tokia: 

Natural Language Engineering 25 (2), 307–321
https://doi.org/10.1017/S1351324918000505

ISBN: 
ISSN: 1351-3249 (Print), 1469-8110 (Online)

Argitalpen mota:

Argitalpen sailkapena:

Datu-base bibliografikoak:

Aldizkariaren balorazioa:

HiTZ: