Adaptation Data Selection using Neural Language Models

Kevin Duh (05/20/2013)

Here are some scripts associated with the paper:
K. Duh, G. Neubig, K. Sudoh, H. Tsukada, Adaptation Data Selection using Neural Language Models: Experiments in Machine Translation, Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), 2013.

Basically it consists of just a few simple scripts that stitch together existing tools, in particular RNNLM by Mikolov and SRILM by Stolcke. Given in-domain and out-of-domain bitexts, it uses the sentence filtering method of Axelrold et. al. [Domain adaptation via Pseudo in-domain data selection, EMNLP2011] in conjunction with neural language models. A modified version of Tomas Mikolov's Neural Language Model is included here for convenience. For SRILM, please download by yourself. Axelrod's data selection method is implemented in a simple Python script.


Last modified: Mon May 20 13:34:42 JST 2013