Language transfer for early warning of epidemics from social media
Abstract
Statements on social media can be analysed to identify individuals who are experiencing red flag medical symptoms, allowing early detection of the spread of disease such as influenza. Since disease does not respect cultural borders and may spread between populations speaking different languages, we would like to build multilingual models. However, the data required to train models for every language may be difficult, expensive and time-consuming to obtain, particularly for low-resource languages. Taking Japanese as our target language, we explore methods by which data in one language might be used to build models for a different language. We evaluate strategies of training on machine translated data and of zero-shot transfer through the use of multilingual models. We find that the choice of source language impacts the performance, with Chinese-Japanese being a better language pair than English-Japanese. Training on machine translated data shows promise, especially when used in conjunction with a small amount of target language data.
Citation
Appelgren , M , Schrempf , P , Falis , M , Ikeda , S & O'Neil , A Q 2019 , ' Language transfer for early warning of epidemics from social media ' , Paper presented at Artificial Intelligence for Humanitarian Assistance and Disaster Response , Vancouver , Canada , 13/12/19 . < https://arxiv.org/abs/1910.04519 > workshop
Status
Peer reviewed
Type
Conference paper
Rights
Copyright © 2019 the Author(s). This work has been made available online in accordance with publisher policies or with permission. Permission for further reuse of this content should be sought from the publisher or the rights holder. This is the author created accepted manuscript following peer review and may differ slightly from the final published version. The final published version of this work is available at https://arxiv.org/abs/1910.04519
Collections
Items in the St Andrews Research Repository are protected by copyright, with all rights reserved, unless otherwise indicated.