Extending defoe for the efficient analysis of historical texts at scale
Abstract
This paper presents the new facilities provided in defoe, a parallel toolbox for querying a wealth of digitised newspapers and books at scale. defoe has been extended to work with further Natural Language Processing () tools such as the Edinburgh Geoparser, to store the preprocessed text in several storage facilities and to support different types of queries and analyses. We have also extended the collection of XML schemas supported by defoe, increasing the versatility of the tool for the analysis of digital historical textual data at scale. Finally, we have conducted several studies in which we worked with humanities and social science researchers who posed complex and interested questions to large-scale digital collections. Results shows that defoe allows researchers to conduct their studies and obtain results faster, while all the large-scale text mining complexity is automatically handled by defoe.
Citation
Filgueira , R , Grover , C , Karaiskos , V , Alex , B , Van Eyndhoven , S , Gotthard , L & Terras , M 2021 , Extending defoe for the efficient analysis of historical texts at scale . in 17th IEEE International Conference on eScience 2021 . IEEE International Conference on eScience , IEEE , United States , pp. 21-29 , 17th IEEE International Conference on eScience 2021 , Innsbruck , Austria , 20/09/21 . https://doi.org/10.1109/eScience51609.2021.00012 conference
Publication
17th IEEE International Conference on eScience 2021
ISSN
2325-3703Type
Conference item
Rights
Copyright © 2021 IEEE. This work has been made available online in accordance with publisher policies or with permission. Permission for further reuse of this content should be sought from the publisher or the rights holder. This is the author created accepted manuscript following peer review and may differ slightly from the final published version. The final published version of this work is available at https://doi.org/10.1109/eScience51609.2021.00012.
Description
Funding: This work was partly funded by the Data-Driven Innovation Programme as part of the Edinburgh and South East Scotland City Region Deal, by the University of Edinburgh, and by Google Cloud Platform research credits program.Collections
Items in the St Andrews Research Repository are protected by copyright, with all rights reserved, unless otherwise indicated.