Templated text synthesis for expert-guided multi-label extraction from radiology reports
Date
24/03/2021Author
Funder
Grant ID
TS/S013121/1
Keywords
Metadata
Show full item recordAltmetrics Handle Statistics
Altmetrics DOI Statistics
Abstract
Training medical image analysis models traditionally requires large amounts of expertly annotated imaging data which is time-consuming and expensive to obtain. One solution is to automatically extract scan-level labels from radiology reports. Previously, we showed that, by extending BERT with a per-label attention mechanism, we can train a single model to perform automatic extraction of many labels in parallel. However, if we rely on pure data-driven learning, the model sometimes fails to learn critical features or learns the correct answer via simplistic heuristics (e.g., that “likely” indicates positivity), and thus fails to generalise to rarer cases which have not been learned or where the heuristics break down (e.g., “likely represents prominent VR space or lacunar infarct” which indicates uncertainty over two differential diagnoses). In this work, we propose template creation for data synthesis, which enables us to inject expert knowledge about unseen entities from medical ontologies, and to teach the model rules on how to label difficult cases, by producing relevant training examples. Using this technique alongside domain-specific pre-training for our underlying BERT architecture i.e., PubMedBERT, we improve F1 micro from 0.903 to 0.939 and F1 macro from 0.512 to 0.737 on an independent test set for 33 labels in head CT reports for stroke patients. Our methodology offers a practical way to combine domain knowledge with machine learning for text classification tasks.
Citation
Schrempf , P , Watson , H , Park , E , Pajak , M , MacKinnon , H , Muir , K W , Harris-Birtill , D & O’Neil , A Q 2021 , ' Templated text synthesis for expert-guided multi-label extraction from radiology reports ' , Machine Learning and Knowledge Extraction , vol. 3 , no. 2 , pp. 299-317 . https://doi.org/10.3390/make3020015
Publication
Machine Learning and Knowledge Extraction
Status
Peer reviewed
ISSN
2504-4990Type
Journal article
Rights
Copyright © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Description
Funding:This work is part of the Industrial Centre for AI Research in digital Diagnostics (iCAIRD), which is funded by Innovate UK on behalf of UK Research and Innovation (UKRI) project number 104690. The Data Lab has also provided support and funding.Collections
Items in the St Andrews Research Repository are protected by copyright, with all rights reserved, unless otherwise indicated.