Information and knowing when to forget it
MetadataShow full item record
In this paper we propose several novel approaches for incorporating forgetting mechanisms into sequential prediction based machine learning algorithms. The broad premise of our work, supported and motivated in part by recent findings stemming from neurology research on the development of human brains, is that knowledge acquisition and forgetting are complementary processes, and that learning can (perhaps unintuitively) benefit from the latter too. We demonstrate that if forgetting is implemented in a purposeful and date driven manner, there are a number of benefits which can be gained from discarding information. The framework we introduce is a general one and can be used with any baseline predictor of choice. Hence in this sense it is best described as a meta-algorithm. The method we described was developed through a series of steps which increase the adaptability of the model, while being data driven.We first discussed a weakly adaptive forgetting process which we termed passive forgetting. A fully adaptive framework, which we termed active forgetting was developed by enveloping a passive forgetting process with a monitoring, self-aware module which detects contextual changes and makes a statistically informed choice when the model parameters should be abruptly rather than gradually updated. The effectiveness of the proposed metaframework was demonstrated on a real world data set concerned with a challenge of major practical importance: that of predicting currency exchange rates. Our approach was shown to be highly effective, reducing prediction errors by nearly 40%.
Sharma , R & Arandelovic , O 2017 , Information and knowing when to forget it . in 2017 IEEE International Joint Conference on Neural Networks (IJCNN) . , 7966253 , IEEE , pp. 3184-3190 , 2017 International Joint Conference on Neural Networks , Anchorage , United States , 14-19 May . DOI: 10.1109/IJCNN.2017.7966253conference
2017 IEEE International Joint Conference on Neural Networks (IJCNN)
© 2017, IEEE. This work has been made available online in accordance with the publisher’s policies. This is the author created, accepted version manuscript following peer review and may differ slightly from the final published version. The final published version of this work is available at ieeexplore.ieee.org / https://doi.org/10.1109/IJCNN.2017.7966253
Items in the St Andrews Research Repository are protected by copyright, with all rights reserved, unless otherwise indicated.