Investigating binary partition power in metric query
Abstract
It is generally understood that, as dimensionality increases, the minimum cost of metric query tends from (log ) to () in both space and time, where is the size of the data set. With low dimensionality, the former is easy to achieve; with very high dimensionality, the latter is inevitable. We previously described BitPart as a novel mechanism suitable for performing exact metric search in “high(er)” dimensions. The essential tradeoff of BitPart is that its space cost is linear with respect to the size of the data, but the actual space required for each object may be small as log2 bits, which allows even very large data sets to be queried using only main memory. Potentially the time cost still scales with (log ). Together these attributes give exact search which outperforms indexing structures if dimensionality is within a certain range. In this article, we reiterate the design of BitPart in this context. The novel contribution is an in-depth examination of what the notion of “high(er)” means in practical terms. To do this we introduce the notion of exclusion power, and show its application to some generated data sets across different dimensions.
Citation
Connor , R , Dearle , A & Vadicamo , L 2022 , Investigating binary partition power in metric query . in G Amato , V Bartalesi , D Bianchini , C Gennaro & R Torlone (eds) , SEBD 2022 : proceedings of the the 30 th Italian Symposium on Advanced Database Systems . vol. 3194 , Italian Symposium on Advanced Database Systems , vol. 3194 , CEUR-WS , Online , pp. 415-426 , Italian Symposium on Advanced Database Systems (SEBD 2022) , Tirrenia (Pisa) , Italy , 19/06/22 . < http://ceur-ws.org/Vol-3194/paper49.pdf > conference
Publication
SEBD 2022
ISSN
1613-0073Type
Conference item
Rights
Copyright © 2022 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
Collections
Items in the St Andrews Research Repository are protected by copyright, with all rights reserved, unless otherwise indicated.