Sampling a Near Neighbor in High Dimensions-Who is the Fairest of Them All?

Publikation: Bidrag til tidsskrift › Tidsskriftartikel › Forskning › fagfællebedømt

Dokumenter

  • Fulltext

    Accepteret manuskript, 1,15 MB, PDF-dokument

  • Martin Aumüller
  • Sariel Har-Peled
  • Sepideh Mahabadi
  • Pagh, Rasmus
  • Francesco Silvestri

Similarity search is a fundamental algorithmic primitive, widely used in many computer science disciplines. Given a set of points S and a radius parameter r > 0, the r-near neighbor (r-NN) problem asks for a data structure that, given any query point q, returns a point p within distance at most r from q. In this paper, we study the r-NN problem in the light of individual fairness and providing equal opportunities: all points that are within distance r from the query should have the same probability to be returned. In the low-dimensional case, this problem was first studied by Hu, Qiao, and Tao (PODS 2014). Locality sensitive hashing (LSH), the theoretically strongest approach to similarity search in high dimensions, does not provide such a fairness guarantee.In this work, we show that LSH based algorithms can be made fair, without a significant loss in efficiency. We propose several efficient data structures for the exact and approximate variants of the fair NN problem. Our approach works more generally for sampling uniformly from a sub-collection of sets of a given collection and can be used in a few other applications. We also develop a data structure for fair similarity search under inner product that requires nearly-linear space and exploits locality sensitive filters. The paper concludes with an experimental evaluation that highlights the unfairness of state-of-the-art NN data structures and shows the performance of our algorithms on real-world datasets.

OriginalsprogEngelsk
Artikelnummer4
TidsskriftACM Transactions on Database Systems
Vol/bind47
Udgave nummer1
Sider (fra-til)1-40
ISSN0362-5915
DOI
StatusUdgivet - 2022

Bibliografisk note

Funding Information:
S. Har-Peled was partially supported by a NSF AF award CCF-1907400. F. Silvestri was partially supported by UniPD SID18 grant and PRIN Project n. 20174LF3T8 AHeAD. Authors’ addresses: M. Aumüller, IT University of Copenhagen, Rued Langgaards Vej 7, DK-2300 København S, Denmark; email: maau@itu.dk; S. Har-Peled, University of Illinois at Urbana-Champaign, 201 N. Goodwin Ave., Urbana, IL, 60801, USA; email: sariel@illinois.edu; S. Mahabadi, Toyota Technological Institute at Chicago and MSR Redmond, 6045 S Ken-wood Ave., Chicago, IL, 60637, USA; email: mahabadi@ttic.edu; R. Pagh, BARC and University of Copenhagen, Univer-sitetsparken 1, 2100, København Ø, Denmark; email: pagh@di.ku.dk; F. Silvestri, University of Padova, Via Gradenigo 6/B, I-35131 Padova, Italy; email: silvestri@dei.unipd.it. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. © 2022 Association for Computing Machinery. 0362-5915/2022/04-ART4 $15.00 https://doi.org/10.1145/3502867

Publisher Copyright:
© 2022 Association for Computing Machinery.

ID: 340698972