[1712.01208] The Case for Learned Index Structures

Indexes are models: a B-Tree-Index can be seen as a model to map a key to the position of a record within a sorted array, a Hash-Index as a model to map a key to a position of a record within an unsorted array, and a BitMap-Index as a model to indicate if a data record exists or not. In this exploratory research paper, we start from this premise and posit that all existing index structures can be replaced with other types of models, including deep-learning models, which we term learned indexes. The key idea is that a model can learn the sort order or structure of lookup keys and use this signal to effectively predict the position or existence of records. We theoretically analyze under which conditions learned indexes outperform traditional index structures and describe the main challenges in designing learned index structures. Our initial results show, that by using neural nets we are able to outperform cache-optimized B-Trees by up to 70% in speed while saving an order-of-magnitude in

1 mentions: @mattn_jp
Date: 2021/01/08 08:21

Referring Tweets

@mattn_jp 以下の件(B-Tree index を AI ベースにしたら性能でたという論文)もそう思ったけど、そろそろ職人による解析処理は AI に負けてしまうのではないか。 t.co/wMuYdpbCLa

Bookmark Comments

Related Entries

Read more [1711.00165] Deep Neural Networks as Gaussian Processes
6 users, 2 mentions 2019/03/21 12:47
Read more 【7日目】TED: A Pretrained Unsupervised Summarization Model with Theme Modeling and Denoising - やむやむもやむな...
1 users, 1 mentions 2020/12/07 19:11
Read more Danbooru2019: A Large-Scale Crowdsourced and Tagged Anime Illustration Dataset · Gwern.net
11 users, 1 mentions 2020/12/15 14:21
Read more [1703.06870] Mask R-CNN
10 users, 1 mentions 2020/12/20 05:21
Read more GitHub - shogo82148/TinySegmenterMaker
5 users, 2 mentions 2020/12/20 14:21