Ideas

[ankane]

Please create a new issue to discuss any ideas or share your own.

HNSW

• Add support for inline filtering
  • Scan only - hnsw-filtering branch
  • HQANN - hqann branch (WIP)
• Use scan->kill_prior_tuple - kill-prior-tuple branch (WIP)
• Add support for index-only scans - hnsw-index-only-v2 branch
• Add support for INCLUDE clause with CREATE INDEX (not more performant since still accesses heap)
• Improve numIndexTuples in cost estimation

IVFFlat

• Add support for product quantization
• Add support for another coarse quantizer
• Add support for parallel k-means
• Update parallel workers calculation - ivfflat-parallel-workers branch
• Increase default value of ivfflat.probes
• Add support for setting default probes on index level
• Scan additional lists if number of entries is low after scanning probes
• Use pairing heap for index scan for performance - stages and pairingheap branches
• Use tuplesort_set_bound for performance - bound branch (not needed w/ pairing heap)
• Add support for parallel index scans (planner gets cost estimate but doesn't use) - parallel-index-scan3 branch
• Use Lloyd's or MO-Elkan's k-means when not enough memory for Elkan's
• Use FB-Elkan's k-means for faster index creation
• Use mini-batch k-means for index creation for reduced memory - minibatch branch
• Add support for index-only scans - ivfflat-index-only branch
• Add support for INCLUDE clause with CREATE INDEX
• Prevent compression for index tuples - ivfflat-storage branch

Indexes

• Include vectors with zero norm in indexes for cosine distance - index-zero-norm branch
• Update cost estimation to not use index if LIMIT + OFFSET > expected tuples - index-limit/index-limit2 branch
• Update cost estimation to not use index if large % of rows will be filtered by WHERE condition - hnsw-filtering-cost branch

Types

• Add support for int8 vectors - intvec branch

Functions

• Add subscript function - subscript branch (requires Postgres 14+)
• Add angular_distance function - angular_distance branch - Adding angular distance for nn search #234
• Add random_vector function - random_vector branch
• Add functions to view lists and/or pages like pageinspect (require superuser)
• Add function to estimate index balance (require superuser?) - Estimating ivfflat balance / need to recompute index #105

Installation

• Create installer for Windows - Offer precompiled OS packages (feature request) #108