Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deprecate COSINE VectorSimilarity function #13308

Closed
wants to merge 45 commits into from
Closed
Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
589d1e2
Mark COSINE Vector Similarity Function as Deprecated
Pulkitg64 May 6, 2024
3f23885
tidy fixes
Pulkitg64 May 6, 2024
0c0c35b
Added entry to CHANGES.txt
Pulkitg64 May 8, 2024
821f775
Mark VectorUtil cosine function as deprecated
Pulkitg64 May 8, 2024
6ca2e43
Move change from 10.0 to 9.11.0
Pulkitg64 May 13, 2024
a644bea
Merge branch 'main' into cosine-deprecate
Pulkitg64 May 28, 2024
54d3ff6
hunspell: speed up "compress"; minimize the number of the generated e…
donnerpeter May 28, 2024
ea0646d
Fixes failing test case for TestOrdinalMap.testRamBytesUsed (#13421)
pseudo-nymous May 28, 2024
b3dc915
Allow users to retrieve counts from taxo association facets (#13414)
stefanvodita May 29, 2024
9a3dbd5
Add next minor version 9.12.0
benwtrent May 29, 2024
750a7c4
Fix TestIndexWriterOnError.testIOError failure. (#13436)
jpountz May 29, 2024
f3c2b91
SimpleText[Float|Byte]VectorValues::scorer should return null when th…
ChrisHegarty May 31, 2024
a540027
Add new dynamic confidence interval configuration to scalar quantized…
benwtrent Jun 1, 2024
a6f920d
Fix test failure on TestPoint#testEqualsAndHashCode (#13433)
easyice Jun 3, 2024
edd7747
Add prefetching support to stored fields. (#13424)
jpountz Jun 3, 2024
c132e95
mention KnnVectorsFormat in o.a.l.codecs package javadocs (#13448)
msokolov Jun 3, 2024
801b822
Avoid unnecessary memory allocation in PackedLongValues#Iterator (#13…
easyice Jun 4, 2024
e868b82
Rewrite newSlowRangeQuery to MatchNoDocsQuery when upper > lower (#13…
ioanatia Jun 4, 2024
846aa2f
Use `ReadAdvice#NORMAL` on files that have a forward-only access patt…
jpountz Jun 5, 2024
05b4639
Add prefetching for doc values and norms. (#13411)
jpountz Jun 5, 2024
fe50e86
Implement Weight#count for vector values in the FieldExistsQuery (#13…
bugmakerrrrrr Jun 5, 2024
9a4caa9
Update Gradle wrapper to 8.8 - supports Java 22 (#13453)
ChrisHegarty Jun 6, 2024
868897e
Update WrapperDownloader to accept java 22 and correct deprecated new…
dweiss Jun 6, 2024
14782a2
Add a github workflow that checks common (and less common) gradle tas…
dweiss Jun 6, 2024
b85c99d
Java 22 has been released, so drop -ea from smoketester gh workflow m…
dweiss Jun 6, 2024
d5aa88b
Add test for ghost fields to BaseKnnVectorQueryTestCase. (#13455)
jpountz Jun 6, 2024
d0d2aa2
Removed Scorer#getWeight (#13440)
iamsanjay Jun 6, 2024
61a6abd
DOAP changes for release 9.11.0
benwtrent Jun 6, 2024
58ab5b7
Merge related HashMaps in FieldInfos#FieldNumbers into one map (#13460)
iverase Jun 6, 2024
51d8d72
Sync CHANGES for 9.11.0
benwtrent Jun 6, 2024
39a7eab
Add back-compat indices for 9.11.0
benwtrent Jun 6, 2024
512ff4a
MultiTermQuery return null for ScoreSupplier (#13454)
mayya-sharipova Jun 6, 2024
9f8e886
Move entry in CHANGES.txt
iverase Jun 7, 2024
c7a7d48
Reduce the heap use of BKDReader instances (#13464)
original-brownbear Jun 7, 2024
a5b4b8c
Document how to make tests run faster in IntelliJ (#13466)
msokolov Jun 7, 2024
2d62faa
Add int8_hnsw backcompat index creawtion to dev tools scripts (#13465)
benwtrent Jun 7, 2024
262341b
on README.md, make links to CONTRIBUTING.md more prominent, and demot…
Jun 7, 2024
71a9aed
fix fumble-finger
Jun 7, 2024
0699117
clarify that IntelliJ UI varies across platforms
Jun 7, 2024
00a8704
Mark COSINE Vector Similarity Function as Deprecated
Pulkitg64 May 6, 2024
be3527a
tidy fixes
Pulkitg64 May 6, 2024
0e91af0
Move changes from 9.11.0 to 9.12.0
Pulkitg64 Jun 9, 2024
1163749
Mark VectorUtil cosine function as deprecated
Pulkitg64 May 8, 2024
ec6a037
Move change from 10.0 to 9.11.0
Pulkitg64 May 13, 2024
2a52ebf
Merge remote-tracking branch 'origin/cosine-deprecate' into cosine-de…
Pulkitg64 Jun 9, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions lucene/CHANGES.txt
Original file line number Diff line number Diff line change
Expand Up @@ -232,6 +232,8 @@ API Changes

* GITHUB#13145: Deprecate ByteBufferIndexInput as it will be removed in Lucene 10.0. (Uwe Schindler)

* GITHUB#13281: Mark COSINE VectorSimilarityFunction as deprecated. (Pulkit Gupta)

New Features
---------------------

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,10 @@ public float compare(byte[] v1, byte[] v2) {
* vectors to unit length, and instead use {@link VectorSimilarityFunction#DOT_PRODUCT}. You
* should only use this function if you need to preserve the original vectors and cannot normalize
* them in advance. The similarity score is normalised to assure it is positive.
*
* @deprecated Use MAXIMUM_INNER_PRODUCT or DOT_PRODUCT instead
*/
@Deprecated
COSINE {
@Override
public float compare(float[] v1, float[] v2) {
Expand Down
9 changes: 8 additions & 1 deletion lucene/core/src/java/org/apache/lucene/util/VectorUtil.java
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,9 @@ public static float dotProduct(float[] a, float[] b) {
* Returns the cosine similarity between the two vectors.
*
* @throws IllegalArgumentException if the vectors' dimensions differ.
* @deprecated use dot-product instead using normalized vectors
*/
@Deprecated
public static float cosine(float[] a, float[] b) {
if (a.length != b.length) {
throw new IllegalArgumentException("vector dimensions differ: " + a.length + "!=" + b.length);
Expand All @@ -80,7 +82,12 @@ public static float cosine(float[] a, float[] b) {
return r;
}

/** Returns the cosine similarity between the two vectors. */
/**
* Returns the cosine similarity between the two vectors.
*
* @deprecated use dot-product instead using normalized vectors
*/
@Deprecated
public static float cosine(byte[] a, byte[] b) {
if (a.length != b.length) {
throw new IllegalArgumentException("vector dimensions differ: " + a.length + "!=" + b.length);
Expand Down