Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CI] CcsCommonYamlTestSuiteIT test {p0=search.vectors/41_knn_search_byte_quantized/Test create, merge, and search dot_product} failing #108339

Closed
ywangd opened this issue May 7, 2024 · 3 comments · Fixed by #108522
Closed

[CI] CcsCommonYamlTestSuiteIT test {p0=search.vectors/41_knn_search_byte_quantized/Test create, merge, and search dot_product} failing #108339

ywangd opened this issue May 7, 2024 · 3 comments · Fixed by #108522
Assignees
@benwtrent
Labels
low-risk An open issue or test failure that is a low risk to future releases :Search/Search Search-related issues that do not fall into other categories Team:Search Meta label for search team >test-failure Triaged test failures from CI

Comments

@ywangd
Copy link
Member

ywangd commented May 7, 2024

This fails on release-tests if that makes any difference.

The failure is probably not specifically about this test since it has a bunch other tests failed with the same test cluster and it has the following AssertionError

[2024-05-07T16:51:03,414][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [remote_cluster-1] fatal error in thread [elasticsearch[remote_cluster-1][search_worker][T#24]], exiting java.lang.AssertionError	
	at [email protected]/org.apache.lucene.search.TopScoreDocCollector$SimpleTopScoreDocCollector$1.collect(TopScoreDocCollector.java:75)	
	at [email protected]/org.apache.lucene.search.Weight$DefaultBulkScorer.scoreRange(Weight.java:277)	
	at [email protected]/org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:236)	
	at [email protected]/org.elasticsearch.search.internal.CancellableBulkScorer.score(CancellableBulkScorer.java:45)	
	at [email protected]/org.apache.lucene.search.BulkScorer.score(BulkScorer.java:38)	
	at [email protected]/org.elasticsearch.search.internal.ContextIndexSearcher.searchLeaf(ContextIndexSearcher.java:431)	
	at [email protected]/org.elasticsearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:360)	
	at [email protected]/org.elasticsearch.search.internal.ContextIndexSearcher.lambda$search$4(ContextIndexSearcher.java:345)	
	at [email protected]/org.apache.lucene.search.TaskExecutor$TaskGroup.lambda$createTask$0(TaskExecutor.java:117)	
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)

Build scan:
https://gradle-enterprise.elastic.co/s/vgcc7jbcvzozo/tests/:qa:ccs-common-rest:yamlRestTest/org.elasticsearch.test.rest.yaml.CcsCommonYamlTestSuiteIT/test%20%7Bp0=search.vectors%2F41_knn_search_byte_quantized%2FTest%20create,%20merge,%20and%20search%20dot_product%7D

Reproduction line:

./gradlew ':qa:ccs-common-rest:yamlRestTest' --tests "org.elasticsearch.test.rest.yaml.CcsCommonYamlTestSuiteIT" -Dtests.method="test {p0=search.vectors/41_knn_search_byte_quantized/Test create, merge, and search dot_product}" -Dtests.seed=CA15C0087EBCFB79 -Dbuild.snapshot=false -Dtests.jvm.argline="-Dbuild.snapshot=false" -Dtests.locale=is-IS -Dtests.timezone=Pacific/Kwajalein -Druntime.java=21

Applicable branches:
8.14

Reproduces locally?:
Didn't try

Failure history:
Failure dashboard for org.elasticsearch.test.rest.yaml.CcsCommonYamlTestSuiteIT#test {p0=search.vectors/41_knn_search_byte_quantized/Test create, merge, and search dot_product}

Failure excerpt:

java.net.ConnectException: Connection refused

  at __randomizedtesting.SeedInfo.seed([CA15C0087EBCFB79:4241FFD2D0409681]:0)
  at org.elasticsearch.client.RestClient.extractAndWrapCause(RestClient.java:934)
  at org.elasticsearch.client.RestClient.performRequest(RestClient.java:304)
  at org.elasticsearch.client.RestClient.performRequest(RestClient.java:307)
  at org.elasticsearch.client.RestClient.performRequest(RestClient.java:292)
  at org.elasticsearch.test.rest.ESRestTestCase.ensureNoInitializingShards(ESRestTestCase.java:1734)
  at org.elasticsearch.test.rest.ESRestTestCase.cleanUpCluster(ESRestTestCase.java:530)
  at jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
  at java.lang.reflect.Method.invoke(Method.java:580)
  at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:1004)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at org.junit.rules.RunRules.evaluate(RunRules.java:20)
  at org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
  at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  at org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
  at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
  at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
  at org.junit.rules.RunRules.evaluate(RunRules.java:20)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
  at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843)
  at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490)
  at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902)
  at org.elasticsearch.test.cluster.local.DefaultLocalElasticsearchCluster$1.evaluate(DefaultLocalElasticsearchCluster.java:47)
  at org.elasticsearch.test.cluster.local.DefaultLocalElasticsearchCluster$1.evaluate(DefaultLocalElasticsearchCluster.java:47)
  at org.junit.rules.RunRules.evaluate(RunRules.java:20)
  at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
  at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
  at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
  at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
  at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
  at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
  at org.junit.rules.RunRules.evaluate(RunRules.java:20)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
  at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
  at java.lang.Thread.run(Thread.java:1583)

  Caused by: java.net.ConnectException: Connection refused

    at sun.nio.ch.Net.pollConnect(Net.java:-2)
    at sun.nio.ch.Net.pollConnectNow(Net.java:682)
    at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:973)
    at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvent(DefaultConnectingIOReactor.java:174)
    at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvents(DefaultConnectingIOReactor.java:148)
    at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.execute(AbstractMultiworkerIOReactor.java:351)
    at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.execute(PoolingNHttpClientConnectionManager.java:221)
    at org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase$1.run(CloseableHttpAsyncClientBase.java:64)
    at java.lang.Thread.run(Thread.java:1583)
@ywangd ywangd added :Search/Search Search-related issues that do not fall into other categories >test-failure Triaged test failures from CI labels May 7, 2024
@elasticsearchmachine elasticsearchmachine added Team:Search Meta label for search team needs:risk Requires assignment of a risk label (low, medium, blocker) labels May 7, 2024
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-search (Team:Search)

@benwtrent
Copy link
Member

That assertion trips for negative scores. Interesting that this occurred. Tells me that quantized scoring could accidentally be negative and we should snap to 0.

@benwtrent benwtrent self-assigned this May 10, 2024
@benwtrent benwtrent added low-risk An open issue or test failure that is a low risk to future releases and removed needs:risk Requires assignment of a risk label (low, medium, blocker) labels May 10, 2024
@benwtrent benwtrent mentioned this issue May 10, 2024
Merged
@valeriy42
Copy link
Contributor

failed again here: https://gradle-enterprise.elastic.co/s/oafv2tkhewejg

elasticsearchmachine pushed a commit that referenced this issue May 13, 2024
@benwtrent
Ensure we return non-negative scores when scoring scalar dot-products (…
e352345 
…#108522)

closes: #108339
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@benwtrent benwtrent

Labels
low-risk An open issue or test failure that is a low risk to future releases :Search/Search Search-related issues that do not fall into other categories Team:Search Meta label for search team >test-failure Triaged test failures from CI
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Ensure we return non-negative scores when scoring scalar dot-products benwtrent/elasticsearch
4 participants
@valeriy42 @ywangd @benwtrent @elasticsearchmachine