Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Embedding exception after v.0.27.1 onwards: ORT_RUNTIME_EXCEPTION #1003

Open
codehustler opened this issue Apr 23, 2024 · 0 comments
Open
Labels
bug Something isn't working P2 High priority

Comments

@codehustler
Copy link

Describe the bug
we have recently updated langchain from 0.25.x to 0.29.1 and are seeing multiple embedding errors. I have tracked down that the error occurs first in v0.28.0 (v0.27.1 is the last version where it still works without the errors). there was a version bump of the onnxruntime from 1.16.3 to 1.17.1 but even when I lock the dependency to 1.16.3, I still get the embedding error in later versions. Is this a known issue? embedding does not always fail, only with certain text content.

Log and Stack trace

org.opentest4j.AssertionFailedError: Unexpected exception thrown: java.lang.RuntimeException: ai.onnxruntime.OrtException: Error code - ORT_RUNTIME_EXCEPTION - message: Non-zero status code returned while running Add node. Name:'/embeddings/Add_1' Status Message: C:\a\_work\1\s\onnxruntime\core/providers/cpu/math/element_wise_ops.h:560 onnxruntime::BroadcastIterator::Append axis == 1 || axis == largest was false. Attempting to broadcast an axis by a dimension other than 1. 512 by 803

	at org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:152)
	at org.junit.jupiter.api.AssertDoesNotThrow.createAssertionFailedError(AssertDoesNotThrow.java:84)
	at org.junit.jupiter.api.AssertDoesNotThrow.assertDoesNotThrow(AssertDoesNotThrow.java:75)
	at org.junit.jupiter.api.AssertDoesNotThrow.assertDoesNotThrow(AssertDoesNotThrow.java:58)
	at org.junit.jupiter.api.Assertions.assertDoesNotThrow(Assertions.java:3228)
	at com.accurids.module.ai.embedding.EmbeddingServiceImplTest.testEmbeddingAfterV0_27_1(EmbeddingServiceImplTest.java:55)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
Caused by: java.lang.RuntimeException: ai.onnxruntime.OrtException: Error code - ORT_RUNTIME_EXCEPTION - message: Non-zero status code returned while running Add node. Name:'/embeddings/Add_1' Status Message: C:\a\_work\1\s\onnxruntime\core/providers/cpu/math/element_wise_ops.h:560 onnxruntime::BroadcastIterator::Append axis == 1 || axis == largest was false. Attempting to broadcast an axis by a dimension other than 1. 512 by 803

	at dev.langchain4j.model.embedding.OnnxBertBiEncoder.embed(OnnxBertBiEncoder.java:57)
	at dev.langchain4j.model.embedding.AbstractInProcessEmbeddingModel.lambda$embedAll$0(AbstractInProcessEmbeddingModel.java:45)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.Collections$2.tryAdvance(Collections.java:5073)
	at java.base/java.util.Collections$2.forEachRemaining(Collections.java:5081)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682)
	at dev.langchain4j.model.embedding.AbstractInProcessEmbeddingModel.embedAll(AbstractInProcessEmbeddingModel.java:46)
	at dev.langchain4j.model.embedding.EmbeddingModel.embed(EmbeddingModel.java:34)
	at dev.langchain4j.model.embedding.EmbeddingModel.embed(EmbeddingModel.java:24)
	at com.accurids.module.ai.embedding.EmbeddingServiceImplTest.lambda$0(EmbeddingServiceImplTest.java:55)
	at org.junit.jupiter.api.AssertDoesNotThrow.assertDoesNotThrow(AssertDoesNotThrow.java:71)
	... 6 more
Caused by: ai.onnxruntime.OrtException: Error code - ORT_RUNTIME_EXCEPTION - message: Non-zero status code returned while running Add node. Name:'/embeddings/Add_1' Status Message: C:\a\_work\1\s\onnxruntime\core/providers/cpu/math/element_wise_ops.h:560 onnxruntime::BroadcastIterator::Append axis == 1 || axis == largest was false. Attempting to broadcast an axis by a dimension other than 1. 512 by 803

	at ai.onnxruntime.OrtSession.run(Native Method)
	at ai.onnxruntime.OrtSession.run(OrtSession.java:301)
	at ai.onnxruntime.OrtSession.run(OrtSession.java:242)
	at ai.onnxruntime.OrtSession.run(OrtSession.java:210)
	at dev.langchain4j.model.embedding.OnnxBertBiEncoder.encode(OnnxBertBiEncoder.java:102)
	at dev.langchain4j.model.embedding.OnnxBertBiEncoder.embed(OnnxBertBiEncoder.java:53)
	... 20 more


To Reproduce

    /**
     * this test fails after version 0.27.1
     */
    @Test
    void testEmbeddingAfterV0_27_1() {
        String sampleText = """
                In Subset: NCIthesaurus
                Type: Class (http://www.w3.org/2002/07/owl#Class)
                Subclass of: DOID_1324 (http://purl.obolibrary.org/obo/DOID_1324), lung cancer (http://purl.obolibrary.org/obo/DOID_1324), DOID_305 (http://purl.obolibrary.org/obo/DOID_305), carcinoma (http://purl.obolibrary.org/obo/DOID_305), RO_0002452 SOME SYMP_0020031 (urn:accblank:4:d739cbbf-ad38-49bf-8d4a-d33a50213106), d739cbbf-ad38-49bf-8d4a-d33a50213106 (urn:accblank:4:d739cbbf-ad38-49bf-8d4a-d33a50213106), 082312e3-d280-401a-a31c-d03ae3b7d1b8 (urn:accblank:4:082312e3-d280-401a-a31c-d03ae3b7d1b8), RO_0002452 SOME SYMP_0000576 (urn:accblank:4:082312e3-d280-401a-a31c-d03ae3b7d1b8), 0d112a82-eaa0-4c9a-90a8-b75b749d050d (urn:accblank:4:0d112a82-eaa0-4c9a-90a8-b75b749d050d), RO_0001000 SOME CL_0000066 (urn:accblank:4:0d112a82-eaa0-4c9a-90a8-b75b749d050d), RO_0002452 SOME SYMP_0000517 (urn:accblank:4:b37c4c5b-06e0-468f-b10f-f0225fee1134), b37c4c5b-06e0-468f-b10f-f0225fee1134 (urn:accblank:4:b37c4c5b-06e0-468f-b10f-f0225fee1134), RO_0002452 SOME SYMP_0000178 (urn:accblank:4:ea6f9eb5-784a-4138-b9b0-1ca05f60b7bb), ea6f9eb5-784a-4138-b9b0-1ca05f60b7bb (urn:accblank:4:ea6f9eb5-784a-4138-b9b0-1ca05f60b7bb), RO_0002452 SOME SYMP_0000614 (urn:accblank:4:b13a8137-b9b3-4bf6-b539-7e07be3fd2b2), b13a8137-b9b3-4bf6-b539-7e07be3fd2b2 (urn:accblank:4:b13a8137-b9b3-4bf6-b539-7e07be3fd2b2)
                """;

        /*
         * it also fails when using E5SmallV2QuantizedEmbeddingModel
         */
        EmbeddingModel embeddingModel = new E5SmallV2EmbeddingModel();

        /*
         * this call fails after v.0.27.1 but only with certain input texts like this one
         */
        assertDoesNotThrow(() -> embeddingModel.embed(sampleText));
    }

Expected behavior
Embedding works without errors :)

Please complete the following information:

  • LangChain4j version: 0.30.0 (embedding worked until 0.27.1 and fails since)
  • LLM(s) used: E5SmallV2EmbeddingModel, E5SmallV2QuantizedEmbeddingModel
  • Java version: eclipse adoptium jdk-21.0.0.35
  • Spring Boot version (if applicable): 3.2.5

Additional context
Windows 11, but also fails on our Linux servers

@codehustler codehustler added the bug Something isn't working label Apr 23, 2024
@langchain4j langchain4j added the P2 High priority label Apr 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working P2 High priority
Projects
None yet
Development

No branches or pull requests

2 participants