AssertionError when attempting to query nodes parsed via llama-parse

[kevalshah90]

I followed the llama_parse advanced_demo.ipynb[1] example and was able to parse and generate nodes.

print(len(nodes))
28

However, when i attempt to query the nodes I get an AssertionError. I am using mistralAPI and here's my re-ranker code.

from llama_index.postprocessor.flag_embedding_reranker import (
    FlagEmbeddingReranker,
)


llm = MistralAI(
                model="mistral-small-latest",
                api_key=userdata.get('MISTRAL_API_KEY')
               )


reranker = FlagEmbeddingReranker(
    top_n=5,
    model="sentence-transformers/all-MiniLM-L6-v2",
)

recursive_query_engine = recursive_index.as_query_engine(
                                                          similarity_top_k=15, 
                                                          node_postprocessors=[reranker], 
                                                          verbose=True,
                                                          llm=llm
                                                        )

raw_query_engine = raw_index.as_query_engine(
                                              similarity_top_k=15, 
                                              node_postprocessors=[reranker],
                                              llm=llm
                                            )   

---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
[<ipython-input-48-dbe09052bd34>](https://localhost:8080/#) in <cell line: 3>()
      1 query = "Query goes here"
      2 
----> 3 response_1 = raw_query_engine.query(query)
      4 print("\n***********New LlamaParse+ Basic Query Engine***********")
      5 print(response_1)

7 frames
[/usr/local/lib/python3.10/dist-packages/llama_index/core/instrumentation/dispatcher.py](https://localhost:8080/#) in wrapper(func, instance, args, kwargs)
    272             )
    273             try:
--> 274                 result = func(*args, **kwargs)
    275             except BaseException as e:
    276                 self.event(SpanDropEvent(span_id=id_, err_str=str(e)))

[/usr/local/lib/python3.10/dist-packages/llama_index/core/base/base_query_engine.py](https://localhost:8080/#) in query(self, str_or_query_bundle)
     51             if isinstance(str_or_query_bundle, str):
     52                 str_or_query_bundle = QueryBundle(str_or_query_bundle)
---> 53             query_result = self._query(str_or_query_bundle)
     54         dispatch_event(QueryEndEvent())
     55         return query_result

[/usr/local/lib/python3.10/dist-packages/llama_index/core/instrumentation/dispatcher.py](https://localhost:8080/#) in wrapper(func, instance, args, kwargs)
    272             )
    273             try:
--> 274                 result = func(*args, **kwargs)
    275             except BaseException as e:
    276                 self.event(SpanDropEvent(span_id=id_, err_str=str(e)))

[/usr/local/lib/python3.10/dist-packages/llama_index/core/query_engine/retriever_query_engine.py](https://localhost:8080/#) in _query(self, query_bundle)
    187             CBEventType.QUERY, payload={EventPayload.QUERY_STR: query_bundle.query_str}
    188         ) as query_event:
--> 189             nodes = self.retrieve(query_bundle)
    190             response = self._response_synthesizer.synthesize(
    191                 query=query_bundle,

[/usr/local/lib/python3.10/dist-packages/llama_index/core/query_engine/retriever_query_engine.py](https://localhost:8080/#) in retrieve(self, query_bundle)
    143     def retrieve(self, query_bundle: QueryBundle) -> List[NodeWithScore]:
    144         nodes = self._retriever.retrieve(query_bundle)
--> 145         return self._apply_node_postprocessors(nodes, query_bundle=query_bundle)
    146 
    147     async def aretrieve(self, query_bundle: QueryBundle) -> List[NodeWithScore]:

[/usr/local/lib/python3.10/dist-packages/llama_index/core/query_engine/retriever_query_engine.py](https://localhost:8080/#) in _apply_node_postprocessors(self, nodes, query_bundle)
    136     ) -> List[NodeWithScore]:
    137         for node_postprocessor in self._node_postprocessors:
--> 138             nodes = node_postprocessor.postprocess_nodes(
    139                 nodes, query_bundle=query_bundle
    140             )

[/usr/local/lib/python3.10/dist-packages/llama_index/core/postprocessor/types.py](https://localhost:8080/#) in postprocess_nodes(self, nodes, query_bundle, query_str)
     53         else:
     54             pass
---> 55         return self._postprocess_nodes(nodes, query_bundle)
     56 
     57     @abstractmethod

[/usr/local/lib/python3.10/dist-packages/llama_index/postprocessor/flag_embedding_reranker/base.py](https://localhost:8080/#) in _postprocess_nodes(self, nodes, query_bundle)
     71                 scores = [scores]
     72 
---> 73             assert len(scores) == len(nodes)
     74 
     75             for node, score in zip(nodes, scores):

AssertionError:

1. https://github.com/run-llama/llama_parse/blob/main/examples/demo_advanced.ipynb

[logan-markewich]

This seems like an issue with flag embedding reranker integrations. For some reason, the length of returned scores is different than the length of nodes

[kevalshah90]

@logan-markewich i see. any suggestions on work arounds or is this something that needs to be fixed in the FlagEmbeddingReranker ?

[logan-markewich]

@kevalshah90 seems like an issue in flag embedding reranker -- but you are also using a model thats not meant to be used in flag embeddings (the model you are using should be a sentence transformer rerank model) -- flag embeddings are mostly for BGE models I think