Hybrid search returns wrong result - 混合检索返回的内容不正确

[daocunyang]

（Chinese is below）
We are using Milvus 2.3. We have some queries like the following, which use filtering:

client = MilvusClient(uri="http://*****:19530", db_name="*****")
    res = client.search(
        collection_name="****",
        data=[embed],
        search_params={
            'metric_type': 'L2',
            'params': {
                'ef': 250
            }
        },
        filter='node_id in [0, 55]',
        output_fields=["node_id", "text", "描述"],
        limit=4,
    )

Like in the above scenario, we have some data associated with node_id 0 and node_id 55. We know for sure that in our collection, there is one data entry (let's call it A) associated with node_id 55. But the problem is, when we use the code above (filter='node_id in [0, 55]') to query for A, Milvus does not return A even though it is an exact match, but instead returns us some other data that is less similar, which is a problem. Once we remove the filter part, it works (A gets returned), but that's not what we want. (we need the filtering functionality). One thing to note is, when using Zilliz Cloud there is no problem - it only seems to be a problem with the open source Milvus.

Can anyone help? Thanks

=======================

如上边代码所示，我们使用混合检索，在ANN检索前先根据node_id做标量检索。现在遇到的问题是，例如，我们库里与node_id=55相关的有一条数据为“你好”，这是我们确认存在的。我们现在用上面的代码，用"node_id in [0, 55]"这个条件去检索“你好”这条数据，但是却检索不到，milvus不会返回“你好”，而是返回一些其他的内容。如果我们去掉filter的部分，检索就没问题，就可以返回“你好”了，但这不是我们想要的，我们不希望全量搜索，而是希望用filter并且也能得到正确的结果。我们试过用Zilliz Cloud上的Milvus就没有这个问题，开源的版本就有这个问题。使用的版本是Milvus 2.3。请问这个问题该如何解决？谢谢

[yhmo]

Because the Zilliz cloud is using a special index named "Cardinal": https://zilliz.com/blog/cardinal-most-performant-vector-search-engine#What-is-Cardinal

If you search with filter expression with normal indexes such as HNSW/IVF, some results might be lost due to the algorithm of the indexes.
But with Cardinal index you can get both high performance and high recall rate, it is not open-source.

[daocunyang]

@yhmo Got it thanks. Other than using Cardinal/HNSW/IVF indexes, is there any other way to solve this issue? We need to use the open-source version

[yhmo]

I am afraid no index can get both high performance and high recall rate except the Cardinal.