[Bug]: Can not find pymilvus.model.hybrid

[vuminhquang]

Is there an existing issue for this?

• I have searched the existing issues

Environment

- Milvus version: 2.4.0
- Deployment mode(standalone or cluster): Lite
- MQ type(rocksmq, pulsar or kafka):    
- SDK version(e.g. pymilvus v2.0.0rc2): Latest (pip install -U pymilvus, pip install -U pymilvus[model])
- OS(Ubuntu or CentOS): Ubuntu WSL2
- CPU/Memory: 
- GPU: No
- Others:

Current Behavior

Can not import from pymilvus.model.hybrid

Expected Behavior

This should work
from pymilvus.model.hybrid import BGEM3EmbeddingFunction

Steps To Reproduce

No response

Milvus Log

No response

Anything else?

No response

[pavluka6]

Have you tried
pip install pymilvus[model]

[vuminhquang]

Yes, for more information, I did
pip install -U pymilvus
pip install -U pymilvus[model]

[yanliang567]

/assign @wxywb
/unassign

[sre-ci-robot]

@yanliang567: GitHub didn't allow me to assign the following users: wxywb.

Note that only milvus-io members, repo collaborators and people who have commented on this issue/PR can be assigned. Additionally, issues/PRs can only have 10 assignees at the same time.
For more information please see the contributor guide

In response to this:

/assign @wxywb
/unassign

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[vuminhquang]

/assign @wxywb

[sre-ci-robot]

@vuminhquang: GitHub didn't allow me to assign the following users: wxywb.

Note that only milvus-io members, repo collaborators and people who have commented on this issue/PR can be assigned. Additionally, issues/PRs can only have 10 assignees at the same time.
For more information please see the contributor guide

In response to this:

/assign @wxywb

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[wxywb]

@vuminhquang , can you run

pip list |grep milvus-model 

to show your model library status?

[khanhj]

i am able to import from pymilvus.model.hybrid import BGEM3EmbeddingFunction
my package version

pip list | grep milvus-model 
milvus-model                     0.2.0

however, i had this error when running code block:

docs = [
    "Artificial intelligence was founded as an academic discipline in 1956.",
    "Alan Turing was the first person to conduct substantial research in AI.",
    "Born in Maida Vale, London, Turing was raised in southern England.",
]

docs_embeddings = bge_m3_ef.encode_documents(docs)

print("Embeddings:", docs_embeddings)
print("Dense document dim:", bge_m3_ef.dim["dense"], docs_embeddings["dense"][0].shape)
print("Sparse document dim:", bge_m3_ef.dim["sparse"], list(docs_embeddings["sparse"])[0].shape)

>
Embeddings: {'dense': [array([-0.02505942, -0.00142191,  0.04015464, ..., -0.02094925,
        0.02623658,  0.00324097], dtype=float32), array([ 0.00118462,  0.00649289, -0.00735763, ..., -0.01446299,
        0.0424368 , -0.0179482 ], dtype=float32), array([ 0.00415286, -0.01014927,  0.00098102, ..., -0.02559664,
        0.0808467 ,  0.00141649], dtype=float32)], 'sparse': <3x250002 sparse matrix of type '<class 'numpy.float32'>'
	with 43 stored elements in COOrdinate format>}
Dense document dim: 1024 (1024,)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
[<ipython-input-11-c15f5f76006f>](https://localhost:8080/#) in <cell line: 11>()
      9 print("Embeddings:", docs_embeddings)
     10 print("Dense document dim:", bge_m3_ef.dim["dense"], docs_embeddings["dense"][0].shape)
---> 11 print("Sparse document dim:", bge_m3_ef.dim["sparse"], list(docs_embeddings["sparse"])[0].shape)

[/usr/local/lib/python3.10/dist-packages/scipy/sparse/_base.py](https://localhost:8080/#) in __iter__(self)
    250     def __iter__(self):
    251         for r in range(self.shape[0]):
--> 252             yield self[r, :]
    253 
    254     def _getmaxprint(self):

TypeError: 'coo_matrix' object is not subscriptable

Is that error on list(docs_embeddings["sparse"]) ?

pip list | grep scipy
scipy                            1.11.4

[vuminhquang]

Hi @wxywb,
This is my result
pip list | grep milvus-model
milvus-model 0.2.0

[wxywb]

@khanhj
it is supposed to be a csr_array instead of coo_matrix, It seems caused by version of scipy, I will fixed it for next release.

from scipy.sparse import csr_array
res  = list(csr_array(docs_embeddings['sparse']))[0]

this should work for now.

[wxywb]

@vuminhquang
please tell me your python version

1. check the milvus-model functionality

from milvus_model.hybrid import BGEM3EmbeddingFunction

2. if milvus-model test passed, please try to run

from pymilvus.model.hybrid import BGEM3EmbeddingFunction

and show me full stacktrace showed in your terminal.

[vuminhquang]

@wxywb
The python version is: 3.11.9
This is my check:

1. from milvus_model.hybrid import BGEM3EmbeddingFunction: OK
2. from pymilvus.model.hybrid import BGEM3EmbeddingFunction: Not passed. There is not hybrid.

[wxywb]

@vuminhquang ,could you run your code , and show information (some error or exceptions) in your bash?
It is not surprised your IDE can not find correct symbols of modules in pymilvus.model, because it used an import trick that import hybrid, dense or sparse when executing.

[vuminhquang]

Both can be import from console @wxywb, If I want to write the code directly from IDE, (not care on the beautiful of code), which class should I call directly? Is milvus_model.hybrid the way to use? Thanks.

[wxywb]

Both can be import from console @wxywb, If I want to write the code directly from IDE, (not care on the beautiful of code), which class should I call directly? Is milvus_model.hybrid the way to use? Thanks.

yes in this case, you should use milvus_model instead of pymilvus.model, we have a separate repo for it https://github.com/milvus-io/milvus-model

[vuminhquang]

Thank you @wxywb for your supporting, I'll close this bug now.