[Bug]: CPU usage is high with huge collection numbers same as #26157

[Seven2Nine]

Is there an existing issue for this?

• I have searched the existing issues

Environment

- Milvus version: 2.3.5
- Deployment mode(standalone or cluster): standalone 
- MQ type(rocksmq, pulsar or kafka): rocksmq
- SDK version(e.g. pymilvus v2.0.0rc2):
- OS(Ubuntu or CentOS): Ubuntu 
- CPU/Memory: 4vCPUs 16G--aws t3.xlarge
- GPU: 
- Others:

Current Behavior

1500 collections
each collection 100 entities.

CPU usage > 70% when idle.

Expected Behavior

https://github.com/milvus-io/milvus/issues/26157

Steps To Reproduce

No response

Milvus Log

No response

Anything else?

No response

[yanliang567]

please retry on latest milvus v2.3.12, which improved a lot for the scenarios of multi collections @Seven2Nine
/assign @Seven2Nine
/unassign

[xiaofan-luan]

Is there an existing issue for this?

• I have searched the existing issues

Environment

- Milvus version: 2.3.5
- Deployment mode(standalone or cluster): standalone 
- MQ type(rocksmq, pulsar or kafka): rocksmq
- SDK version(e.g. pymilvus v2.0.0rc2):
- OS(Ubuntu or CentOS): Ubuntu 
- CPU/Memory: 4vCPUs 16G--aws t3.xlarge
- GPU: 
- Others:

Current Behavior

1500 collections each collection 100 entities.

CPU usage > 70% when idle.

Expected Behavior

https://github.com/milvus-io/milvus/issues/26157

Steps To Reproduce

No response

Milvus Log

No response

Anything else?

No response

Did you have pprof for datacoord?
how many segments/partititons are there in each collection on avg?

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Rotten issues close after 30d of inactivity. Reopen the issue with /reopen.

[xiaofan-luan]

@yanliang567
is this still a problem in 2.4?