[Bug]: [ERROR] [components/root_coord.go:55] #33085
Assignees
Labels
kind/bug
Issues or changes related a bug
triage/needs-information
Indicates an issue needs more information in order to work on it.
Comments
Storm0921
added
kind/bug
|
I am not sure what release are you running, please retry on milvus 2.3.15 or 2.4.1 |
yanliang567
added
triage/needs-information
|
seems that your etcd is running. |
|
Hello, I am facing the same problem with a self-built milvus (I build milvus following the instruction here: https://github.com/milvus-io/milvus/blob/master/DEVELOPMENT.md#docker--docker-compose), the milvus version is 2.4.2. I used command line May I know how to solve the problem? Thank you for the help! |
etcd seems not running as expect. did you check the etcd docker ? |
|
May I know how to check the etcd docker? I just ran the milvus locally,
didn’t start docker containers.
- Runyu
…On Wed, May 22, 2024 at 22:48 Xiaofan ***@***.***> wrote:
Hello,
I am facing the same problem with a self-built milvus (I build milvus
following the instruction here:
https://github.com/milvus-io/milvus/blob/master/DEVELOPMENT.md#docker--docker-compose
<https://urldefense.com/v3/__https://github.com/milvus-io/milvus/blob/master/DEVELOPMENT.md*docker--docker-compose__;Iw!!IKRxdwAv5BmarQ!dVJQi6nxsnNwMFk69x2gcI6Wx1QDksqBR-gwW0uOwoLixu-EwRw5D9yk5OGRnbFs_BdZGkE0T1ws0wzG_Zl61qYfLFQ$>),
the milvus version is 2.4.2. I used command line sudo
./scripts/start_standalone.sh. I am pasting my error message here:
"/tmp/standalone.log" [readonly] 1189L, 85523B 1,2 Top
[2024/05/22 20:35:46.536 +00:00] [INFO] [distance/calc_distance_amd64.go:14] ["Hook avx for go simd distance computation"]
2024/05/22 20:35:46 run subprocess with cmd: [../bin/milvus run standalone]
[2024/05/22 20:35:46.842 +00:00] [INFO] [distance/calc_distance_amd64.go:14] ["Hook avx for go simd distance computation"]
2024/05/22 20:35:46 maxprocs: Leaving GOMAXPROCS=32: CPU quota undefined
__ _________ _ ____ ______
/ |/ / _/ /| | / / / / / __/
/ /|_/ // // /_| |/ / /_/ /\ \
/_/ /_/___/____/___/\____/___/
Welcome to use Milvus!
Version: v2.4.2-70-ge1bafd710
Built: Wed May 22 18:45:19 UTC 2024
GitCommit: e1bafd7
GoVersion: go version go1.21.0 linux/amd64
TotalMem: 135048794112
UsedMem: 35815424
open pid file: /run/milvus/standalone.pid
lock pid file: /run/milvus/standalone.pid
[2024/05/22 20:35:46.869 +00:00] [INFO] [roles/roles.go:306] ["starting running Milvus components"]
[2024/05/22 20:35:46.883 +00:00] [DEBUG] [config/refresher.go:67] ["start refreshing configurations"] [source=FileSource]
[2024/05/22 20:35:46.885 +00:00] [DEBUG] [config/etcd_source.go:50] ["init etcd source"] [etcdInfo="{\"UseEmbed\":false,\"EnableAuth\":false,\"UserName\":\"\",\"PassWord\":\"\",\"UseSSL\":false,\"Endpoints\":[\"localhost:2379\"],\"KeyPrefix\":\"by-dev\",\"CertFile\":\"/path/to/etcd-client.pem\",\"KeyFile\":\"/path/to/etcd-client-key.pem\",\"CaCertFile\":\"/path/to/ca.pem\",\"MinVersion\":\"1.3\",\"RefreshInterval\":5000000000}"]
[2024/05/22 20:35:46.885 +00:00] [INFO] [etcd/etcd_util.go:47] ["create etcd client"] [useEmbedEtcd=false] [useSSL=false] [endpoints="[localhost:2379]"] [minVersion=1.3]
[2024/05/22 20:35:51.886 +00:00] [INFO] [paramtable/base_table.go:209] ["init with etcd failed"] [error="context deadline exceeded"]
[2024/05/22 20:35:51.895 +00:00] [INFO] [paramtable/hook_config.go:21] ["hook config"] [hook={}]
[2024/05/22 20:35:51.895 +00:00] [INFO] [roles/roles.go:255] [setupPrometheusHTTPServer]
[2024/05/22 20:35:51.895 +00:00] [DEBUG] [config/refresher.go:67] ["start refreshing configurations"] [source=FileSource]
[2024/05/22 20:35:51.895 +00:00] [INFO] [rootcoord/root_coord.go:154] ["update rootcoord state"] [state=Abnormal]
[2024/05/22 20:35:51.895 +00:00] [INFO] [http/server.go:112] ["management listen"] [addr=:9091]
[2024/05/22 20:35:51.895 +00:00] [DEBUG] [rootcoord/service.go:184] ["init params done.."]
[2024/05/22 20:35:51.895 +00:00] [INFO] [etcd/etcd_util.go:47] ["create etcd client"] [useEmbedEtcd=false] [useSSL=false] [endpoints="[localhost:2379]"] [minVersion=1.3]
[2024/05/22 20:35:51.886 +00:00] [INFO] [paramtable/base_table.go:209] ["init with etcd failed"] [error="context deadline exceeded"]
[2024/05/22 20:35:51.895 +00:00] [INFO] [paramtable/hook_config.go:21] ["hook config"] [hook={}]
[2024/05/22 20:35:51.895 +00:00] [INFO] [roles/roles.go:255] [setupPrometheusHTTPServer]
[2024/05/22 20:35:51.895 +00:00] [DEBUG] [config/refresher.go:67] ["start refreshing configurations"] [source=FileSource]
[2024/05/22 20:35:51.895 +00:00] [INFO] [rootcoord/root_coord.go:154] ["update rootcoord state"] [state=Abnormal]
[2024/05/22 20:35:51.895 +00:00] [INFO] [http/server.go:112] ["management listen"] [addr=:9091]
[2024/05/22 20:35:51.895 +00:00] [DEBUG] [rootcoord/service.go:184] ["init params done.."]
[2024/05/22 20:35:51.895 +00:00] [INFO] [etcd/etcd_util.go:47] ["create etcd client"] [useEmbedEtcd=false] [useSSL=false] [endpoints="[localhost:2379]"] [minVersion=1.3]
[2024/05/22 20:35:51.898 +00:00] [INFO] [etcd/etcd_util.go:47] ["create etcd client"] [useEmbedEtcd=false] [useSSL=false] [endpoints="[localhost:2379]"] [minVersion=1.3]
[2024/05/22 20:35:51.899 +00:00] [INFO] [components/index_coord.go:38] ["IndexCoord running ..."]
[2024/05/22 20:35:51.899 +00:00] [INFO] [etcd/etcd_util.go:47] ["create etcd client"] [useEmbedEtcd=false] [useSSL=false] [endpoints="[localhost:2379]"] [minVersion=1.3]
[2024/05/22 20:35:51.899 +00:00] [DEBUG] [querynode/service.go:104] [QueryNode] [port=21123]
[2024/05/22 20:35:51.900 +00:00] [INFO] [etcd/etcd_util.go:47] ["create etcd client"] [useEmbedEtcd=false] [useSSL=false] [endpoints="[localhost:2379]"] [minVersion=1.3]
[2024/05/22 20:35:51.900 +00:00] [INFO] [etcd/etcd_util.go:47] ["create etcd client"] [useEmbedEtcd=false] [useSSL=false] [endpoints="[localhost:2379]"] [minVersion=1.3]
[2024/05/22 20:35:51.901 +00:00] [DEBUG] [indexnode/indexnode.go:115] ["New IndexNode ..."]
[2024/05/22 20:35:51.901 +00:00] [DEBUG] [indexnode/service.go:87] [IndexNode] ["network address"=10.168.113.68:21121] ["network port: "=21121]
[2024/05/22 20:35:51.901 +00:00] [INFO] [proxy/lb_policy.go:78] ["use look_aside policy on replica selection"]
[2024/05/22 20:35:51.902 +00:00] [INFO] [runtime/asm_amd64.s:1650] ["Start check query node health loop"]
[2024/05/22 20:35:51.902 +00:00] [DEBUG] [proxy/simple_rate_limiter.go:226] ["RateLimiter register for rateType"] [rateType=DMLInsert] [rateLimit=+inf] [burst=1.7976931348623157e+308]
[2024/05/22 20:35:51.902 +00:00] [DEBUG] [proxy/simple_rate_limiter.go:226] ["RateLimiter register for rateType"] [rateType=DMLDelete] [rateLimit=+inf] [burst=1.7976931348623157e+308]
[2024/05/22 20:35:51.902 +00:00] [INFO] [hookutil/hook.go:46] ["empty so path, skip to load plugin"]
[2024/05/22 20:35:51.902 +00:00] [DEBUG] [proxy/service.go:122] ["create a new Proxy instance"] [state=2]
[2024/05/22 20:35:51.902 +00:00] [DEBUG] [proxy/service.go:423] ["init Proxy server"]
[2024/05/22 20:35:51.902 +00:00] [DEBUG] [proxy/service.go:454] ["Proxy init service's parameter table done"]
[2024/05/22 20:35:51.902 +00:00] [DEBUG] [proxy/service.go:456] ["Proxy init http server's parameter table done"]
[2024/05/22 20:35:51.902 +00:00] [DEBUG] [proxy/service.go:463] ["init Proxy's parameter table done"] [internalAddress=10.168.113.68:19529] [externalAddress=10.168.113.68:19530]
[2024/05/22 20:35:51.902 +00:00] [INFO] [accesslog/global.go:146] ["Init access logger success"]
[2024/05/22 20:35:51.902 +00:00] [DEBUG] [proxy/service.go:470] ["init Proxy's tracer done"] ["service name"="Proxy ip: 10.168.113.68, port: 19530"]
[2024/05/22 20:35:51.902 +00:00] [INFO] [etcd/etcd_util.go:47] ["create etcd client"] [useEmbedEtcd=false] [useSSL=false] [endpoints="[localhost:2379]"] [minVersion=1.3]
[2024/05/22 20:35:52.002 +00:00] [INFO] [etcd/etcd_util.go:47] ["create etcd client"] [useEmbedEtcd=false] [useSSL=false] [endpoints="[localhost:2379]"] [minVersion=1.3]
[2024/05/22 20:35:56.896 +00:00] [DEBUG] [rootcoord/service.go:198] ["RootCoord connect to etcd failed"] [error="context deadline exceeded"]
[2024/05/22 20:35:56.896 +00:00] [ERROR] [components/root_coord.go:55] ["RootCoord starts error"] [error="context deadline exceeded"] [stack="github.com/milvus-io/milvus/cmd/components.(*RootCoord).Run\n\t/home/lisa/milvus/milvus_code/milvus/cmd/components/root_coord.go:55\ngithub.com/milvus-io/milvus/cmd/roles.runComponent[...].func1\n\t/home/lisa/milvus/milvus_code/milvus/cmd/roles/roles.go:113 <http://github.com/milvus-io/milvus/cmd/components.(*RootCoord).Run%5Cn%5Ct/home/lisa/milvus/milvus_code/milvus/cmd/components/root_coord.go:55%5Cngithub.com/milvus-io/milvus/cmd/roles.runComponent%5B...%5D.func1%5Cn%5Ct/home/lisa/milvus/milvus_code/milvus/cmd/roles/roles.go:113>"]
panic: context deadline exceeded
goroutine 230 [running]:
panic({0x53975c0?, 0x8584b40?})
/usr/local/go/src/runtime/panic.go:1017 +0x3ac fp=0xc0017ddf70 sp=0xc0017ddec0 pc=0x1dec88cgithub.com/milvus-io/milvus/cmd/roles.runComponent[...].func1() <http://github.com/milvus-io/milvus/cmd/roles.runComponent%5B...%5D.func1()>
/home/lisa/milvus/milvus_code/milvus/cmd/roles/roles.go:114 +0x108 fp=0xc0017ddfe0 sp=0xc0017ddf70 pc=0x4d399a8
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0017ddfe8 sp=0xc0017ddfe0 pc=0x1e260c1
created by github.com/milvus-io/milvus/cmd/roles.runComponent[.. <http://github.com/milvus-io/milvus/cmd/roles.runComponent%5B..>.] in goroutine 1
/home/lisa/milvus/milvus_code/milvus/cmd/roles/roles.go:105 +0x138
goroutine 1 [semacquire]:
runtime.gopark(0x0?, 0xc0017a4c70?, 0x60?, 0x49?, 0xc0010922c0?)
/usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc001a1f630 sp=0xc001a1f610 pc=0x1df08ae
runtime.goparkunlock(...)
/usr/local/go/src/runtime/proc.go:404
runtime.semacquire1(0xc0019a4598, 0x60?, 0x1, 0x0, 0x1?)
May I know how to solve the problem? @xiaofan-luan
<https://urldefense.com/v3/__https://github.com/xiaofan-luan__;!!IKRxdwAv5BmarQ!dVJQi6nxsnNwMFk69x2gcI6Wx1QDksqBR-gwW0uOwoLixu-EwRw5D9yk5OGRnbFs_BdZGkE0T1ws0wzG_Zl6XX49IkQ$>
mentioned about the etcd, may I know how to check if my etcd is running?
Thank you for the help!
etcd seems not running as expect. did you check the etcd docker ?
—
Reply to this email directly, view it on GitHub
<https://urldefense.com/v3/__https://github.com/milvus-io/milvus/issues/33085*issuecomment-2126284047__;Iw!!IKRxdwAv5BmarQ!dVJQi6nxsnNwMFk69x2gcI6Wx1QDksqBR-gwW0uOwoLixu-EwRw5D9yk5OGRnbFs_BdZGkE0T1ws0wzG_Zl6eUggP4I$>,
or unsubscribe
<https://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/ADMNK6TZH4YJ2E4I7DGCDF3ZDV7JPAVCNFSM6AAAAABHZKZ4UGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMRWGI4DIMBUG4__;!!IKRxdwAv5BmarQ!dVJQi6nxsnNwMFk69x2gcI6Wx1QDksqBR-gwW0uOwoLixu-EwRw5D9yk5OGRnbFs_BdZGkE0T1ws0wzG_Zl6WYqKq9g$>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
|
|
Hello, I don't have etcd docker container running. I only built Milvus and then run ./bin/milvus, does this step automatically start etcd container? Or how can I start etcd container manually? Thanks |
|
Hello, are there any updates on the issue? |
|
/assign @czs007 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Is there an existing issue for this?
Environment
Current Behavior
我运行下面的命令得到的日志如下，怎么解决这个问题？
nohup ./bin/milvus run standalone > running.log 2>&1 &有时候还会出现Datacord的类似问题
Expected Behavior
No response
Steps To Reproduce
No response
Milvus Log
No response
Anything else?
No response
The text was updated successfully, but these errors were encountered: