使用 MongoDB4.0 进行密码认证,无法成功接入数据源。

[lchben]

What happened?

已实验验证过。在保证测试环境不变，仅进行mongodb升级降级。升级MongoDB 7.0可正常链接。降级为4.0.10Dashboard 显示为数据源已断开。重复两次升级降级测试。结果一致。查看/var/log/emqx 日志如下

2024-04-16T13:17:03.390570+08:00 [error] Supervisor: {<0.929796.0>,ecpool_pool_sup}. Context: start_error. Reason: {shutdown,{failed_to_start_child,{worker,1},{{error,{op_msg_response,#{<<"$clusterTime">> => #{<<"clusterTime">> => {mongostamp,1,1713244619},<<"signature">> => #{<<"hash">> => {bin,bin,<<73,188,197,69,100,177,143,52,126,37,48,181,112,249,175,84,30,141,25,239>>},<<"keyId">> => 7341261941194620929}},<<"code">> => 59,<<"codeName">> => <<"CommandNotFound">>,<<"errmsg">> => <<"no such command: 'hello'">>,<<"ok">> => 0.0,<<"operationTime">> => {mongostamp,1,1713244619}}}},[{mc_connection_man,op_msg_raw_result,2,[{file,"mc_connection_man.erl"},{line,59}]},{mc_worker_api,command,2,[{file,"mc_worker_api.erl"},{line,375}]},{mc_topology_logics,validate_server_and_config,3,[{file,"mc_topology_logics.erl"},{line,140}]},{mc_topology,init,1,[{file,"mc_topology.erl"},{line,68}]},{gen_server,init_it,2,[{file,"gen_server.erl"},{line,980}]},{gen_server,init_it,6,[{file,"gen_server.erl"},{line,935}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,241}]}]}}}. Offender: id=worker_sup,pid=undefined.

What did you expect to happen?

希望能解决此问题，我们对接的服务只有MongoDB4.0版本

How can we reproduce it (as minimally and precisely as possible)?

No response

Anything else we need to know?

No response

EMQX version

$ ./bin/emqx_ctl broker
# paste output here
[5.6.0 (Open Source)](https://www.emqx.com/zh/changelogs/broker/5.6.0)

OS version

# On Linux:
$ cat /etc/os-release
# paste output here
$ uname -a
# paste output here
NAME="CentOS Linux"
VERSION="8 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="8"
PLATFORM_ID="platform:el8"
PRETTY_NAME="CentOS Linux 8 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:8"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-8"
CENTOS_MANTISBT_PROJECT_VERSION="8"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="8"

Log files

[lchben]

[zmstone]

请问可以提供一下mongo的具体部署方式吗？我们来复现一下。

[zmstone]

OK. The automatic version detection does not work quite well with MongoDB v4.
You can however set this flag to make it work:

[lchben]

还行。自动版本检测在 MongoDB v4 中效果不佳。但是，您可以设置此标志以使其正常工作：

@zmstone 我需要如何找到此设置？我使用5.6.1 (Open Source)版本。但是操作界面没有这项设置。另外如果mongodb不开启秘钥认证，EMQX是可以正常链接的。mongodb4.0开启秘钥认证才出现的上述现象。

[zmstone]

抱歉，认证配置中可能没有包含这个选项。我们会在后续版本中加上
可以试一下这个命令：
emqx eval application:set_env(mongodb,use_legacy_protocol,true).

[nikosheng]

Hi @zmstone 我成功設置了這個指令後, 是否這個指令直接應用到所有目前的 mongo authentication?

[zmstone]

@nikosheng 是的。

[nikosheng]

Hi @zmstone , 我嘗試了一下, 發現通過emqx ctl 或者修改 config 文件都是不行, 如下是我的 emqx.conf

node {
  name = "[email protected]"
  cookie = "emqxsecretcookie"
  data_dir = "/var/lib/emqx"
}

cluster {
  name = emqxcl
  discovery_strategy = manual
}

mongodb {
  use_legacy_protocol = "true"
}

請問我可以在哪裏可以看到這個 config 已經應用到 emqx 的運行時?

[zmstone]

mongodb不能这样配置。

可以这样: 在集群中所有节点都修改： data/config/cluster.hocon 然后重启

authentication = [
  {
    backend = mongodb
    collection = users
    database = mqtt
    filter {
      username = "${username}"
    }
    mechanism = password_based
    mongo_type = single
    password_hash_algorithm {name = sha256, salt_position = suffix}
    password_hash_field = password_hash
    pool_size = 8
    salt_field = salt
    server = "127.0.0.1:27017"
    srv_record = false
    ssl {enable = false, verify = verify_peer}
    topology {
      connect_timeout_ms = "20s"
    }
    use_legacy_protocol = true # <================ 把这行加上
  }
]

[nikosheng]

Thanks @zmstone

However, I still cannot connect to my mongo v4 after I configure use_legacy_protocol to true (after emqx restart).

My mongo version is

Using MongoDB:		4.2.14

Here is the error log

2024-05-24T02:19:43.218437+00:00 [warning] msg: start_resource_failed, id: <<"emqx_authn_mongodb:1">>, reason: {start_pool_failed,<<"emqx_authn_mongodb:1">>,{connect_failed,{saslStart,#{<<"code">> => 18,<<"codeName">> => <<"AuthenticationFailed">>,<<"errmsg">> => <<"Authentication failed.">>}}}}

Here is my cluster.hocon

authentication = [
  {
    backend = mongodb
    collection = users
    database = mqtt
    enable = true
    filter {
      username = "${username}"
    }
    is_superuser_field = is_superuser
    mechanism = password_based
    mongo_type = single
    password = xxx
    password_hash_algorithm {name = sha256, salt_position = suffix}
    password_hash_field = password_hash
    pool_size = 8
    salt_field = salt
    server = "127.0.0.1:27019"
    srv_record = false
    ssl {
      ciphers = []
      depth = 10
      enable = false
      hibernate_after = 5s
      log_level = notice
      reuse_sessions = true
      secure_renegotiate = true
      verify = verify_none
      versions = [tlsv1.3, tlsv1.2]
    }
    topology {
      connect_timeout_ms = 20s
      heartbeat_frequency_ms = 200s
      max_overflow = 0
    }
    use_legacy_protocol = true
    username = eqmx
  }
]

[nikosheng]

Besides, I assume that the original setting auto can help to decide whether to use the latest mongo api.

use_legacy_protocol

Type	Enum(auto,true,false)
Default	auto
Description	Whether to use MongoDB's legacy protocol for communicating with the database. The default is to attempt to automatically determine if the newer protocol is supported.

[zmstone]

FYI. 5.7.0 have it added to the UI.

[zmstone]

2024-05-24T02:19:43.218437+00:00 [warning] msg: start_resource_failed, id: <<"emqx_authn_mongodb:1">>, reason: {start_pool_failed,<<"emqx_authn_mongodb:1">>,{connect_failed,{saslStart,#{<<"code">> => 18,<<"codeName">> => <<"AuthenticationFailed">>,<<"errmsg">> => <<"Authentication failed.">>}}}}

This log hints bad username password for mongodb connector.

[zmstone]

Besides, I assume that the original setting auto can help to decide whether to use the latest mongo api.

auto is explained here:
https://github.com/emqx/mongodb-erlang/blob/c70e158fb8b67a0034ad3c17ed544fa29d11d358/src/main/mc_worker_pid_info.erl#L131-L151

If there is any doubt about whether or not the auto resolution is trust-worth:

• For mogodb after 5.1,use_legacy_protocol=false ref
• For mogodb before 3.6, use_legacy_protocol=true ref
• Mogodb version earlier than 5.1, but newer than 3.6, technically, it supports both versions, but we don't know which works the best.