Randomly disconnecting: "Server connection failed, not responding to TCP pings"

[pa5tlife]

Can anyone help me discover why mumble is randomly disconnecting? I recently switched hosts and moved my website, my SSL certificates, and mumble server over to a new VPS. This wasn't happening before I moved hosts. One time when this happened, I logged in through a terminal and the server was so unresponsive I couldn't type. Didn't resolve until I did a hard reboot at the host's panel.

About Server
It's a single virtual CPU with 1GB RAM and 25GB NVME drive & 2000GB bandwidth
Running Apache2 with 4 virtual hosts serving up very small and simple static websites + my mumble server.
murmurd -- 1.3.4-1
Debian 11 bullseye kernel 5.10.0-16-amd64
Free memory is around 42% (using 561MB out of 976MB)

I check the mumble log and the only thing I see during the times it's randomly disconnecting was this:

2022-07-18 23:36:53.628 1 => <2:USERNAME(1)> Timeout
2022-07-18 23:36:53.633 1 => <2:USERNAME(1)> Connection closed:

kern.log is basically a constant barrage of UFW blocks
I have fail2ban set with jail.local sending bad SSH attempts to the jail.

Server is secured with SSH keypair authentication and root login disabled.

About client
Mumble (1.3.4-1)
Debian 11 bullseye kernel 5.10.0-16-amd64

I have experienced this with "Force TCP mode" ticked on and off. Same result.
Thanks for your help.

[Krzmbrzl]

To me this seems as if the underlying issue is that for some reason you are really losing the connection to your host - aka you can't reach it over the network anymore. At least long enough for Mumble to decide that the connection is probably dead and so automatically performs a disconnect.

Did you check whether you can successfully ping your server when your Mumble client disconnects?

[pa5tlife]

Working on that, it reconnects so fast that by the time I try to do this it reconnects. Today I'll have time to hold a terminal open just waiting to test this.

I have another webserver that I did the exact same thing to move it to a new host, except there is only 3 virtual hosts on that one. It is flawless so far.

[pa5tlife]

So apparently this issue only happens when I'm live streaming on Odysee. Perhaps there is an issue with routing somewhere that causes client to disconnect? Maybe it's my wiring in the house. Cant test unless I'm live, and I cant do that til Friday or Saturday night.

I have been connected to the server almost continuously since I posted without a hiccup. Strange.

[Krzmbrzl]

Oh actually this should be purely client-side, unless I am misremembering something here 🤔

[pa5tlife]

I learned that this issue is not present in 1.3.4 in Windows. I've had 2 computers connected to it steady for many days. So it has to be another client side issue besides whatever was fixed it the latest versions right?

[pa5tlife]

OK so I switched to Ubuntu MATE to get newer versions of software and support for hardware I needed.

I recently built the latest version in Ubuntu MATE 22.04 and I'm having issues already with audio.
The first screen was selecting the audio service and I selected PulseAudio the it crashed and now I cannot run mumble client at all.

Some questions now:
1.) Should I have left it as Pipewire (which was pre-selected) ?
2.) How do I reset to default settings?

This is the output from the terminal:

./mumble

<D>2022-11-07 11:45:37.737 libopus 1.3.1-97-g6b6035ae from /home/user/Tools/mumble/build/libopus.so.0
<D>2022-11-07 11:45:37.754 JACK 1.9.20 from libjack.so.0
<X>2022-11-07 11:45:37.755 PipeWire 0.3.48 from libpipewire-0.3.so.0
ALSA lib pcm_dmix.c:1032:(snd_pcm_dmix_open) unable to open slave
ALSA lib pcm.c:2664:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.rear
ALSA lib pcm.c:2664:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.center_lfe
ALSA lib pcm.c:2664:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.side
ALSA lib pcm_route.c:877:(find_matching_chmap) Found no matching channel map
Cannot connect to server socket err = No such file or directory
Cannot connect to server request channel
jack server is not running or cannot be started
JackShmReadWritePtr::~JackShmReadWritePtr - Init not done for -1, skipping unlock
JackShmReadWritePtr::~JackShmReadWritePtr - Init not done for -1, skipping unlock
Cannot connect to server socket err = No such file or directory
Cannot connect to server request channel
jack server is not running or cannot be started
JackShmReadWritePtr::~JackShmReadWritePtr - Init not done for -1, skipping unlock
JackShmReadWritePtr::~JackShmReadWritePtr - Init not done for -1, skipping unlock
ALSA lib pcm_oss.c:397:(_snd_pcm_oss_open) Cannot open device /dev/dsp
ALSA lib pcm_oss.c:397:(_snd_pcm_oss_open) Cannot open device /dev/dsp
ALSA lib confmisc.c:160:(snd_config_get_card) Invalid field card
ALSA lib pcm_usb_stream.c:482:(_snd_pcm_usb_stream_open) Invalid card 'card'
ALSA lib confmisc.c:160:(snd_config_get_card) Invalid field card
ALSA lib pcm_usb_stream.c:482:(_snd_pcm_usb_stream_open) Invalid card 'card'
ALSA lib pcm_dmix.c:1032:(snd_pcm_dmix_open) unable to open slave
Cannot connect to server socket err = No such file or directory
Cannot connect to server request channel
jack server is not running or cannot be started
JackShmReadWritePtr::~JackShmReadWritePtr - Init not done for -1, skipping unlock
JackShmReadWritePtr::~JackShmReadWritePtr - Init not done for -1, skipping unlock
<D>2022-11-07 11:45:37.841 PortAudio V19.6.0-devel, revision 396fe4b6699ae929d3a685b3ef8a7e97396139a4 from libportaudio.so.2
<D>2022-11-07 11:45:37.842 PulseAudio 15.99.0 from libpulse.so.0
<W>2022-11-07 11:45:37.844 Theme: "Mumble"
<W>2022-11-07 11:45:37.844 Style: "Lite"
<W>2022-11-07 11:45:37.844 --> qss: ":themes/Default/Lite.qss"
<W>2022-11-07 11:45:37.845 Locale is "en_US" (System: "en_US")
<W>2022-11-07 11:45:37.872 Database SQLite: "3.37.2"
<W>2022-11-07 11:45:37.899 GlobalShortcutX: Using XI2 2.4
<W>2022-11-07 11:45:37.999 QMetaObject::connectSlotsByName: No matching signal for on_qtvUsers_customContextMenuRequested(QPoint,bool)
<W>2022-11-07 11:45:38.138 AudioInput: Opus encoder set for high quality speech
<W>2022-11-07 11:45:38.138 AudioInput: 40000 bits/s, 48000 hz, 480 sample
<W>2022-11-07 11:45:38.142 AudioInput: Initialized mixer for 1 channel 48000 hz mic and 0 channel 48000 hz echo
<W>2022-11-07 11:45:38.142 AudioOutput: Initialized 2 channel 48000 hz mixer
<W>2022-11-07 11:45:38.170 AudioInput: Opus encoder set for high quality speech
<W>2022-11-07 11:45:38.170 AudioInput: 40000 bits/s, 48000 hz, 480 sample
<W>2022-11-07 11:45:38.172 AudioInput: Initialized mixer for 1 channel 48000 hz mic and 0 channel 48000 hz echo
<W>2022-11-07 11:45:38.173 AudioOutput: Initialized 2 channel 48000 hz mixer
<W>2022-11-07 11:45:38.173 AudioInput: Opus encoder set for high quality speech
<W>2022-11-07 11:45:38.173 AudioInput: 40000 bits/s, 48000 hz, 480 sample
<W>2022-11-07 11:45:38.175 AudioInput: Initialized mixer for 1 channel 48000 hz mic and 0 channel 48000 hz echo
<W>2022-11-07 11:45:38.176 AudioOutput: Initialized 2 channel 48000 hz mixer
<W>2022-11-07 11:45:38.211 Updating application palette
<W>2022-11-07 11:46:16.029 AudioInput: Opus encoder set for high quality speech
<W>2022-11-07 11:46:16.029 AudioInput: 40000 bits/s, 48000 hz, 480 sample
<W>2022-11-07 11:46:16.035 AudioInput: Initialized mixer for 1 channel 48000 hz mic and 0 channel 48000 hz echo
<W>2022-11-07 11:46:16.037 AudioOutput: Initialized 2 channel 48000 hz mixer
<W>2022-11-07 11:46:21.424 AudioInput: Opus encoder set for high quality speech
<W>2022-11-07 11:46:21.424 AudioInput: 40000 bits/s, 48000 hz, 480 sample
<W>2022-11-07 11:46:21.427 AudioInput: Initialized mixer for 1 channel 48000 hz mic and 0 channel 48000 hz echo
<W>2022-11-07 11:46:23.395 AudioInput: Opus encoder set for high quality speech
<W>2022-11-07 11:46:23.395 AudioInput: 40000 bits/s, 48000 hz, 480 sample
<W>2022-11-07 11:46:23.398 AudioInput: Initialized mixer for 1 channel 48000 hz mic and 0 channel 48000 hz echo
<W>2022-11-07 11:47:51.055 AudioInput: Opus encoder set for high quality speech
<W>2022-11-07 11:47:51.056 AudioInput: 40000 bits/s, 48000 hz, 480 sample
<W>2022-11-07 11:47:51.056 PulseAudio: Starting input alsa_input.usb-046d_Logitech_StreamCam_54B18175-02.analog-stereo
<W>2022-11-07 11:47:51.062 AudioInput: Initialized mixer for 1 channel 48000 hz mic and 0 channel 48000 hz echo
<W>2022-11-07 11:47:51.067 AudioInput: Using Speex as noise canceller
warning: The VAD has been replaced by a hack pending a complete rewrite
<W>2022-11-07 11:47:54.343 PulseAudio: Starting output: alsa_output.pci-0000_00_1f.3.analog-stereo
<W>2022-11-07 11:47:54.344 AudioOutput: Initialized 2 channel 48000 hz mixer
Fatal (internal) error in /home/user/Tools/mumble/3rdparty/opus/celt/celt_decoder.c, line 118: assertion failed: st->mode == opus_custom_mode_create(48000, 960, NULL)
Aborted (core dumped)

The line in that .c file is this:
celt_assert(st->mode == opus_custom_mode_create(48000, 960, NULL));

Thanks so much for your help

[Krzmbrzl]

You are suffering from the known Opus crash. Using -Dbundled-opus=OFF in a completely fresh build should fix the issue.

Restoring default settings can be performed by deleting the settings JSON file (should be somewhere in ~/.config/Mumble/Mumble)

[pa5tlife]

Thanks I succeeded in building and connected. I'll be testing a bit today. My kids PCs (the Windows ones I mentioned) are running latest version after all, not 1.3.4. They are still connected to my server since Sunday. Very stable application!!

[pa5tlife]

I've been doing a lot of testing and have a few print outs of syslogs during times it disconnects. The issue is persistent on all version of mumble I've tried, and on both Windows, and Linux. It happens both on Ethernet and Wifi. The Linux computers show dhcp6 state changed new lease in the log during the time that it disconnects. IDK how to do anything on Windows though honestly.

I bought a Klein Tools ethernet cable tester to test all my terminations and cables. Everything checks out as good. I'm wondering now if there is something wrong with the router.

I always see this in logs when it disconnects:
NetworkManager[850]: <info> [1688078340.9929] dhcp6 (eno1): state changed new lease, address=2600:000:8600:73f0::47

[davidebeatrici]

Looks like it's often giving you a new IPv6 address and immediately discarding the previous one, causing the TCP connection to drop.