Support standard VNC protocol authentication

[trinitronx]

Given that:

• wayvnc is currently the only VNC server implementation for Wayland
• VNC standard-compliance requires the plaintext authentication protocol VNC Authentication Security Type.
  • macOS Screen Sharing VNC client will not connect to wayvnc due to this.
• Most standard VNC users already tunnel over SSH to server listening only on localhost
• Majority of VNC clients do not support encrypted authentication (e.g. built-in macOS Screen Share VNC client, RealVNC Free, TightVNC, etc...)
  • TigerVNC is the only macOS client option, but it has poor support for keyboard capture, and no scaling support
• VNC Password handling was and never will be secure. 1
  • Password storage was never secure, even if "encrypted".
  • Even wayvnc stores the password= option in a plaintext config file.

Would you be willing to revisit #176 so that standard VNC clients will work properly with wayvnc server? (Either implementing this feature or accepting a Pull Request for this.)

The option could be behind a config file key to switch on this functionality while maintaining that the user knows what they're doing is insecure (by design due to VNC basic auth protocol ¹). For example, something like insecure_plaintext_auth = true, insecure_vnc_auth = true, dont_blame_wayvnc = true ², or even please_hack_me_use_plaintext_insecure_stupid_old_vnc_auth = true.

_{1: Historically, VNC was never secure for many reasons. Plaintext auth was one problem, but encryption/decryption of the VNC password config file was also trivially possible. Full disclosure: I'm the author of vncpasswd.py, which decrypts ~/.vnc/passwd and/or Windows Registry key for this using the known insecure AES decryption key.
2: This is in reference to an option in Enterprise-level software, Nagios NRPE, dont_blame_nrpe which is actually used in practice on production servers, quite often. Engineers well-trained in security often enable it alongside firewall rules and/or secure tunnels, because they know what they're doing, and this option enables certain functionality which is otherwise unavailable.}

[any1]

I've seen no mention of plain text auth in rfc6143 or that it is required for conformance. Do you mean "VNC Authentication" (3des)? That's the only authentication method mentioned in the spec. I also do not see any mention of that being mandatory.

Majority of VNC clients do not support encrypted authentication (e.g. built-in macOS Screen Share VNC client, RealVNC Free, TightVNC, etc...)

This is inaccurate. RealVNC, for instance supports a security type which uses RSA for authentication and key derivation and AES-XTS for encryption thereafter. I would also think that the Apples support a diffie-hellman based security type (30). It is their own extension, after all. Wayvnc implements both RealVNC's RSA-AES and Apple's security type 30. I would be curious to know if it works for macOS users.

VNC Password handling was and never will be secure

I'd say it's secure enough for most users if they set the permissions correctly on their config files. It would be possible to make things more secure with a new security type, but only if all other security types are disabled.

[trinitronx]

I've seen no mention of plain text auth in rfc6143 or that it is required for conformance. Do you mean "VNC Authentication" (3des)? That's the only authentication method mentioned in the spec. I also do not see any mention of that being mandatory.

Yes, this is what I'm referring to. Given that the 3des algorithm and 8-character password limitation is very insecure by today's standards, and trivially bruteforced or decrypted, it's effectively similar to "plaintext" in practice. ¹ A bit of a tongue-in-cheek reference to that insecurity, but effectively accurate.

Although not "mandatory" per-se by the spec and "in theory", based on limited client support for the RSA-AES method, and other custom security protocol extensions made by many proprietary software vendors, "in practice" this becomes a bit more mandatory to implement as a basic level of support for insecure clients.

_{1: Some discussion of this over at TigerVNC's repo here}

[trinitronx]

This is inaccurate. RealVNC, for instance supports a security type which uses RSA for authentication and key derivation and AES-XTS for encryption thereafter.

Yeah, maybe I'm not being as clear or accurate as I could be. The idea I'm trying to get across is that although RealVNC does offer AES encryption as part of a paid/commercial packaged version, historically it wasn't offered with many free clients (whether OSS, freeware or "shareware").

[trinitronx]

Wayvnc implements both RealVNC's RSA-AES and Apple's security type 30. I would be curious to know if it works for macOS users.

🤔 Hmm... I wasn't able to get the RSA config option working in my testing. Yet, upon closer inspection it looks like this feature was added in v0.7.0. Testing the wayvnc server on Manjaro, I was only able to obtain a packaged version 0.6.2-2. So, it seems they're lagging a bit behind in releasing this. I'll try testing that method again once I can get v0.7.0 or v0.7.1.

[trinitronx]

I would be curious to know if it works for macOS users.

As a quick followup to this:

I was able to build packages neatvnc 0.7.0-1 and wayvnc version 0.7.1-1, from the AUR PKGBUILD just now.

Testing this with the RSA Security Type (rsa_private_key_file config option), using macOS Monterey's built-in Screen Sharing app shows the same error as before (when I tested VeNCrypt TLS Security Type):

Unable to communicate with “127.0.0.1:7900”. Make sure the remote computer is available and the firewall is not blocking screen sharing.

127.0.0.1:7900 in this case is an SSH tunnel forwarding to the remote host running wayvnc, also only on localhost for that system.

Again, TigerVNC does seem to support the RSA Security Type (as it also does for VeNCrypt TLS). TigerVNC connects fine after the TOFU key trust dialog box is accepted.

I'm not sure what exactly Apple's Screen Sharing.app wants from the server in order to connect successfully, other than perhaps what RealVNC's RFB protocol doc calls "VNC Authentication". (That's what I was playfully referring to as "plaintext" above, but the basic Security Type of "None" could also fit that category.)

So, based on this macOS client testing, we're still effectively left with a blocker for migrating old VNC servers to wayvnc Wayland.

_{For what it's worth, x11vnc server does work with the macOS Screen Sharing.app 🤷}

[any1]

But if you enable security type 30 via the relax_encryption config option? Does the macOS client work with that?

[trinitronx]

But if you enable security type 30 via the relax_encryption config option? Does the macOS client work with that?

Testing this suggestion: relax_encryption=true with rsa_private_key_file

macOS Screen Sharing.app still shows the same message:

[trinitronx]

But if you enable security type 30 via the relax_encryption config option? Does the macOS client work with that?

Testing this suggestion: relax_encryption=true with rsa_private_key_file

In case it helps w.r.t. testing security type 30 support... I also just tested this configuration with another paid macOS VNC client called bVNC. It fails to connect also, but crashes wayvnc with the following debug log messages:

Info: New client connection from localhost: 0x561a7c26c680 (ref 1)
DEBUG: ../neatvnc/src/server.c: 183: close_after_write(0x561a7c26c680): ref 1
Info: Closing client connection 0x561a7c26c680: ref 0
Info: New client connection from localhost: 0x561a7c26c680 (ref 1)
DEBUG: ../neatvnc/src/server.c: 788: Client chose security type: 30
Info: User "d�V�Q����M�,�R6��ߐh���*6�q�=I"���;����6B��2�,ca��ND�@" rejected
DEBUG: ../neatvnc/src/server.c: 183: close_after_write(0x561a7c26c680): ref 1
free(): double free detected in tcache 2
[1]    2355153 IOT instruction (core dumped)  wayvnc --gpu --performance --output=HDMI-A-1 --log-level=debug

## From SSH tunnel:
channel 5: open failed: connect failed: Connection refused

macOS Screen Sharing.app does not crash wayvnc, but I also don't see any message about security type: 30. Logs from an attempted connection from the built-in macOS VNC client:

Info: New client connection from localhost: 0x562c3aeda680 (ref 1)
DEBUG: ../neatvnc/src/server.c: 183: close_after_write(0x562c3aeda680): ref 1
Info: Closing client connection 0x562c3aeda680: ref 0

[trinitronx]

Some more testing, using another macOS VNC client called Screens 4

• RSA-AES security type (only rsa_private_key_file set, without relax_encryption=true):

This client helpfully reports more information about unsupported Security Type, "VNC Password", presumably this is referring to either "VNC Authentication" or maybe "VeNCrypt Plain subtype". Although, the app only asks for "VNC Password", and has no "username" field when using this auth type.

Screens 4 App log message:

The authentication method (VNC Password) is not supported by the remote computer.

wayvnc --gpu --performance --log-level=debug output:

Info: New client connection from localhost: 0x556d54bc6690 (ref 1)
Info: Client 0x556d54bc6690 (1) hung up
Info: Closing client connection 0x556d54bc6690: ref 0

Same results when I tried relax_encryption=true with rsa_private_key_file, whether I tried "Authentication type" of "VNC Password" and "None".

Finally, this app has an idea of Apple's macOS User + Password authentication. Trying that with relax_encryption=true and RSA-AES same as above did not work either. Very similar "New client" -> "Client .. hung up" -> "Closing client connection ..." logs for each test. This app did not cause wayvnc to crash.

[trinitronx]

Sorry for the noise, but just had another thought... Given that last app's mention of None Security Type, maybe we should add that to this feature request? It could be even simpler to implement as a first-pass for adding support for these kinds of VNC clients. 🤷

[any1]

The "None" security type is what you get if authentication is disabled.

[trinitronx]

The "None" security type is what you get if authentication is disabled.

Got it! 🙏 Thanks... just tested with just enable_auth=false in ~/.config/wayvnc/config. Tested Screens 4 or bVNC and these work! 🎉

Yet using Screen Sharing.app, same as before:

Unable to communicate with “127.0.0.1:7900”. Make sure the remote computer is available and the firewall is not blocking screen sharing.

And log:

Info: New client connection from localhost: 0x562977886690 (ref 1)
DEBUG: ../neatvnc/src/server.c: 183: close_after_write(0x562977886690): ref 1
Info: Closing client connection 0x562977886690: ref 0

[any1]

I pushed a fix for type 30 to neatvnc master, so it should at least work with bVNC now.

[any1]

It looks like the "Screen Sharing App" might not like something that it sees. It might be interesting to look at this in wireshark. I don't have a mac and haven't used one since the noughties, so I can't really help you there, but if you record the network traffic I'd be happy to take a look.

[trinitronx]

I pushed a fix for type 30 to neatvnc master, so it should at least work with bVNC now.

Thanks! 🙏 I was able to test with bVNC against neatvnc 0.8_pre.git.0e262c8f33c and wayvnc 0.8_pre.git.a53ff1f7698, and it worked! 🎉

I don't have a mac and haven't used one since the noughties, so I can't really help you there, but if you record the network traffic I'd be happy to take a look.

I'd be happy to help with testing this! I'll have to get a pcap file once I'm back at home, given that I'm testing via SSH tunnel at the moment remotely. It'd be easier to see what's going on in Wireshark without SSH encryption in the mix too.

[any1]

Well, traffic on localhost:7900 won't be encrypted, but I'm off to bed anyhow. Thanks for testing!

[Lancelotbronner]

Screen Sharing.app still cannot connect, my use case is connecting to a Raspberry Pi 5.
Tried with enable_auth both true and false but nothing changed, still the same error as higher up in this thread.

[any1]

Ahh, could it be that it just wants an older version of the RFB protocol?

Edit: That could very well be the case. See: any1/neatvnc#48

[cRoCx]

I also want to get back to the OP's initial request.
To me his idea seems to offer the best middle ground of security and compatibility, especially if #284 and #285 would be fixed to limit access to certain networks (localhost, VPN subnet, local network subnet, etc.)
Also rate limiting like in SSH and/or fail2ban helps with the problem of any 8 character limitation, i.e. if you have only 10 attempts per hour. Note to myself: Check if fail2ban works with wayvnc.
There are also server administration tools out there, which rely on the "standard vnc auth" aka unencrypted password, like Apache Guacamole or noVNC. But they offer their own encryption via TLS and are designed to be installed in private networks anyway.
And if you tunnel traffic through VPN (Wireguard, OpenVPN, ...), then you're good to go anyway.

[any1]

I'm rather apprehensive about fail2ban. I wouldn't trust it myself.

The problem here seems to be that Screen Sharing.app wants an older protocol version that Neat VNC doesn't support.

[s-aversa-fd]

There are also server administration tools out there, which rely on the "standard vnc auth" aka unencrypted password, like Apache Guacamole or noVNC. But they offer their own encryption via TLS and are designed to be installed in private networks anyway. And if you tunnel traffic through VPN (Wireguard, OpenVPN, ...), then you're good to go anyway.

I agree with @cRoCx : I use Apache Guacamole to access some devices exclusively using an OpenVPN tunnel and, at the moment, I can't use devices installed with raspbian+wayvnc .

It would be helpful to have the standard VNC auth implemented in wayvnc.

[any1]

I use Apache Guacamole to access some devices exclusively using an OpenVPN tunnel and, at the moment, I can't use devices installed with raspbian+wayvnc .

You can turn off authentication completely.

Edit: I looked at guacamole's source and found out that it's using libvncclient, so you can use security types supported by libvncclient. This includes VeNCrypt and Apple's diffie-hellman based authentication type 30, which are both supported by wayvnc. See: https://github.com/LibVNC/libvncserver?tab=readme-ov-file#security-types

[s-aversa-fd]

Edit: I looked at guacamole's source and found out that it's using libvncclient, so you can use security types supported by libvncclient. This includes VeNCrypt and Apple's diffie-hellman based authentication type 30, which are both supported by wayvnc. See: https://github.com/LibVNC/libvncserver?tab=readme-ov-file#security-types

Interesting, I'll take a look.

Thanks