Problematic dependencies

[YoshiRulz]

If it wasn't obvious, these prevent us from moving to .NET 6 (see #1415).

package	targets	purpose / notes
FlatBuffers.Core	?	Unknown (Emulation.Cores). IDK, natt hacked this in for Nyma. Presumably serialisation. switched to NuGet, .NET Standard in 158c897
GongShell	Framework (Win)	UI (EmuHawk). Used in File > Recent ROM > <a file> > Shell Context Menu (right-click file to show submenu). removed in 98a8cdf
NLua	Framework (Win)	Lua (Client.Common, EmuHawk). Needs to be loaded at runtime for either Lua engine to function. See #2260. switched to Standard-compatible engine in 45fbdb4
OpenTK v3.3.0	Framework (Win, Linux)	Host graphics (BizwareGL, EmuHawk), host input/sound (EmuHawk). OpenTK 4 is now available, though not for Standard 2.0. replaced in 78f5e75
OpenTK.GLControl v3.1.0	Framework (Win, Linux)	Host graphics (EmuHawk). IIRC natt said this isn't in OpenTK 4. replaced in 78f5e75
SlimDX	Framework (Win)	Host graphics (EmuHawk), host input/sound (EmuHawk). replaced in 78f5e75
System.Data.SQLite	Framework (Win)	DB manip (Client.Common). Probably very easy to replace w/ NuGet package. replaced in bb96825
System.Drawing.Common	Framework (Win, Linux), .NET 6 (Win)	Host graphics (EmuHawk). (Partial) Linux implementation was deprecated, need to switch any drawn controls to Microsoft.Maui.Graphics. Point et al. but not Icon will remain available via a separate package.
System.Windows.Forms	Framework (Win, Linux), .NET 6 (Win)	UI (WinForms.Controls, EmuHawk). If we want to change UI framework after .NET 6, Linux will need to stay on .NET Framework and have separate "binaries" (it mostly does already).

see also #2471 re: OpenTK 3.x

There are a couple of other libraries that I haven't mentioned:

• In addition to the above, EmuHawk's dependency on Microsoft.VisualBasic is only for the type WindowsFormsApplicationBase, inherited by EmuHawk.Program.SingleInstanceController. There's no compatibility issues but it does add a runtime dep that's annoying to install on Linux. fixed for Linux in f6dde59, <PackageReference/> removed in fa07dc8
• System.Web is only used in DiscoHawk, by static method DiscoHawk.GetExeDirectoryAbsolute removed in 681b564
• We also have the option of replacing Newtonsoft.Json with System.Text.Json at any time. The more I learn about Newtonsoft.Json, the more I think we should replace it as soon as possible. See docs supplement. As of 2021-07, System.Text.Json uses Source Generators instead of reflection.
• We may be able to replace some external methods with CsWin32.
• There's this shim for the new (well, "new") NativeLibrary API, which we may be able to replace DynamicLibraryImportResolver with.
• The P/Invoke source generator may only require new SDK and attribute polyfill (like [Nullable]).
• Everything else is working fine with no issues. Some may have newer versions on NuGet.

[YoshiRulz]

So I had a play with this thing (the Windows version works in Mono, who would have guessed) and the situation doesn't look too bad. GongShell, OpenTK 3, and SlimDX all reference System.Windows.Forms, but should work in .NET 6 on Windows. IIRC natt saw that in practice when testing the hammer-and-duct-tape branch. and don't seem to work in .NET 6 on Windows.

In other news, .NET 6 doesn't include WinForms for Linux, even though Mono currently supports it, and they're instead pushing ahead with MAUI (a.k.a. Xamarin.Forms). Given that, my plans for the Linux port are as follows:

• Split the binaries. This is basically done in the hammer-and-duct-tape branch. In effect, the current EmuHawk project becomes EmuHawkMono and a new EmuHawk which targets .NET 6 is created. In practice, the *Mono project files are new, and minimal changes are made to the existing project files.
• Keep OpenTK 3. I think we can use OpenTK 4 in EmuHawk and OpenTK 3 in EmuHawkMono with a bit of abstraction effort.
  • I'm assuming that OpenTK 3 and/or SlimDX works in .NET 6, otherwise OpenTK 4 obviously needs to happen before EmuHawk is retargeted.
  • The OpenTK project has since decided that gradual migration is dumb, nobody would want that, and dropped .NET Standard 2.0 -_-
• Do nothing. The Linux port can keep the ugly-but-functional status quo for a couple of years maybe, until Mono starts being dropped from distros.
  • I'm building Mono from source now, so it can be forever.
• Get tired of doing nothing. WinForms might not be getting removed from .NET, but it's also not getting any better. At the same time, all our library projects are stuck on Standard 2.0, unable to use some of the C# 8.0 through 10.0 language features. To reunify the binaries, we'll first need to decide if we want to switch to MAUI or to a third-party option (I'm aware of Avalonia, Dear ImGui, Eto, and Uno), and then focus on the one framework instead of dividing our efforts.
  • Once the Linux front-end is replaced with one that works on .NET 6, every project can target .NET 6. But, IMO, we should create a new front-end with the expectation of using it on Windows too as having two front-ends is just more maintenance work. Especially when one of them is WinForms.

edit 2022-08-12: I've just been made aware of the ActiveQt library/framework which appears to require choice of Managed C++ or COM? Not sure if Mono supports either.
edit 2024-03-21: Found this thing on HN which I think is ImGui but with vectors? https://github.com/rive-app/rive-sharp

[Sappharad]

• Get tired of doing nothing. WinForms might not be getting removed from .NET, but it's also not getting any better. At the same time, all our library projects are stuck on Standard 2.0, unable to use some of the C# 8.0 and 9.0 language features. To reunify the binaries, we'll first need to decide if we want to switch to MAUI or to a third-party option (I'm only aware of Dear ImGui), and then focus on the one framework instead of dividing our efforts.

The other third-party option that was recommended to me several years ago was Eto, which translates to native options on their respective operating systems (WinForms, WPF, GTK, Cocoa):
https://github.com/picoe/Eto
The API very much resembles that of WinForms, which made it an easy choice back then. Although it seems Eto is still getting active commits, I'm not necessarily recommending it but rather just issuing a reminder that it exists.

I had started a crappy recreation of the WinForms UI in it several years ago, only to abandon it because I didn't want to spend the time necessary to recreate everything and moved back to WinForms before mostly suspending the macOS port entirely.
https://github.com/Sappharad/BizHawk/tree/EtoHawk/BizHawk.Client.EtoHawk
(That last commit from 2018 is actually a problem I ran into later with WinForms and ended up solving, but I don't think it affected Windows.)

[YoshiRulz]

SharpCompress cannot be upgraded past 0.26.0—currently tracking as adamhathcock/sharpcompress#604. fixed in ae91358, I was overcomplicating it

---

tracking opentk/opentk#937 re: OTK3 keyboard init heisenbug; my OpenTK watchlist

[leo60228]

https://github.com/emclient/mac-playground has a port of Mono's WinForms to netstandard2.1. I'm not clear on whether it's compatible with Linux or just macOS, though.

[Sappharad]

That is just for macOS. It’s what I was using for the Mac port after Apple dropped 32-bit support. It won’t help you on Linux

[leo60228]

OpenTK.GLControl was renamed to OpenTK.WinForms, which is in beta and supports netcoreapp3.1: https://www.nuget.org/packages/OpenTK.WinForms

[jdpurcell]

Potential SlimDX replacement, targets .NET Standard 2.0:
https://github.com/amerkoleci/Vortice.Windows

Supports Direct3D9, DirectInput, XInput, and XAudio2 among others, but not DirectSound. A few years ago I tried out the XAudio2 part and got it working in my forked repo:
https://github.com/jdpurcell/BizHawk/blob/35ef85936666dbfdc1d6ad6bf87251f95cb51dbd/BizHawk.Client.EmuHawk/Sound/Output/XAudio2SoundOutput.cs

[YoshiRulz]

Vortice.XAudio2 doesn't seem to have a method for enumerating devices, i.e. to replace this:

BizHawk/src/BizHawk.Bizware.DirectX/XAudio2SoundOutput.cs

Line 27 in c005612

for (int i = 0, l = _device.DeviceCount; i < l; i++)

I also noticed SharpAudio in its description which encapsulates both XAudio2 and OpenAL, but that library is .NET Standard 2.1+ (see feliwir/SharpAudio#44).
edit: and adopted Vortice recently, so the latest beta on NuGet doesn't include it
edit edit: forked it, this branch is 2 tiny commits on top of the latest beta (not on NuGet) which add .NET Standard 2.0 support, and this branch has a Nix expression which should be able to build any future beta/release w/ the same patches.

edit 2023-04: Vortice.Win32 is not suitable as a replacement for the external methods in BizHawk.Common/Win32.

[jdpurcell]

I believe that's because Vortice uses XAudio2 2.9/2.8 whereas SlimDX uses XAudio2 2.7. Microsoft stated in the changes for XAudio2 2.8:

Device enumeration functionality has been removed from XAudio2; the GetDeviceDetails and GetDeviceCount functions have been removed from the IXAudio2 interface. Apps that want to render to other audio devices on the system must pass a device identifier string to CreateMasteringVoice instead of a device index. The default audio render device can still be created without enumeration.

They don't make it clear how to get the "device identifier string" but I found a discussion where someone mentions Windows.Devices.Enumeration.DeviceInformation.FindAllAsync. I didn't want to mess with it anymore after that 😋.

I think Direct3D9 might be the bigger problem though - I did start trying to convert some of that code out of curiosity, and although I'm not very familiar with Direct3D, I'm pretty sure Vortice's implementation of D3D9 is incomplete. My impression is that the author would be unwilling to fix that himself, although would accept pull requests if someone wanted to finish the missing features.

[YoshiRulz]

I have a feeling we stumbled onto the same social.MSDN thread (login required)... edit: found another C++ code snippet that doesn't require login

We won't have any problems w/ D3D9 if we switch to its D3D12 backend instead. (And if we use Vortice via Veldrid then Vulkan is also available. Verily.)

[YoshiRulz]

This seems a good place to put this—.NET Framework 4.8.1 is real and available now, but only for Win10+, and it only includes the promised accessibility changes, not .NET Standard 2.1 support or anything "new". I don't think we should upgrade, and I'm not even going to bother checking if Mono runtime supports it.

[CasualPokePlayer]

Something to note, OpenTK.Input 4 is actually netstandard 2.0 (the rest of OpenTK is not), so us using it for host input would be (mostly) a non-blocker (host graphics/sound is obviously still an issue).

[YoshiRulz]

There was already an option for input on Linux, evdev-sharp, which would have involved building from source. But while we're on the subject, allow me to quote myself:

OpenTK 4 and 5 are .NET Core only, though IIRC OTK 4 was .NET Standard until late in beta, maybe it can still work just by adding the target and building from source?

[CasualPokePlayer]

Regardless, using OpenTK.Input means no building from source, the latest nuget package is already netstandard2.0 compatible.

If we want another input backend, something to look into with Silk.NET is its SDL2 Input bindings, which may be nice in any case due to SDL2 supporting a ton of different controllers.

[CasualPokePlayer]

Doing some work on the input, it seems more complicated when I thought. OpenTK.Input in 4.x is not the same as OpenTK.Input in 3.x, as it is supposed to be some "Hid/Controller" code (in reality, there's barely anything in there and looks to be just incomplete all in all). The old OpenTK input code was moved to OpenTK.Windowing.GraphicsLibraryFramework, and has a significantly different API. Differently enough that I'm not sure we can use it anymore. The main issue is simply that it requires that an OpenTK NativeWindow be created, and that window be focused, in order for it to obtain any input. This obviously is a bit not good, as obviously the MainForm is what we'd want focused (and since it has to be an OpenTK NativeWindow, we can't give it the MainForm anyways), and we would lose the ability to have input without focus like with OpenTK 3.x (which is probably important too for the context of controller dialog, where that might not have MainForm neccessarily focused in this context? maybe not, not sure).

(sidenote: OpenTK 4.x internally just uses GLFW to handle getting input in a platform agnostic way, while OpenTK 3.x just used its own platform abstractions to native APIs)

SDL2 similarly has this issue, but it does have an option to create a window from an existing window using a native window handle (which is handed over in the first place by our input API). Even then however, that doesn't work entirely, at least on Windows (maybe on Linux too, dunno), as for whatever reason as WM_KEYDOWN/WM_KEYUP just never arrive, despite the MainForm clearly having focus, so keyboard input just doesn't work at all.

Not entirely sure what'd be a good solution here. We could use evdev-sharp as suggested, but since that's just evdev that will only work for Linux, and not any future macOS/whatever port and would mean Windows users are stuck with DirectInput+XInput (which is fine mostly, but it is limiting and options like SDL2 would provide more options in this respect). (Not that this doesn't mean evdev-sharp shouldn't be a future option, but a catch-all works on all OSes would be beneficial for us to have first).

[YoshiRulz]

a catch-all works on all OSes would be beneficial for us to have first

I'm increasingly souring on this position. macOS all but precludes that anyway (as does Windows UWP I believe).

[CasualPokePlayer]

Continuing to look and it really just seems like we will forced to just write OS specific code for each platform for keyboard input. We can have SDL2 at least maybe as a nice catch all for gamepads (doesn't have dumb focused window requirement), but that's it. At least it seems possible across all platforms.

I currently envision this for input:

Windows: DirectInput+XInput, DirectInput+SDL2
Linux: evdev, evdev+SDL2
macOS: Quartz+SDL2

On that, this isn't perfect. We may need to change the keyboard from using DirectInput, for UWP support, as UWP does not support DirectInput, the only alternative here seems to be using the RAWINPUT API (as a bonus, we can just make DirectInput back to no longer a hard dependency this way, as the removal of OpenTK would implicitly do for Windows users). Linux doesn't necessarily have to use evdev for keyboards, X11 and libinput can work, although evdev probably would be better as a nice catch all for Linux (and BSDs?). macOS lol find someone who cares enough for some OS specific macOS code, otherwise that would stay with SDL2 for gamepad support (and I'll write up the keyboard support here).

On another note here, I don't like how the main form handle is passed on to input APIs, especially with our move to a new gui backend not making that a possibly not particularly possible idea (it is predicated on able to obtain that handle and being able to hook onto the window proc for reiniting gamepads for directinput), it would function the same if it created a hidden window internally (nb: also this would be needed for RAWINPUT use, as it responds to WM_INPUT messages), and that would be easy enough to message pump that new native window on some new thread...

EDIT: RAWINPUT handler is done and used with the SDL2 backend (also DirectInput actually wraps around RAWINPUT internally anyways, lol)

[YoshiRulz]

Possibly not the right thread for this:
https://github.com/meziantou/Meziantou.Polyfill is similar to PolySharp, but the list of backports is different. Most relevant is that it includes some extension methods. (PolySharp seems to eschew methods added to existing types—which is fair enough, since without partial static classes, the call must be to a nonstandard type when polyfilled i.e. not binary-compatible. But we needn't care about that for our internal libraries, only whether the combined app works.)
Naively throwing it in results in a build failure because both trigger for the same polyfills (I'm pretty sure they run in series and the second could detect the first...), but both are also configurable. If it can reduce the amount of backports we have to maintain then I say it's worth a try.