Stutter fix

[vbousquet]

This PR contains the change discussed in issue #283

There were 2 problems:

• SharpGL implementation performs rendering in a FBO, then copy the FBO to the native window. This copy was somewhat inefficient, and the code did trigger a GC at each copy leading to poor performance. This is discussed here dwmkerr/sharpgl@c436fef and here FBORenderContextProvider optimization dwmkerr/sharpgl#145.
• The other issue is that the implementation used a fixed rendering loop (timer based at a fixed framerate). The rendering frequency was not the one of the DMD source, leading to animation stuttering. This was not a performance problem but gave bad smoothness and lead to a perceptions of stutter and a feeling of a performance issue.

The PR change this by:

• implementing our own OpenGLControl based on the proposed fix in the previous links,
• modify rendering to be event based instead of render loop based, triggering a render each time a frame is submitted by the DMD source.

This is a follow up from my canceled PR #295 with more testing and cleaned up. It is also on a work branch to allow easy corrections.

[freezy]

Thanks a lot!

Any idea why dwmkerr/sharpgl#121 didn't go upstream? Are there any drawbacks? Doesn't seem to be clear when reading through the issue.

Anyway, due to VPE I might be spending some time to make dmdext cross-platform. Is there any particular reason you've chosen sharpgl over Silk.NET? Silk seems more maintained and cross-platform. @jsm174 already ported your code over at pinmame-silk in case you're interested :)

[vbousquet]

I do not know why dwmkerr/sharpgl#121 was not merged. SharpGL seems a bit out of date and not that well maintained, so it may be just by lack of time or interest of the project maintainer. The fact that the upstream version has a direct call to the GC on the render loop (in BitmapConversion) is also not that positive and let me think that this is not a completely mature library.

I have looked at solutions to move away from it. The core problem is that DMDExt is currently tied to WPF (all the UI) and there are no easy path for efficient rendering with WPF. SharpGL has its drawback, but it offers a working integration with WPF. I do not know of other big library that offer this (Silk.Net does not as far as I know, nor does any of the librayr identified by Khronos at https://www.khronos.org/opengl/wiki/Language_bindings). That's the reason of this choice.

IMHO, the best for the rendering path would be to use a de-facto standard cross-platform windowing library like GLFW, SDL, or one of the others. But this would need to completely refactor the UI code.

[freezy]

@jsm174 is using ImGUI via mellinoe/ImGui.NET, which is cross-platform, so the idea would be to rip out WPF and replace it with ImGUI or similar.

I don't think it's enormous work, the "only" places where UI is involved are the two config dialogs and the actual displays, which for the DMD is just a wrapper around the OpenGL surface. For segment rendering it's probably a bit more work, but we might be able to do it in shaders as well (for VPE it's 100% shaders, but we're getting the post-processing effects basically for free from Unity).

To think about :)

I'll merge the PR later tonight unless you think anything's missing.

[vbousquet]

I think the PR is safe to be merged.

To push it further, I would be happy to help with a WPF free version of DMDExt. I think dropping Skia in favor of what you have done in VPE for segment rendering would be nice too, and would lead to a better/simpler DMDExt.

For the windowing library, I do not have enough experience to point to the right one for DMDExt use case. My feeling is that it is more a question of taste and skills. ImGUI.Net / Veldrid looks nice though.

[jsm174]

@vbousquet, I'm pretty new to anything graphics programming and I initially looked at Veldrid.

After seeing your OpenGL code, I figured taking something known to work would be the easiest path. I won't lie, it took me a while to understand what the shaders and fbos were all doing, but I think I learned a ton. Plus osx has its own challenges with crappy OpenGL support -- I had to convert some of the shaders to 330, and downgrade Silk.NET's ImGui to use OpenGL 3.

Anyway, Silk.NET has a Veldrid extension, but I haven't tried it out. The only other thing I can say is the main developers are beyond nice and extremely responsive on Discord. They worked with me throughout the entire ImGui issue.

[vbousquet]

After a deeper look, my opinion is that going Silk.Net + ImGui would be a reasonnable choice.

• ImGui seems just perfect, largely used, really stable... More or less a de facto standard, and since jsm174 already did the job, why would we go another path ?
• Silk.Net seems also very stable, well supported, backed by the .Net Foundation. So it seems a reasonnable choice. The only drawback I see is that it is a thin layer between the app and OpenGL. So it doesn't add a level of abstraction allowing to switch between OpenGL / Metal / DirectX (like Veldrid or others do) which would be nice since OpenGL is more or less legacy, especially for MacOs, and replaced by Vulkan / Metal (or DirectX for Microsoft platforms).

I think it is really worth the effort since it should at least gives better performance since native windowing will allow to bypass 3 full size processing (FBO reading to memory, WPF upload to GPU, WPF final surface rendering), a full size rendering of the frame, (which can be merged with the DMD rendering) and a GPU/CPU sync (when we read the FBO). I will try to find a bit of free time and initiate something.

Tell me if you think another pair (native windowing+GPU / UI) would be better.

[freezy]

Awesome!

Fine from my side you're good to go. There are a few more Windows-specific things like the native USB drivers, but I'm sure we can work around that too.

[vbousquet]

Just an update on this since it's been quite a while and to coordinate better.
I had a go to Silk.Net + ImGui. After some efforts, it worked but it does not integrate that well with .NetFramework. So I have splited the work in 2 phases to keep things clean:

• First, cleanup and port to netcore
• Second, implements Silk.Net native windowing

Regarding the netcore port. It is going nicely but leads to wide project change (not code):

• move from Reference to PackageReference,
• full manual rewrite of all the project file,
• update all the outdated dependencies to a netcore compatible one,
• update the build process since Fody/Costura and publish works differently under netcore (use publish, with single file publishing for DmdExt, and fody/costura for native embedding in LibDmd).

For the moment, this is going nicely and DmdExt.exe seems ready (needs more testing though). I still have to work on PinMameDevice and the installer (build but untested). I will keep you update and open an early PR when things are nearing stabilization.

[freezy]

This is so great. Thanks a lot, @vbousquet!