Improvements 3D, AutoConfigurations(Res,Fps,Bitrate), DualScreenDeviceSupport #437
Conversation
|
Please revert the resolution settings in external display mode part in Game.java, I've mentioned that some display gets the wrong resolution when connected to external displays and causes the image to be crushed, force reading the values and forcing it to stretch mode workarounds the problem. |
found the bug I mentioned on reddit and added the auto configuration setting for external displays :) |
See latest push. People can now even keep render mode 3d active and it will only apply if external 3d enabled device is connected in 3d mode. I like it, thanks for suggesting. Maybe we could also make internal display Auto config with max refresh rate and resolution... either with new pref setting, or even the same. |
|
Thanks for all your work!
That would be the best, but I think maybe add an "auto" option in the resolution list is a better idea than separate checkboxes |
As it would also affect refresh rate what about a auto configuration with a description what it does right above resolution and frame rate picker, which are disabled if auto configuration is on? It would Auto match resolution, frame rate (Bitrate?), render Mode, external display res and refresh rate...so basically plug & play, which 1-2 sentences could help users understand it. |
|
I think the best option is to have auto in both resolution and framerate, while leaving bitrate manual. Additional auto toggle for specific modes can still be confusing. |
Got it, will do. Is it okay to cover the 3d/external auto resolution with the same auto mode, or leave it as separate option? |
|
Hmmm it's a bit hard to design... I'll let you know after I have some thoughts. Also, do you have any idea on how to implement this? The biggest issue is I don't know if 1 pixel interleaving fits these device... |
Hm...the implementation itself should be not too hard and the shader Open gl pipeline we have can be re-used with an interlaced mode. Maybe we could add several interlace patterns to choose from so more than one device works. We could also update the 2d to 3d conversation to work with interlaced displays first... basically it's 2 birds with one stone. |
I have done this now and it works really well, love it. Also added with it a fix to bitrate calculation to get the actual auto detected fps/res for the calculation. Just one minor issue stays, if you connect glasses after you have set it to AutoMatch, the bitrrate is already calculated based on phone screen right...so turning 3d on etc works but bitrate is probably higher than needed as phone screeens tend to be higher in refreshrate + res than glasses. However, manually adjusting it works still. I could add in code an "ifAutoMatch, if onExternal...calculate bitrate here and set prefsConfig.bitrate = ..calculated real time - but that would override everything they might have configured on their own?... |
|
Hmmm... Maybe allow bitrate be 0 and when it's 0, automatically calculate it each time the stream starts? |
That could work... I was even thinking if network quality might then be added to the calculation, but not now...just a thought |
I actually dislike that any change to res or fps automatically changed my Bitrate. I know the limit of my wifi and just set it to 100, done...but whenever I check it was below. |
Yeah, that's my idea too
I was thinking about it, but the more needed situation for this is auto adjust during stream... Which requires quite some changes to both the client and host side |
That be amazing indeed. |
Did you know that when you put 0 as bitrate, its just working fine as "unlimited"? |
See latest push... this PR should be ready to go after a review from your side! |
Actually that's incorrect, "unlimited" is limited by the host side, and may be too high for usual network setups. |
|
And there're more problems, like... When using auto fps on phone screen, should we use the current fps or the highest fps? And here's a problem for newer dual screen Android handhelds... |
It was only a question, I didn't use 0 at all. |
Janyger
changed the title
…viewing experience if the FOV is too big
Janyger
changed the title
Regarding your question....it's taking the highest even if adaptive (the signal is 120 but the system renders only 60-120). At least on my phone, only on battery saver mode it picks 60. When someone selects manually 60 in his phone settings, we don't even see 120 |
|
Lots of Chinese phones do weird things. They default at a lower refresh rate and won't always be full refresh rate, unless explicitly requested by the app in some non-standard ways. |
We check the currently connected active signal to the display. There is and never was a way to force change that as far as I know. So we will take the active available option and set it to setFrameRate to the active surface etc as before. setting anything higher than the active won't change anything on the result as we can't change it |
|
When you read the display info, you'll get a list of available modes. There can be modes that have higher refresh rate than the current active one. Like for a device with 165hz display, it might default to 120 or 90 when idle or not actively requested from the app, and only go up to 165 if the app requested. |
Yeah...you might be right. Then for internal main display we could/should potentially pick the highest in that list. Hm, but in battery mode I get 60hz mode, so we would have 120 wrong...ah. I need a test device :D On external screens this setFrameRate does not work at all, unfortunately. :) Id probably leave it as is and user's with 165hz Chinese phone's can still pick 165 hz manually...it's called match, not force highest |
How is it going? Anything you want me to change? Used this build now for quite a while... Would love to get people to use the better 3d rather than the broken experimental build :-) so let me know if I can speed up things |
|
What about add options for AI 3D full width/half width? When in AI 3D full width, request the full resolution, and in half width, request halved width of the active display / selected Should not mix with host side 3D though, so interlaced should be a separate option That leaves
And we keep the generic resolution settings as is |
Good approach, just wondering if I/we can make this a new PR /enhancement afterwards? |
Yes, that's fine I'm really busy recently so I don't have time to fully review this PR yet, sorry for the delay... |
2 participants
Major 3D Rendering Refactor: Overhauled the AI 3D pipeline (Stereo3DRenderer, ShaderUtils, AI tasks) with new shaders (Dilation, optimized Blur), RGB input for AI, frame tracking, and refined depth map smoothing logic using OpenCV (accumulateWeighted), aiming for improved quality and performance. Removed old OpenCV padding code.
Fixed forced Balance mode (60fps) when external display is connected
Improved performance and load on cpu etc in 3d
Auto Detection of 3d glasses and switing to 3dmode/resolution
General AutoMatchDisplayResolution and FPS settings added
Bitrate calculated bitrate setting on stream start based on used fps/resolution when 0 Mbps is
+ Removed automatic change of bitrate when resolution or FPS got changed (confusing to users and requiredtweaking and double checking)
+ Added a toast when hitting bitrate preference, letting the user know the current recommendion based on settings or select 0 for auto match.
Fixed Display Selection on Dual-Screen Devices: Correctly identifies and utilizes the larger internal screen for game streaming and the smaller screen for controls, particularly when using "Fully external display mode", addressing issues on devices like the AYN Thor.
Improved External Display Handling: Refined heuristics (using Manufacturer ID, size, and aspect ratio) to better distinguish between true external displays (like AR glasses) and potentially misflagged secondary internal screens.
Disabled Dimming on Secondary Control Screens: Automatic screen dimming for inactivity is now disabled when the touch controls are active on a non-default display (e.g., the smaller screen of a dual-screen device).
Basically with AutoMatchResolution, AutoMatchFps, AutoMatchBitrate and 3d Mode Glasses detection. Its seamless plug and play for usage on the phone, with AR glasses and switch to a higher resolution external screen which only supports 2d, or switching to 3d on glasses mode doesnt require change of anything anymore. Additionally supports now Nintento3DS like devices.