Browser frames and OBS frames not in sync. Rendering therefore choppy.

[bluelightzero]

The browser is not rendering my web page animation smoothly.

I have no problems running other features in OBS at 60fps smoothly such as webcams, it's not even maxing out any of my cores.

The web page animation is perfectly smooth in my normal browser. The animation is very simple.

I'm just making a guess here, but I think it might be because the browser framerate is not synced up with the rendering framerate so sometimes it skips a frame and sometime it captures the same frame twice, causing the stuttering.

• Fedora 29

• OBS 23.2.1

• linuxbrowser0.6.1

[bluelightzero]

Ok so I've been looking at the code and I think i see the problem.

There are 2 parts to the code; the browser and the plugin.

The browser has a method on line 48 of "src/browser/browser-client.cpp" that sets the data.

obs-linuxbrowser/src/browser/browser-client.cpp

Lines 40 to 51 in b618678

	void BrowserClient::OnPaint(CefRefPtr<CefBrowser> browser, CefRenderHandler::PaintElementType type,
	const CefRenderHandler::RectList& dirtyRects, const void* buffer,
	int vwidth, int vheight)
	{
	// Don't draw popups for now
	if (type == PET_VIEW) {
	pthread_mutex_lock(&data->mutex);
	size_t len = std::min(vwidth * vheight * 4, int(data->width * data->height * 4));
	std::memcpy(&data->data, buffer, len);
	pthread_mutex_unlock(&data->mutex);
	}
	}

The plugin has a method on line 380 of "src/plugin/main.c" that pulls this data

obs-linuxbrowser/src/plugin/main.c

Lines 367 to 386 in b618678

	static void browser_tick(void* vptr, float seconds)
	{
	UNUSED_PARAMETER(seconds);
	struct browser_data* data = vptr;
	pthread_mutex_lock(&data->textureLock);
	if (!data->activeTexture || !obs_source_showing(data->source)) {
	pthread_mutex_unlock(&data->textureLock);
	return;
	}
	lock_browser_manager(data->manager);
	obs_enter_graphics();
	gs_texture_set_image(data->activeTexture, get_browser_manager_data(data->manager),
	data->width * 4, false);
	obs_leave_graphics();
	unlock_browser_manager(data->manager);
	pthread_mutex_unlock(&data->textureLock);
	}

I think that these 2 methods are causing the sync problems. Sometimes the browser is setting the data again before the plugin is able to read and render the last one. Sometimes the plugin reads and renders the data twice before the browser has a chance to update with the new data. This would explain the choppiness I am experiencing.

A potential fix would be to add a queue, so the browser adds to the queue and the plugin removes from it. This way, the plugin will render every frame and won't miss any.

If you need an easy way to perform your own tests, I recommend this video: https://www.youtube.com/watch?v=Cyxixzi2dgQ

[NexAdn]

First of all – thank you for this detailed review. Indeed, these methods aren't synced at all. Unfortunately, CEF required to run in the main thread, which forced us to spawn a separate process for each browser. This implies that we need some ipc channel to exchange the frames, which is currently a shared memory region just as large as required for a single frame plus a few bytes of metadata.

Implementing a queue where the browser pushes items and the plugin pops these items would require a much larger shared memory area as well as a queue implementation in C. Another thing to consider is: What happens, if the browsers framerate is higher than framerate configured in OBS? Using a queue would then cause the output video stream to get slower as for instance the browser provides 60 frames per second to the queue, whereas the plugin only pops 30 frames per second, so one second of video in the browsers becomes two seconds in the plugin/OBS. Also the queue would become very full very fast, consuming more and more memory. The same problems appear if the queue runs solely in the browser process and the pop mechanism works by writing the current frame in the shared memory and reading the frame means setting some shared boolean to true.

I am not sure if implementing a queue is such a good idea. On the other hand I don't see any other working approach at keeping the frames in sync.

But as I am currently quite busy, I don't know if there will be a fix soon. But maybe you can provide a pull request with a fix.

[bluelightzero]

Thank you for your response.

I know that OBS lets you change the framerate for your stream/recording in Settings->Video. The default is 30 fps which I'm assuming is where the plugin gets it. I actually have mine set to 60fps.

Can the browser's frame rate be set to match OBS's setting? If we can't get the framerate of the browser to change, then maybe we can set the browser to simply skip frames consistently to match the target framerate. If it's a nice multiple like 30 into 60 then this should still produce smooth output.

Instead of a queue, 2 frames should be enough. The browser can alternate between which one it sets, and give an incrementing index for each frame. The plugin can read and render the frame with the next index. If the plugin falls behind it will simply have to skip the an index to catch up. In my opinion this is acceptable, even my normal browser skips frames occasionally.

I will have a go at it, and if I make any progress I will provide a pull request.

[AlexDaniel]

Instead of a queue, 2 frames should be enough

Yep.

The state of this plugin is rather sad:

• No proper audio routing (Custom audio routing / Browser sound listed in OBS mixer #15)
• Transparency doesn't work as it should (transparent white appears gray #58)
• Choppy rendering (this issue)

It is kinda forcing me to stream using windows, which I really don't want to do as I have other linux-oriented software that I need to run too. So unfortunate :(