Performance issues with many ROMs in browser

[DerekTurtleRoe]

I'm not really sure how you would fix this, since I haven't looked at the code yet, but performance is really bad with many ROMs.

There are hangs and crashes as well, although those are a bit more rare. These mostly occur when trying to stop emulation, or when trying to scroll through ROMs.

I have an Intel Pentium G2030 @ 3GHz, 16GB of RAM, an Nvidia GTX 750 Ti, and a Samsung 850 EVO SSD. Using CEN64 git and CEN64-Qt git.

I have about 3500 ROMs, so not a typical use scenario. But probably one you should prepare for, since the best way to test a cycle-accurate emulator is with games and demos. 😉

Do you know how to tackle this issue @dh4 ?

[dh4]

I played around some with this on my Core 2 Duo machine. I hadn't bothered to test performance with any more than 700 or so ROMs before since I figured no one would have a collection larger than that (~ total number of commercial games released between Japan, USA, and Europe).

With 1000 ROMs, performance is still reasonable. The views each take a few seconds to load, but once they do, they scroll fine and are perfectly usable.

With 4000 ROMs, the views are basically unusable with images enabled. If I disable images, the table view still has good performance so this leads me to believe it's mainly that Qt can't handle painting so many large QPixmap images. I initially thought it might be because CEN64-Qt scales the source images to the correct size on the fly while building the view rather than storing a bunch of different size images of the box art within the cache to pull from, though this doesn't seem to matter, only the size of the original source image it is scaling. If I disable scaling, performance appears to be the same, but if I downsize the size of the source images within the cache, performance improves slightly. Even with images disabled completely though, the grid and list views don't perform greatly (though significantly better).

An easy option would be to disable images as well as the grid and possibly list view if someone has a collection greater than 1000 ROMs. My assumption would be they would use the table view anyways with that large of a collection since it has superior sorting capabilities. This is what I would suggest doing in the meantime to make it usable anyways.

When I get some time, I'll test it more thoroughly and see what optimizations I can make to improve performance with such a large collection, though it may not be possible to get amazing performance out of the grid and list views without completely reworking how they are rendered using sometime like a QGraphicsScene.

[DerekTurtleRoe]

@dh4 Let me give you a screenshot of my settings, I think that might give you an idea of my setup:

I have an Intel Pentium G2030 running at 3GHz, 16GB of DDR3 RAM, and an Nvidia GTX 750 Ti graphics card. I am also running 64-bit Windows 10 Pro.

The thing is, I really don't have any images to load. 😖

Table view is the only somewhat speedy option, and even that hangs sometimes.

The application as a whole seems to slow down a little if I have a lot of ROMs, does it do anything in the background that would slow it down?

EDIT: I changed the columns and image size back to defaults since they don't affect the performance at all...

@dh4 I don't know much about QT specifically, but does it offer a "virtualization mode" for collection-based controls? That seems pretty standard with UI toolkits, and would definitely improve performance in this scenario.

Basically in virtual mode, only the items on screen are allocated in memory. This makes the view render almost instantly regardless of the size of the set (since only the first x amount are processed). This approach usually utilizes caching so once an item is created, it's stored away in memory for repeat scrolls.

I had ran into this problem when building my own ROM browser for my personal projects, and after enabling virtualization the controls loaded immediately and were very responsive.

Just a thought 😄

[DerekTurtleRoe]

@beannaich Interesting. I know psxact doesn't have a ROM browser, so you must be talking about Nintemulator? 😄

@dh4 Are you by chance using QListView?

[dh4]

@beannaich Thanks. The equivalent of this in Qt would probably be the view/model architecture that decouples the data from the widget. I originally used a QTreeWidget for the table view when first coding the application because it was simpler and the scope of this was small, though a QTreeView would probably make more sense at this point and perform better with large a large collection. This would only affect the table view.

@vgturtle127 It uses a QTreeWidget for the table view.

[DerekTurtleRoe]

@dh4 Oh, OK. I was thinking you could switch to QTableView for possibly better performance, but if you aren't using that there is most likely not going to be a performance increase.

Well, I suppose I could just move small amounts of ROMs into a test folder. 😆

[dh4]

Switching to a QTreeView would probably result in a decent speedup if you're experiencing issues with the table view, though would require a good bit of refactoring at this point. I'll work on it if I get a chance.

[DerekTurtleRoe]

@dh4 Take your time, no need to rush. It isn't a big issue with most typical scenarios and people.

Hopefully, most people just will play a few games so most people won't see an issue. I like to test emulators, though, so I think people like me may want something a little more speedy. 😉

[dh4]

Could you build the model/view-refactor branch and tell me if performance is improved with the table view? With ~4000 ROMs loaded for me it still takes a couple seconds to load the view, but once the view is loaded the scrolling and selecting is as responsive as if there are only a couple ROMs.

The above branch isn't a full implementation, so it will only show Filename and Size for the columns and won't let you launch ROMs, but should give an idea of if this is the correct direction to go before I implement it further.

[DerekTurtleRoe]

@dh4 Let me see if I can setup my development environment and all that. I may try using the Linux subsystem for Windows.

[dh4]

There's a Windows build here:
https://s3.amazonaws.com/dh4/cen64-qt/model/view-refactor/cen64-qt_win_git-latest.zip

[DerekTurtleRoe]

@dh4 Sorry for the long delay, I have been busy. I am going to compile newer versions of both and re-test with all my ROMs.

[dh4]

@vgturtle127 Did the changes result in a noticeable performance increase?