Per-world threading

[dobegor]

Is there actually any sense doing per-world threading?
I.e. my server's runtime is spent mostly on dim0 ticking of tile entities, and updating a few chunks in nether and ender isn't a game changer.

The problem is the high number of Entities/TileEntities in dim0 being updated in a single thread. And that's really the most common problem over all minecraft servers in the wild, especially modded ones.

That's true that implementing per-world threading is much easier and will create less bugs, but I don't think it will raise the performance as high as much efforts you will put in developing this.

Please, consider developing a true multithreaded tick system. For example, at first you can provide a way to mods to perform updateAsync() instead of update(), as an api. So the problem of synchronizing mod's internal stuff would be solved by mod author.

It's all about your time/result coeffiecent. Minecraft really needs real multithreaded logic today.

[LunNova]

Is there actually any sense doing per-world threading?

Yes. It's a precursor to more invasive threading, and definitely does improve performance enough that it's worth it even on servers which use only the vanilla dimensions.

The concurrency model in previous TT versions is described here: https://github.com/nallar/TickThreading/blob/1.6.4/IMPLEMENTATION.md

Please, consider developing a true multithreaded tick system. For example, at first you can provide a way to mods to perform updateAsync() instead of update(), as an api. So the problem of synchronizing mod's internal stuff would be solved by mod author.

Doesn't seem like a good idea. Mod authors aren't going to bother to do it, and the APIs required for fully parallel updating of TEs just aren't there. For example, none of the current inventory APIs aren't designed to be threaded. There's no concept of a tryRemoveItem(Item, count) : ItemStack, just checking which items are in which slots and mucking with counts.

I resolved this as discussed in the document linked above by mostly ensuring that TE/entity updates didn't need to do any synchronisation through the TickRegion system. Unfortunately there are many TEs which like to do more complex actions such as interacting with far away parts of the world, or interacting with per-world or static data which isn't thread-safe. These all require patching.

[dobegor]

Well, I understand these reasons now. Perhaps it's a good idea to leave this issue open if people will come up with some ideas about concurrency model.

Thanks for a good answer and your work!

[TheElectronWill]

As nallar said, a true multithreaded system would require an adapted API and/or a lot of entity patching, so it's hard for a mod to do this. But it can be done by an entirely new program, and that's the goal of Photon :)

[magneticflux-]

Perhaps a hybrid between entity locking and the Sudoku-board-like divisions detailed here would be optimal? Threads/TEs would never have contention accessing adjacent chunks since they're spaced apart, and per-chunk locking would eliminate issues of mods accessing large/distant areas.

[LunNova]

That's interesting. Factorio's system is quite similar to the TickRegion system TT used to use, although their's works by updating all in a certain grid position at once, meaning there's a predetermined order of which regions are updated. TT updated regions in any order, but avoided updating adjacent regions at the same time.

I'm not sure which approach is better, but I suspect that a subdivided grid is suboptimal especially in modded MC where it's not uncommon to have single very slow entities eg TileController from AE.

If a single 1 grid takes far longer to update than all the others, no 2...9 grid blocks can be updated until it finishes. With the adjacent region locking (but non-blocking locking) approach, a single slow entity will only block up the immediately adjacent regions.

edit: I suspect factorio is limited to using a more deterministic update order with that grid system as their multiplayer system depends on as little network traffic as possible and everything being kept perfectly in sync on clients. In MC lots of stuff is only handled server side and not kept track of by clients, eg state of items in machines is only received when a gui is opened.

[magneticflux-]

You're probably right. I didn't know that there was already a region system tried. Factorio is significantly more deterministic, and it's a lot more efficient because of it. I doubt that would ever be possible in Minecraft though since there are so many random elements. Profiling which chunks are "touched" at a time might give enough insight to devise an efficient threading system, or even an adaptive one that can accommodate monolithic entities as well as vast interlocking redstone/etc.

[cpypcy]

Factorio is going apeshit crazy in 0.15 they are going to do whole game update system multithreaded! I can't wait to see the results. Maybe you can get some tips on forum, or maybe they were inspired by your mod a bit? Who knows?

[magneticflux-]

While Factorio is no doubt a shining example of how to build an efficient multithreaded game engine, I believe that the most difficult problems are unique to TickThreading. Notably, not being able to rely on mods to supply appropriate synchronization themselves and having to handle situations impossible or extremely unlikely on more controlled systems. I also don't think that Factorio's Lua modding is multithreaded, instead it just execute's each mod linearly.
Edit: Coincidentally, I encountered nallar himself on the Factorio subreddit while Googling for more information about modded multithreading.