Improve performance for certain /web/ routes

[mellowagain]

I'm currently measuring execution time of the major parts of Shiro including routes and handlers. Most of the routes and handlers execute in less than 1ms which is already faster than all publicly available osu!Bancho re-implementations. This is good and reaches our expectations and goals very clearly. Here are the logs from the performance regression: shiro 2019-05-18 13:24:39.log

The following handlers however did not perform to the expectations and need to be optimized / redesigned to perform better:

• web::get_scores::handle: If the beatmap is already fetched and in the database, execution time is from 4ms (no scores) up to 22ms (many scores and filtering) which is acceptable. However, for beatmaps which need to be fetched from the osu!api this takes at least 200ms up to even 1s.
• web::submit_score::handle: Consistently takes over 900ms up to 1.1s to execute.

The above two cases need to be optimized for a seamless player experience. Please discuss below ways on how we would go on and optimize these methods.

Machine this performance regression was executed on

Hardware:
CPU: AMD Ryzen 5 1600 (12) @ 3.200GHz
Memory: 16'052MiB (~16 GB) DDR4 2800MHz
Graphics Card: NVIDIA GeForce GTX 1060 6GB
Disk: Shiro was run off an HDD

Software:
OS: Arch Linux
Kernel: Linux Zen 5.0.10-zen1-1-zen
Compiler: Clang 8.0.0
Mode: Debug (unoptimized)
MySQL: 10.3.14-MariaDB

[hazel0177]

This is usually caused by the osu!api being slow, gatari (attempts) to get every map possible and enter it into the database, and is why they're very fast. I see that is being the only possible way to improve speed.

[mellowagain]

Performance for score submission can be improved by implementing caching between score submissions for common values such as the beatmap difficulty (prevents re-calculation of beatmap difficulty).

[mellowagain]

I've benchmarked score submission on my machine and came to the following timings (in seconds, lowest to highest):

0.000000    Ranked/Passed checks
0.000000    User stats refreshing (memory)
0.000001    Stats refresh (memory)
0.000002    Replay exists check
0.000006    Flag checking
0.000012    Field checks
0.000035    Validity checks
0.000040    Table display building
0.000084    Score construction
0.000102    Score decryption to String
0.000664    Score struct construction
0.000822    Multiform parsing
0.000901    PP calculation
0.001040    Searching score in db (dup check)		#   1.040ms
0.001485    Replay saving							#   1.485ms
0.002346    Stats refresh (db)						# 	2.346ms
0.002550    Score overwriting						# 	2.550ms
0.002807    Adding score to db						#   2.807ms
0.003356    Init table display						#   3.356ms
0.007840    #1 bot check in #announce				# 	7.840ms
0.009854    User validity checks					#   9.854ms
0.019503    Beatmap fetch + stats refresh (db)		#  19.503ms
0.240826    PP + Acc recalculation					# 240.826ms
-----------------------------------------------------------------
0.294276	Total									# 294.276ms

Details:

• The beatmap was already fetched previously and was saved on disk.
• Shiro was running off an HDD.
• The database contained 256 scores.
• The database contained 2'526 beatmaps.

As we can see, the most time is taken up by beatmap fetching from db as well as PP and accuracy recalculation.