Memory optimisations

[SmCTwelve]

Issue
Process memory accumulating after repeated command execution and seemingly not being freed. Memory usage would spike when processing large data e.g. telemetry and not return to normal after, often leading to over 1GB reserved memory when leaving the bot running for some time.

Investigating with tracemalloc pointed to FastF1 and Pandas with the highest consumption, which makes sense given the data processing involved and that they are primarily intended for single run batch processing rather than a continuous application.

• Mem at start: ~180MB
• Mem after /plot-telemetry: ~550MB
• Mem after multiple commands: ~1GB

Reserve blocks are not freed when the command is complete, nor does it appear to be reused for the next command as additional memory gets allocated on each execution.

Solution
#21

Not much can be done about the core Pandas memory consumption but using memory_profiler I was able to narrow down where in the bot code large chunks were being allocated. Unsurprisingly using fastf1.load_session which involves a lot of Pandas operations; more surprising was matplotlib.pyplot figures which would often allocate over 200MB on each execution and then linger in memory. Researching showed some discussion related to pyplot figures sometimes not being garbage collected.

Replacing plt.subplots with directly instantiating Figure showed less allocated memory used. Calling plt.close() when saving, combined with deleting intermediate DataFrames after processing and forcing collection with gc.collect() after execution shows memory consumption closer to what I expect.

Peak usage can still spike to ~750MB for a few seconds when processing large uncached data, but returns back to the ~200MB range once the command has executed instead of remaining reserved.

Further optimisations
Usage may still slowly creep up over time but not as drastic as before. I'm not sure what else can be optimised that is not related to internal Pandas/FastF1 functions.
Perhaps replacing the use of BytesIO memory buffer in utils.plot_to_file before uploading with a temp file on disk.