Skip to content

Improve the speed of plot-ts#105

Merged
lispandfound merged 16 commits intomasterfrom
limited_plot_ts_fix
Jul 10, 2025
Merged

Improve the speed of plot-ts#105
lispandfound merged 16 commits intomasterfrom
limited_plot_ts_fix

Conversation

@lispandfound
Copy link
Contributor

This PR improves the speed of plot-ts to reduce the memory usage and time this script takes to run. Average frame generation speeds have reduced from ~5 seconds per frame to 10 frames per second on Hypocentre, a 50x speedup.

The main achievements to do this are:

  1. Properly parallelise the main loop without copying as much between threads.
  2. Making memmap access patterns more efficient.
  3. Introducing a downsampling option for the waveform mesh.

Copilot AI review requested due to automatic review settings June 22, 2025 23:53
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary of Changes

Hello @lispandfound, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request focuses on a major performance overhaul for the plot-ts script, aiming to significantly reduce both memory consumption and execution time when generating ground motion animations. The changes involve optimizing data access patterns, improving parallel processing efficiency, and introducing a new downsampling capability to provide faster rendering options.

Highlights

  • Performance Optimization: The core optimization involves a significant shift in how ground motion data is handled. Instead of pre-calculating and loading the entire ground_motion_magnitude array into memory, individual frames are now loaded on demand within the render_single_frame function using the new tslice_get helper. This drastically reduces memory usage and improves overall processing speed, especially for large datasets, contributing to the reported 50x speedup.
  • Parallelization Efficiency: By passing the xyts_file_path to the render_single_frame function (which is executed in parallel), rather than a large NumPy array, the amount of data copied between processes is significantly reduced. This change directly addresses the goal of properly parallelizing the main loop without excessive data copying, enhancing the efficiency of multi-threaded rendering.
  • Downsampling Feature: A new downsample option has been introduced, allowing users to specify a downsampling factor for the waveform mesh. This enables rendering frames at a lower resolution, which can further accelerate the animation generation process, particularly when high fidelity is not required or for quick previews.
  • Flexible Frame Control: The animate_low_frequency_mpl_nztm function now accepts a frame_start parameter, enabling users to begin rendering an animation from a specific frame index. This provides greater control and flexibility for generating partial animations or resuming interrupted processes.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in issue comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR accelerates plot-ts by parallelizing frame rendering, optimizing memmap access, and adding a downsampling option to reduce memory and compute overhead.

  • Added tslice_get to read and downsample individual time slices without precomputing full magnitude arrays.
  • Refactored render_single_frame to accept an XYTS file path and downsample parameter and updated the animation entrypoint.
  • Changed frame‐range handling to support a configurable frame_start.
Comments suppressed due to low confidence (2)

visualisation/plot_ts.py:314

  • Consider adding unit tests for tslice_get to cover both downsample > 1 and default cases to ensure correct slicing and norm computation.
def tslice_get(xyts_file: XYTSFile, index: int, downsample: int = 1) -> np.ndarray:

visualisation/plot_ts.py:589

  • The functools.partial call omits required parameters simple_map, scale, map_quality, and title, which will cause a TypeError when invoking render_single_frame. Include these arguments or provide defaults.
        render_frame = functools.partial(

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request improves the performance and memory usage of plot-ts by optimizing data loading and enabling parallel frame rendering. The introduction of a downsampling option is a great feature. The review focuses on refining these improvements, including optimizing downsampling, exposing the downsample parameter to the command line, and addressing a minor logic issue in the frame rendering loop.

@lispandfound
Copy link
Contributor Author

Jenkins builds are failing because of some stuff that changed in the realisation format recently. Will fix that in a separate PR.

@lispandfound lispandfound merged commit 5e33dcf into master Jul 10, 2025
4 of 6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants